Channels
#flyte-deployment
Title
# flyte-deployment
c
Cody Scandore
09/12/2023, 6:50 PMDoes the flyte-binary helm chart come with support for scheduling / ~cron out of the box? Or are there additional components that need to be deployed?
d
David Espejo (he/him)
09/12/2023, 7:49 PMI don't think so @Cody Scandore
I mean, the scheduler is a native component of Flyte, regardless of the deployment method.
Adding a schedule to your Launch plan and activating it, should be enough: https://docs.flyte.org/projects/cookbook/en/latest/getting_started/run_schedule.html#scheduling-a-launch-plan
c
Cody Scandore
09/12/2023, 7:50 PMThanks @David Espejo (he/him)..
Hm. It appears as active in the console UI, but fetching the lp through flytectl is not encouraging..
Copy code
-latest
-------------------------- ----------------------------------- ------ ------- ---------- -------- ---------
| VERSION | NAME | TYPE | STATE | SCHEDULE | INPUTS | OUTPUTS |
-------------------------- ----------------------------------- ------ ------- ---------- -------- ---------
| 9xlF6CkNO1rB46vBvlGNuA== | schedule_test.print_time_workflow | | | | | o0 |
-------------------------- ----------------------------------- ------ ------- ---------- -------- ---------
1 rowsimage.png
d
David Espejo (he/him)
09/12/2023, 7:52 PMcan you share the launch plan definition?
c
Cody Scandore
09/12/2023, 7:53 PM Copy code
cron_lp_every_min_of_hour = LaunchPlan.get_or_create(
name="my_cron_scheduled_lp",
workflow=print_time_workflow,
schedule=CronSchedule(schedule="*/5 * * * *")
)d
David Espejo (he/him)
09/12/2023, 7:56 PMand did you activated it?
c
Cody Scandore
09/12/2023, 7:56 PMyes, through flytectl. That's when it appeared in the UI
d
David Espejo (he/him)
09/12/2023, 8:00 PMand what's the symptom. it hasn't been executed?
anything interesting on the logs?
c
Cody Scandore
09/12/2023, 8:03 PMYes, there's no executions kicking off. I didnt' see anything in the flyte-binary pod
Anywhere else I can check?
k
Ketan (kumare3)
09/13/2023, 12:50 AM
It does come with scheduling
You just have to activate the schedule
I am working on a PR that will make this nicer
c
Cody Scandore
09/13/2023, 1:23 AMKetan (kumare3) [8:50 PM]
You just have to activate the scheduleIs there anything in addition to registering the launchplan and then
flytectl update launchplan ... --activatek
Ketan (kumare3)
09/13/2023, 3:03 AM
No
c
Cody Scandore
09/13/2023, 2:15 PMIs this helpful at all?
Copy code
2023/09/13 14:14:31 /go/pkg/mod/gorm.io/gorm@v1.24.1-0.20221019064659-5dd2bb482755/callbacks.go:134
[0.946ms] [rows:0] SELECT * FROM "schedulable_entities"
{"json":{"src":"composite_workqueue.go:88"},"level":"debug","msg":"Subqueue handler batch round","ts":"2023-09-13T14:14:32Z"}
{"json":{"src":"composite_workqueue.go:98"},"level":"debug","msg":"Dynamically configured batch size [-1]","ts":"2023-09-13T14:14:32Z"}
{"json":{"src":"composite_workqueue.go:129"},"level":"debug","msg":"Exiting SubQueue handler batch round","ts":"2023-09-13T14:14:32Z"}
{"json":{"src":"composite_workqueue.go:88"},"level":"debug","msg":"Subqueue handler batch round","ts":"2023-09-13T14:14:33Z"}
{"json":{"src":"composite_workqueue.go:98"},"level":"debug","msg":"Dynamically configured batch size [-1]","ts":"2023-09-13T14:14:33Z"}Inside the db, there is a table
launch_plansschedulable_entitiesIs there some way to inspect the running processes inside the
flyte-binaryroot 7 0.6 8.1 2079380 160480 ? Sl 15:41 2:45 /usr/local/bin/flyte start --config /etc/flyte/config.d/*.yamlflyte.admin.disableScheduler: falsek
Ketan (kumare3)
09/13/2023, 11:29 PM
there should be only one process
everything runs as a single process (go program) this is why flyte binary is small and light
i can try and run it locally
c
Cody Scandore
09/13/2023, 11:33 PMThe workflow or the binary? That would be fantastic
k
Ketan (kumare3)
09/13/2023, 11:33 PM
the binary
flytectl demo start runs the binary
c
Cody Scandore
09/13/2023, 11:34 PMEverything seems so straightforward but my scheduler must not be configured properly? The schedule appears in the console, and the workflow executes manually just fine.
The only hangup is there is no execution time, and when fetching the launchplan after updating with
--activatek
Ketan (kumare3)
09/13/2023, 11:35 PM
i will be in a flight, then will try
c
Cody Scandore
09/13/2023, 11:35 PMthanks Ketan, I will try to local binary in the meantime
k
Ketan (kumare3)
09/14/2023, 2:07 AM
@Cody Scandore I was able to try it out locally in the sandbox
This is the xample code
Copy code
from datetime import datetime
from flytekit import task, workflow, LaunchPlan, CronSchedule
@task
def say_hello() -> str:
return "Hello"
@workflow
def talker() -> str:
return say_hello()
example_launch_plan = LaunchPlan.get_or_create(
name="talk-every-minute",
workflow=talker,
schedule=CronSchedule(
schedule="* * * * *", # Following schedule runs every min
),
) Copy code
pyflyte register backfill2.py --activate-launchplansso scheduler is on by default
c
Cody Scandore
09/14/2023, 12:52 PMYour example worked for me local and remote. Thanks Ketan
d
David Espejo (he/him)
09/14/2023, 3:20 PMCool @Cody Scandore, so what's the main difference between your schedule definition and Ketan's example? is it maybe the cron syntax? Just trying to learn
c
Cody Scandore
09/15/2023, 4:06 AMThat's a great question. I believe the syntax was effectively the same. I don't think it's the cron, these crons are dead simple.
The only difference was that I bumped pyflyte to the
mainpyflyte register .. --activate-launchplanspyflyte register..flytectl update launchplan ... --activatek
Ketan (kumare3)
09/15/2023, 4:25 AM
Really that does not make sense
Cc @Samhita Alla can me add this to the docs as a way to register and activate and also deactivate schedules