Hema Jayachandran
07/13/2023, 11:51 AMinput_date
as one of the parameter. This launch plan was updated in the cluster yesterday (2023-07-12).
While we were expecting todays execution to pick the input date as 2023-07-13 we noticed that it picked 2023-07-12 instead (This is our first scheduled run). Is our assumption right that its because of this default_inputs={"date": datetime.today().strftime("%Y-%m-%d")}
that it is set only once while the launch plan is updated or created and hence picking that date?
LaunchPlan.get_or_create(
name=f"launch_plan_{ecosystem}",
workflow=test_workflow,
schedule=CronSchedule(schedule="0 7 * * *"),
default_inputs={"date": datetime.today().strftime("%Y-%m-%d")},
fixed_inputs={"ecosystem": ecosystem},
)
Is there a better way to handle input dates ? All our pipelines expects an input date as parameter; for some it’s the same date as execution date and for some it’s one day behind the execution date.Thomas Blom
07/13/2023, 2:14 PMHema Jayachandran
07/13/2023, 2:23 PMDavid Espejo (he/him)
07/13/2023, 2:56 PMSamhita Alla
kickoff_time_input_arg
to send the current date.
https://docs.flyte.org/projects/cookbook/en/latest/getting_started/run_schedule.html#passing-in-the-scheduled-kick-off-timeHema Jayachandran
07/14/2023, 5:27 AMcurrent_date - 1
Is it possible to achieve it through this kickoff_time_input_arg
?Samhita Alla
Hema Jayachandran
07/14/2023, 6:56 AMOptional[str]
so that when it’s None
, it picks the current date else we manually override it with a date if needed.
It works fine when I launch the workflow from the console but it doesn’t work when launching the workflow with exec_spec.yaml
in this case
inputs:
input_date: null
It throws
Message:
time data '' does not match format '%Y-%m-%d'
It looks like the null
is not picked as None.Samhita Alla
Hema Jayachandran
07/14/2023, 7:23 AMflytectl create execution -p ${MODULE_NAME} -d ${DOMAIN} --execFile exec_spec.yaml
Samhita Alla
Optional[str]
, why does the error include the date time format?Hema Jayachandran
07/14/2023, 7:27 AM