hey guys, context: i have a workflow that runs onc...
# ask-the-community
hey guys, context: i have a workflow that runs once a day and it takes a parameter
so that i can run it dynamically on different data. I also have a launch plan based on it which defaults to that today. i have a jenkins job which registers the launch plan each day. i make the
of the launch plan = the latest commit hash so that i can identify what state the codebase was in when i executed a flyte workflow. problem: when the launch plan gets registered two consecutive days with no changes in git commit hash /
), the ID of the launch plan is the same but the input start date has changed which throws this error:
Copy code
Error: rpc error: code = InvalidArgument desc = launch plan with different structure already exists with id resource_type:LAUNCH_PLAN project:<project> domain:<domain> name:<name> version:<git commit hash>
afaik, the launch plan's ID is a combination of {
} anyone know how to work around this? i'd like to keep the version of the flyte launch plan still connected to a git commit hash so that i can trace workflow failures to code changes but i also want to retain being able to run a launch plan with different arguments each day. thanks!
for the time being what i can do is append a timestamp onto the version so that i have <revision>-<timestamp> but im wondering is there a better practice out there? has anyone done anything related to launch plans and default inputs
hi @Erich Shan, if it's just the argument that you want to change, why do you want to reregister your launch plan everyday?
we have a nightly job that rebuilds and deploys master to our production instance. That same job ends up re-registering the launch plans as well since we have that workflow running once a day
So we wanted the flexibility where the workflow can take an input date (allowing us to do a backfill) or just compute the previous day’s date and run with that
We are trying to figure out what’s the best way to handle both use cases with the same workflow
understood. the problem is you cannot reregister a launch plan if there are no changes in the code. it indeed doesn't make much sense to do so.
Got it.. let me see if there is a cleaner way to do this
@Samhita Alla what if the source code doesn't change but the it dynamically executes based on the date? for example having a timestamp associated with an input variable:
start_date: str = datetime.strftime(datetime.now() - timedelta(days=1), '%Y-%m-%d-%H:%M:%S')
this variable needs to change each time it's called, does this mean we need to re-register the files each time or is a flytectl update call sufficient to re-evaluate the
you needn't re-register your launch plan. i think the best practice would be to send the current datetime to your workflow using the `kickoff_time_input_arg`: https://docs.flyte.org/projects/cookbook/en/latest/getting_started/run_schedule.html#passing-in-the-scheduled-kick-off-time, and do all the other manipulations within the task/workflow. you can register it once and it has to just work!
oh.. that’s awesome! Didn’t know about this at all