hey guys, context: i have a workflow that runs onc...
# ask-the-community
e
hey guys, context: i have a workflow that runs once a day and it takes a parameter
date
so that i can run it dynamically on different data. I also have a launch plan based on it which defaults to that today. i have a jenkins job which registers the launch plan each day. i make the
version
of the launch plan = the latest commit hash so that i can identify what state the codebase was in when i executed a flyte workflow. problem: when the launch plan gets registered two consecutive days with no changes in git commit hash /
version
), the ID of the launch plan is the same but the input start date has changed which throws this error:
Copy code
Error: rpc error: code = InvalidArgument desc = launch plan with different structure already exists with id resource_type:LAUNCH_PLAN project:<project> domain:<domain> name:<name> version:<git commit hash>
afaik, the launch plan's ID is a combination of {
resourceType
project
domain
name
and
version
} anyone know how to work around this? i'd like to keep the version of the flyte launch plan still connected to a git commit hash so that i can trace workflow failures to code changes but i also want to retain being able to run a launch plan with different arguments each day. thanks!
for the time being what i can do is append a timestamp onto the version so that i have <revision>-<timestamp> but im wondering is there a better practice out there? has anyone done anything related to launch plans and default inputs
s
hi @Erich Shan, if it's just the argument that you want to change, why do you want to reregister your launch plan everyday?
r
we have a nightly job that rebuilds and deploys master to our production instance. That same job ends up re-registering the launch plans as well since we have that workflow running once a day
So we wanted the flexibility where the workflow can take an input date (allowing us to do a backfill) or just compute the previous day’s date and run with that
We are trying to figure out what’s the best way to handle both use cases with the same workflow
s
understood. the problem is you cannot reregister a launch plan if there are no changes in the code. it indeed doesn't make much sense to do so.
r
Got it.. let me see if there is a cleaner way to do this
e
@Samhita Alla what if the source code doesn't change but the it dynamically executes based on the date? for example having a timestamp associated with an input variable:
start_date: str = datetime.strftime(datetime.now() - timedelta(days=1), '%Y-%m-%d-%H:%M:%S')
this variable needs to change each time it's called, does this mean we need to re-register the files each time or is a flytectl update call sufficient to re-evaluate the
start_date
variable
s
you needn't re-register your launch plan. i think the best practice would be to send the current datetime to your workflow using the `kickoff_time_input_arg`: https://docs.flyte.org/projects/cookbook/en/latest/getting_started/run_schedule.html#passing-in-the-scheduled-kick-off-time, and do all the other manipulations within the task/workflow. you can register it once and it has to just work!
r
oh.. that’s awesome! Didn’t know about this at all