wooden-sandwich-59360
12/30/2022, 6:55 AMtall-lock-23197
freezing-airport-6809
freezing-airport-6809
freezing-airport-6809
freezing-airport-6809
tall-lock-23197
but it would be really nice to be able to take start date and number of days back as an input to the workflow (the blocker is that we cannot iterate over a Promise).You can use a dynamic workflow to perform iterations over workflow inputs. (thanks @hallowed-mouse-14616)
freezing-airport-6809
wooden-sandwich-59360
01/02/2023, 7:36 AMfreezing-airport-6809
freezing-airport-6809
freezing-airport-6809
freezing-airport-6809
broad-monitor-993
01/03/2023, 3:17 PMbut it would be really nice to be able to take start date and number of days back as an input to the workflowIf I understand correctly, is this what you want to achieve at a high level? Assumptions: • I have a
process_dataset
routine that processes a single dataset for a particular date
• Every day I process today’s dataset, in addition to datasets for some number of days back.
• Each invocation of process_dataset
for a particular day’s dataset is completely independent from processing other day’s datasets.
Steps:
Every day, kick off scheduled job at `kickoff_datetime`:
1. Pass in kickoff_datetime
into main launchplan entrypoint, in addition to a num_days_back
parameter
2. Generate a list of dates, which includes today’s date and some num_days_back
worth of days
3. Apply process_dataset
to each dataset in an embarrassingly parallel fashion
Does this seem correct?freezing-airport-6809
freezing-airport-6809
wooden-sandwich-59360
01/04/2023, 11:10 AMwooden-sandwich-59360
01/05/2023, 9:19 AMbroad-monitor-993
01/05/2023, 2:54 PMuser
01/05/2023, 2:54 PMwooden-sandwich-59360
01/05/2023, 3:13 PMfreezing-airport-6809
wooden-sandwich-59360
01/06/2023, 8:17 AMfreezing-airport-6809