magnificent-arm-99553
11/27/2024, 7:58 PMmyproject.models.alabama
, myproject.models.california
, myproject.models.texas
. Each one has the same API.
How would you all create tasks/workflows for each of these models, without having to copy/paste the same code for each model?
I've tried things like making a function/task factory, but Flyte keeps telling me ValueError: TaskFunction cannot be a nested/inner or local function.
. Using importlib to import-by-string works okay, but then pyflyte run --remote
doesn't package the model's code and its dependencies when submitting the workflow to the remote.glamorous-carpet-83516
11/27/2024, 8:11 PMglamorous-carpet-83516
11/27/2024, 8:11 PMthankful-minister-83577
magnificent-arm-99553
11/27/2024, 8:14 PMmagnificent-arm-99553
11/27/2024, 8:17 PM--copy auto
pickup modules imported by calling importlib
inside a task run? I'm not sure it can, since those won't show up in sys.modules
until the task is run.magnificent-arm-99553
11/27/2024, 8:18 PMfetch_training_data
and not fetch_training_data_texas
and fetch_training_data_alabama
thankful-minister-83577
fetch_training_data
task take a string input?magnificent-arm-99553
11/27/2024, 8:40 PM