Dan Farrell
09/19/2023, 9:06 PMfetch_task
... I'm leaning towards a library because then we get type hints which I forsee as solving some input/output headaches...
Edit: I take it back... I see the utility of registering repos ( disconnected python env requirements )... lack of python typing does seem sad though....Yee
Dan Farrell
09/19/2023, 10:46 PMYee
Dan Farrell
09/19/2023, 10:49 PMYee
Ketan (kumare3)
another question, Are there any examples of repos with many workflows that have overlapping tasks, and/or combined workflows?Flytesnacks is an example. Massive repo, lots of workflows. Not overlaping though. But we do have these examples at all the large companies.
Dan Farrell
09/20/2023, 2:54 PMif sklearn_image_spec.is_container():
works if the types are contained to within the task that the container is used in. But what about if I want a type that is specific to that container to be a return type? what would you suggest the type-hinting be?Ketan (kumare3)
Samhita Alla
is_container()
block?Ketan (kumare3)
multi/
├── py_task.py
├── spark_task.py
└── wf.py
Dan Farrell
10/04/2023, 4:29 PMflytekit/core/interface.py
seems to be grabbing all tasks's type hints via type_hints = get_type_hints(fn, include_extras=True)
but this fails because my Foo
class isn't available in the combo-wf
environment.Ketan (kumare3)
Samhita Alla
Dan Farrell
10/05/2023, 8:56 AMSamhita Alla
Dan Farrell
10/05/2023, 2:38 PMone_off_combo...
script? I'd prefer not to, would like to allow scientists to work outside of the git-repo if they choose to (most do)Samhita Alla
name 'Foo' is not defined
the error?Foo
is the output type of your task, correct? And that isn't being found by Flyte.Dan Farrell
10/05/2023, 4:47 PMSamhita Alla