Dan Farrell09/19/2023, 9:06 PM
... I'm leaning towards a library because then we get type hints which I forsee as solving some input/output headaches... Edit: I take it back... I see the utility of registering repos ( disconnected python env requirements )... lack of python typing does seem sad though....
Dan Farrell09/19/2023, 10:46 PM
Dan Farrell09/19/2023, 10:49 PM
another question, Are there any examples of repos with many workflows that have overlapping tasks, and/or combined workflows?Flytesnacks is an example. Massive repo, lots of workflows. Not overlaping though. But we do have these examples at all the large companies.
Dan Farrell09/20/2023, 2:54 PM
works if the types are contained to within the task that the container is used in. But what about if I want a type that is specific to that container to be a return type? what would you suggest the type-hinting be?
multi/ ├── py_task.py ├── spark_task.py └── wf.py
Dan Farrell10/04/2023, 4:29 PM
seems to be grabbing all tasks's type hints via
but this fails because my
type_hints = get_type_hints(fn, include_extras=True)
class isn't available in the
Dan Farrell10/05/2023, 8:56 AM
Dan Farrell10/05/2023, 2:38 PM
script? I'd prefer not to, would like to allow scientists to work outside of the git-repo if they choose to (most do)
name 'Foo' is not defined
is the output type of your task, correct? And that isn't being found by Flyte.
Dan Farrell10/05/2023, 4:47 PM