polite-easter-71504
08/15/2024, 10:41 PMdataclass
when I'm running a workflow, and I need some help.
I get this error when the tasks/workflow are being compiled: TypeError: Invalid base type typing.Union in call to isinstance
. This workflow ran fine, until I started using a dataclass
that I defined.
To figure out the problem, I decided to use the dataclass.py
example from here. I'm running everything locally (ideally without the Flyte demo cluster) to keep things as simple as possible.
I noticed that if I run the example with pyflyte run dataclass.py dataclass_wf --x 10 --y 20
, the workflow runs fine and outputs the result. OTOH, if I run it with python dataclass.py
(which I assume should work because of the if ___name___ == "main"
directive at the bottom of dataclass.py
, I get a large number of errors that seem to either be TypeError: Invalid base type typing.Union in call to isinstance
or TypeError: isinstance() arg 2 must be a type, a tuple of types, or a union
.
For context: the reason I'm trying to run my Flyte workflow via the main method is because I'm using hydra and I could not figure out another way to make sure that the Hydra configs are loaded and made available to the program. I want able to write my workflow so it worked with Omegaconf, but ran into this issue when I introduced dataclasses.
so I am wondering:
1. Is it really only possible to run Flyte workflows via pyflyte
?
2. Should I not expect things to work if I run a Flyte workflow through a file's main function?
3. How can I write my workflow so that I can use dataclasses and I can populate Hydra configs (or somewhat equivalently, run the workflow via the file's main function)?high-accountant-32689
08/16/2024, 6:57 PMpolite-easter-71504
08/20/2024, 5:12 PMflytekitplugins-omegaconf
though. If i run into further issues, I'll pipe up again here.high-accountant-32689
08/20/2024, 5:14 PM