hello, I am trying to work through an issue with u...
# flyte-support
p
hello, I am trying to work through an issue with using
dataclass
when I'm running a workflow, and I need some help. I get this error when the tasks/workflow are being compiled:
TypeError: Invalid base type typing.Union in call to isinstance
. This workflow ran fine, until I started using a
dataclass
that I defined. To figure out the problem, I decided to use the
dataclass.py
example from here. I'm running everything locally (ideally without the Flyte demo cluster) to keep things as simple as possible. I noticed that if I run the example with
pyflyte run dataclass.py dataclass_wf --x 10 --y 20
, the workflow runs fine and outputs the result. OTOH, if I run it with
python dataclass.py
(which I assume should work because of the
if ___name___ == "main"
directive at the bottom of
dataclass.py
, I get a large number of errors that seem to either be
TypeError: Invalid base type typing.Union in call to isinstance
or
TypeError: isinstance() arg 2 must be a type, a tuple of types, or a union
. For context: the reason I'm trying to run my Flyte workflow via the main method is because I'm using hydra and I could not figure out another way to make sure that the Hydra configs are loaded and made available to the program. I want able to write my workflow so it worked with Omegaconf, but ran into this issue when I introduced dataclasses. so I am wondering: 1. Is it really only possible to run Flyte workflows via
pyflyte
? 2. Should I not expect things to work if I run a Flyte workflow through a file's main function? 3. How can I write my workflow so that I can use dataclasses and I can populate Hydra configs (or somewhat equivalently, run the workflow via the file's main function)?
h
hi, @polite-easter-71504, answers in order: 1. no, you should be able to invoke flyte's taks and workflows like regular python functions 2. answered above, i.e. this should work. 3. an OSS user recently developed https://pypi.org/project/flytekitplugins-omegaconf/. Can you give it a try? Can you give more details about the dataclass you're using?
p
hi eduardo, thanks for your response. I ended up finding a way around this issue by simply not using the dataclass. I don't quite remember the dataclass I was using at this point. I'm already using
flytekitplugins-omegaconf
though. If i run into further issues, I'll pipe up again here.
h
Alright, thank you.