I am running into a predicament when I use dataclasses for params. Sometimes, I want to pass in the entire dataclass into a task, other times I want to reference a specific field.
I have explored two options:
⢠Have a task that converts a dataclass to a named tuple
⢠Use @dynamic workflows
Using dynamic workflows works, but maybe there are side effects? I would love to learn from folks who have gone down this path before.
Cc @future-monitor-58430
I think Ankit and team offered to implement this in Flyte
š 1
š 1
freezing-airport-6809
02/16/2023, 4:03 PM
So @abundant-hamburger-66584 I think we will support this in native Flyte in sometime
h
hallowed-mouse-14616
02/16/2023, 4:04 PM
For some context - this conversation. Basically, we could encode the Flyte
Literal
as a field of the dataclass rather than just the dataclass itself, then propeller internally can parse and retrieve the field when passing input values. I'm convinced it would be possible.
hallowed-mouse-14616
02/16/2023, 4:05 PM
Then you wouldn't need to use a dynamic workflow here and incur the overhead of starting a separate Pod just to retrieve the field value from your Params dataclass. Propeller could parse it directly.