https://flyte.org logo
Powered by
r

Robin Kahlow

09/15/2022, 12:20 PM
Hey, I have an existing Config class that has ~50-60 fields (with type info + defaults), and I want those to show up as parameters on the Flyte console. Naively I can copy paste all of them to the args of a workflow including their types, but I wonder if there's a better way. I can make a function that takes kwargs and create the Config object from those, but ofc having kwargs on a workflow doesn't really work. I saw the PythonFunctionWorkflow constructor has this line 
native_interface = transform_function_to_interface(workflow_function, docstring=docstring)
I assume I could make the workflow function take kwargs, but then change this line to create the interface from the field names+types+default obtained by using 
inspect
on the Config constructor. Does that make sense or is there an easier way?
Or maybe I could use a decorator which rewrites the kwargs function to take explicit parameters, but not sure how to do that yet (Python question in general, not Flyte specific) https://stackoverflow.com/a/33112180 should work
k

Ketan (kumare3)

09/15/2022, 1:20 PM
@Robin Kahlow the only supported way is to use a dataclass with json. But it will remain as a single object If you end up adding a new way please contribute
r

Robin Kahlow

09/15/2022, 1:57 PM
I ended up using 
dataclasses.make_dataclass
to create a dataclass from my existing config class, using those as workflow parameters, then converting the dataclass obj to a dict and passing that as kwargs to existing config constructor
Copy code
def pydantic_class_to_dataclass(pydantic_class):
    fields = list(inspect.signature(pydantic_class).parameters.values())
    return dataclass_json(
        make_dataclass(
            f"{pydantic_class.__name__}DataClass",
            fields=[
                (p.name, p.annotation, p.default)
                for p in fields
                if not p.name.startswith("_")
            ],
        )
    )
k

Ketan (kumare3)

09/15/2022, 2:02 PM
Ohh is it pydantic
r

Robin Kahlow

09/15/2022, 2:02 PM
yeh
its nested too which makes it a bit trickier too
k

Ketan (kumare3)

09/15/2022, 2:02 PM
We should probably add support for pydantic models - this was on the roadmap
Basically similar to dataclass add type transformer for the base ones
r

Robin Kahlow

09/15/2022, 2:03 PM
yea that would be cool
k

Ketan (kumare3)

09/15/2022, 2:03 PM
Contribution 😍
r

Robin Kahlow

09/15/2022, 2:03 PM
😄 need to take a look at how the existing one for dataclass w orks
k

Ketan (kumare3)

09/15/2022, 11:50 PM
the dataclass one is a little crazy
3 Views
Powered by