https://flyte.org
Join Slack
Hey, I have an existing Config class that has ~50-60 fields (with type info + defaults), and I want ...
# flytekit
e
Hey, I have an existing Config class that has ~50-60 fields (with type info + defaults), and I want those to show up as parameters on the Flyte console. Naively I can copy paste all of them to the args of a workflow including their types, but I wonder if there's a better way. I can make a function that takes kwargs and create the Config object from those, but ofc having kwargs on a workflow doesn't really work. I saw the PythonFunctionWorkflow constructor has this line 
native_interface = transform_function_to_interface(workflow_function, docstring=docstring)
I assume I could make the workflow function take kwargs, but then change this line to create the interface from the field names+types+default obtained by using 
inspect
on the Config constructor. Does that make sense or is there an easier way?
Or maybe I could use a decorator which rewrites the kwargs function to take explicit parameters, but not sure how to do that yet (Python question in general, not Flyte specific) https://stackoverflow.com/a/33112180 should work
f
@elegant-petabyte-32634 the only supported way is to use a dataclass with json. But it will remain as a single object If you end up adding a new way please contribute
👍 1
e
I ended up using 
dataclasses.make_dataclass
to create a dataclass from my existing config class, using those as workflow parameters, then converting the dataclass obj to a dict and passing that as kwargs to existing config constructor
Copy code
def pydantic_class_to_dataclass(pydantic_class):
    fields = list(inspect.signature(pydantic_class).parameters.values())
    return dataclass_json(
        make_dataclass(
            f"{pydantic_class.__name__}DataClass",
            fields=[
                (p.name, p.annotation, p.default)
                for p in fields
                if not p.name.startswith("_")
            ],
        )
    )
f
Ohh is it pydantic
e
yeh
its nested too which makes it a bit trickier too
f
We should probably add support for pydantic models - this was on the roadmap
Basically similar to dataclass add type transformer for the base ones
e
yea that would be cool
f
Contribution 😍
e
😄 need to take a look at how the existing one for dataclass w orks
f
the dataclass one is a little crazy
163 Views
Open in Slack
PreviousNext
Powered by

Flyte

Flyte enables production-grade orchestration for machine learning workflows and data processing created to accelerate local workflows to production.

Powered by