Robin Kahlow

    Robin Kahlow

    1 week ago
    Hey, I have an existing Config class that has ~50-60 fields (with type info + defaults), and I want those to show up as parameters on the Flyte console. Naively I can copy paste all of them to the args of a workflow including their types, but I wonder if there's a better way. I can make a function that takes kwargs and create the Config object from those, but ofc having kwargs on a workflow doesn't really work. I saw the PythonFunctionWorkflow constructor has this line
    native_interface = transform_function_to_interface(workflow_function, docstring=docstring)
    I assume I could make the workflow function take kwargs, but then change this line to create the interface from the field names+types+default obtained by using
    inspect
    on the Config constructor. Does that make sense or is there an easier way?
    Or maybe I could use a decorator which rewrites the kwargs function to take explicit parameters, but not sure how to do that yet (Python question in general, not Flyte specific) https://stackoverflow.com/a/33112180 should work
    Ketan (kumare3)

    Ketan (kumare3)

    1 week ago
    @Robin Kahlow the only supported way is to use a dataclass with json. But it will remain as a single object If you end up adding a new way please contribute
    Robin Kahlow

    Robin Kahlow

    1 week ago
    I ended up using
    dataclasses.make_dataclass
    to create a dataclass from my existing config class, using those as workflow parameters, then converting the dataclass obj to a dict and passing that as kwargs to existing config constructor
    def pydantic_class_to_dataclass(pydantic_class):
        fields = list(inspect.signature(pydantic_class).parameters.values())
        return dataclass_json(
            make_dataclass(
                f"{pydantic_class.__name__}DataClass",
                fields=[
                    (p.name, p.annotation, p.default)
                    for p in fields
                    if not p.name.startswith("_")
                ],
            )
        )
    Ketan (kumare3)

    Ketan (kumare3)

    1 week ago
    Ohh is it pydantic
    Robin Kahlow

    Robin Kahlow

    1 week ago
    yeh
    its nested too which makes it a bit trickier too
    Ketan (kumare3)

    Ketan (kumare3)

    1 week ago
    We should probably add support for pydantic models - this was on the roadmap
    Basically similar to dataclass add type transformer for the base ones
    Robin Kahlow

    Robin Kahlow

    1 week ago
    yea that would be cool
    Ketan (kumare3)

    Ketan (kumare3)

    1 week ago
    Contribution 😍
    Robin Kahlow

    Robin Kahlow

    1 week ago
    😄 need to take a look at how the existing one for dataclass w orks
    Ketan (kumare3)

    Ketan (kumare3)

    1 week ago
    the dataclass one is a little crazy