Nicholas LoFaso

    Nicholas LoFaso

    1 month ago
    Hi, I’m excited to start using FlyteRemote. My first usecase is I’d like to retrieve a launchplan and modify a default input before executing. I have successfully done this for primitive inputs (str, int, etc.), but FlyteRemote complained when I tried to pass in a
    @dataclass_json
    class
    Foo
    . It said i needed
    types.FooSchema
    not
    Foo
    . I assume there is some way to transform
    Foo
    into
    FooSchema
    using either dataclass_json api or flyte api, but I’m not sure how to do it. Could you direct me? Exception and sample code in thread Thanks!
    Traceback (most recent call last):
      File "orchestration/msat/utility/flyte_remote.py", line 26, in <module>
        print(remote.execute(entity=level2_single_retrieval_lp, inputs={"proxy_params": proxy_params}))
      File "/Users/nlofaso/.virtualenvs/msat/lib/python3.8/site-packages/flytekit/remote/remote.py", line 842, in execute
        return self.execute_remote_task_lp(
      File "/Users/nlofaso/.virtualenvs/msat/lib/python3.8/site-packages/flytekit/remote/remote.py", line 919, in execute_remote_task_lp
        return self._execute(
      File "/Users/nlofaso/.virtualenvs/msat/lib/python3.8/site-packages/flytekit/remote/remote.py", line 715, in _execute
        lit = TypeEngine.to_literal(ctx, v, hint, variable.type)
      File "/Users/nlofaso/.virtualenvs/msat/lib/python3.8/site-packages/flytekit/core/type_engine.py", line 692, in to_literal
        transformer.assert_type(python_type, python_val)
      File "/Users/nlofaso/.virtualenvs/msat/lib/python3.8/site-packages/flytekit/core/type_engine.py", line 97, in assert_type
        raise TypeTransformerFailedError(f"Type of Val '{v}' is not an instance of {t}")
    flytekit.core.type_engine.TypeTransformerFailedError: Type of Val 'Level2ProxyParameters(id='FlyteRemoteTest', job_i_step=1, job_j_step=300, control_template={'CO2': 'CO2_GGG2020_1x1_MAIR.control', 'H2O': 'H2O_GGG2020_1x1_MAIR.control', 'O2': 'O2_GGG2020_1x1_MAIR.control'}, aggregation=<Aggregation.GROUP_1x1: '1x1'>)' is not an instance of <class 'types.Level2proxyparametersSchema'>
    I tried using
    proxy_params.schema()
    but received
    flytekit.core.type_engine.TypeTransformerFailedError: Type of Val '<Level2proxyparametersSchema(many=False)>' is not an instance of <class 'types.Level2proxyparametersSchema'>
    Niels Bantilan

    Niels Bantilan

    1 month ago
    hey @Nicholas LoFaso can you try providing the
    type_hints
    argument to the
    remote.execute
    call?
    remote.execute(..., type_hints={"proxy_params": types.Level2proxyparametersSchema})
    I’ve come across this issue before, if I recall correctly in my case it happened because the dataclass was defined programmatically and wasn’t accessible in the module-level scope. Let me know what you get
    Nicholas LoFaso

    Nicholas LoFaso

    1 month ago
    Incredible that was exactly the issue! Thanks @Niels Bantilan
    remote.execute(..., type_hints={"proxy_params": Level2ProxyParameters})
    Niels Bantilan

    Niels Bantilan

    1 month ago
    @Nicholas LoFaso great! just to confirm, is
    Level2ProxyParameters
    being defined programmatically (say, within a function), or is it defined in the top-level scope of your module?
    Nicholas LoFaso

    Nicholas LoFaso

    1 month ago
    It’s defined in a different package (not within a function)
    Niels Bantilan

    Niels Bantilan

    1 month ago
    gotcha. @Eduardo Apolinario (eapolinario) @Yee we should probably • look into why the type transformer can handle these cases and raises:
    flytekit.core.type_engine.TypeTransformerFailedError: Type of Val '<Level2proxyparametersSchema(many=False)>' is not an instance of <class 'types.Level2proxyparametersSchema'>
    • if appropriate, update the error message here to direct the user to use the
    type_hints
    argument • document this behavior better @Smriti Satyan @Samhita Alla
    Calvin Leather

    Calvin Leather

    3 weeks ago
    This workaround fixed things for us too. If other people find this (can add this to an issue if one exists, will look in a minute), be sure to pass default_project and default_domain when creating your remote object. We passed them during fetch_workflow, and got an error during execute about missing project/domain (edit: don't see an issue from https://github.com/flyteorg/flyte/issues?q=Level2proxyparametersSchema, may be tangentially related to https://github.com/flyteorg/flyte/issues/2502, but not trivially the same as this is about dataclasses)
    Opened https://github.com/flyteorg/flyte/issues/2844 to help with search in the future
    Samhita Alla

    Samhita Alla

    3 weeks ago
    cc: @Eduardo Apolinario (eapolinario) / @Yee