Hi I m excited to start using FlyteRemote My first usecase i Flyte #flytekit

Hi, I’m excited to start using FlyteRemote. My fir...

rough-rose-81585

08/23/2022, 2:50 PM

Hi, I’m excited to start using FlyteRemote. My first usecase is I’d like to retrieve a launchplan and modify a default input before executing. I have successfully done this for primitive inputs (str, int, etc.), but FlyteRemote complained when I tried to pass in a

@dataclass_json

class

Foo

. It said i needed

types.FooSchema

not

Foo

. I assume there is some way to transform

Foo

into

FooSchema

using either dataclass_json api or flyte api, but I’m not sure how to do it. Could you direct me? Exception and sample code in thread Thanks!

rough-rose-81585

08/23/2022, 3:04 PM

Copy code

Traceback (most recent call last):
  File "orchestration/msat/utility/flyte_remote.py", line 26, in <module>
    print(remote.execute(entity=level2_single_retrieval_lp, inputs={"proxy_params": proxy_params}))
  File "/Users/nlofaso/.virtualenvs/msat/lib/python3.8/site-packages/flytekit/remote/remote.py", line 842, in execute
    return self.execute_remote_task_lp(
  File "/Users/nlofaso/.virtualenvs/msat/lib/python3.8/site-packages/flytekit/remote/remote.py", line 919, in execute_remote_task_lp
    return self._execute(
  File "/Users/nlofaso/.virtualenvs/msat/lib/python3.8/site-packages/flytekit/remote/remote.py", line 715, in _execute
    lit = TypeEngine.to_literal(ctx, v, hint, variable.type)
  File "/Users/nlofaso/.virtualenvs/msat/lib/python3.8/site-packages/flytekit/core/type_engine.py", line 692, in to_literal
    transformer.assert_type(python_type, python_val)
  File "/Users/nlofaso/.virtualenvs/msat/lib/python3.8/site-packages/flytekit/core/type_engine.py", line 97, in assert_type
    raise TypeTransformerFailedError(f"Type of Val '{v}' is not an instance of {t}")
flytekit.core.type_engine.TypeTransformerFailedError: Type of Val 'Level2ProxyParameters(id='FlyteRemoteTest', job_i_step=1, job_j_step=300, control_template={'CO2': 'CO2_GGG2020_1x1_MAIR.control', 'H2O': 'H2O_GGG2020_1x1_MAIR.control', 'O2': 'O2_GGG2020_1x1_MAIR.control'}, aggregation=<Aggregation.GROUP_1x1: '1x1'>)' is not an instance of <class 'types.Level2proxyparametersSchema'>

rough-rose-81585

08/23/2022, 3:04 PM

flyte_remote.py

rough-rose-81585

08/23/2022, 3:04 PM

I tried using

Copy code

proxy_params.schema()

but received

Copy code

flytekit.core.type_engine.TypeTransformerFailedError: Type of Val '<Level2proxyparametersSchema(many=False)>' is not an instance of <class 'types.Level2proxyparametersSchema'>

broad-monitor-993

08/23/2022, 3:07 PM

hey @rough-rose-81585 can you try providing the

type_hints

argument to the

remote.execute

call?

Copy code

remote.execute(..., type_hints={"proxy_params": types.Level2proxyparametersSchema})

I’ve come across this issue before, if I recall correctly in my case it happened because the dataclass was defined programmatically and wasn’t accessible in the module-level scope. Let me know what you get

rough-rose-81585

08/23/2022, 3:28 PM

Incredible that was exactly the issue! Thanks @broad-monitor-993

Copy code

remote.execute(..., type_hints={"proxy_params": Level2ProxyParameters})

thx 1

broad-monitor-993

08/23/2022, 3:31 PM

@rough-rose-81585 great! just to confirm, is

Level2ProxyParameters

being defined programmatically (say, within a function), or is it defined in the top-level scope of your module?

rough-rose-81585

08/23/2022, 3:31 PM

It’s defined in a different package (not within a function)

broad-monitor-993

08/23/2022, 4:10 PM

gotcha. @high-accountant-32689 @thankful-minister-83577 we should probably • look into why the type transformer can handle these cases and raises:

Copy code

flytekit.core.type_engine.TypeTransformerFailedError: Type of Val '<Level2proxyparametersSchema(many=False)>' is not an instance of <class 'types.Level2proxyparametersSchema'>

• if appropriate, update the error message here to direct the user to use the

type_hints

argument • document this behavior better @billowy-sundown-31926 @tall-lock-23197

👀 1

shy-holiday-15500

09/02/2022, 7:47 PM

This workaround fixed things for us too. If other people find this (can add this to an issue if one exists, will look in a minute), be sure to pass default_project and default_domain when creating your remote object. We passed them during fetch_workflow, and got an error during execute about missing project/domain (edit: don't see an issue from https://github.com/flyteorg/flyte/issues?q=Level2proxyparametersSchema, may be tangentially related to https://github.com/flyteorg/flyte/issues/2502, but not trivially the same as this is about dataclasses)

shy-holiday-15500

09/02/2022, 7:54 PM

Opened https://github.com/flyteorg/flyte/issues/2844 to help with search in the future

tall-lock-23197

09/03/2022, 7:55 AM

cc: @high-accountant-32689 / @thankful-minister-83577

163 Views

Open in Slack

Previous Next