https://flyte.org logo
Title
n

Nicholas LoFaso

08/23/2022, 2:50 PM
Hi, I’m excited to start using FlyteRemote. My first usecase is I’d like to retrieve a launchplan and modify a default input before executing. I have successfully done this for primitive inputs (str, int, etc.), but FlyteRemote complained when I tried to pass in a
@dataclass_json
class
Foo
. It said i needed
types.FooSchema
not
Foo
. I assume there is some way to transform
Foo
into
FooSchema
using either dataclass_json api or flyte api, but I’m not sure how to do it. Could you direct me? Exception and sample code in thread Thanks!
Traceback (most recent call last):
  File "orchestration/msat/utility/flyte_remote.py", line 26, in <module>
    print(remote.execute(entity=level2_single_retrieval_lp, inputs={"proxy_params": proxy_params}))
  File "/Users/nlofaso/.virtualenvs/msat/lib/python3.8/site-packages/flytekit/remote/remote.py", line 842, in execute
    return self.execute_remote_task_lp(
  File "/Users/nlofaso/.virtualenvs/msat/lib/python3.8/site-packages/flytekit/remote/remote.py", line 919, in execute_remote_task_lp
    return self._execute(
  File "/Users/nlofaso/.virtualenvs/msat/lib/python3.8/site-packages/flytekit/remote/remote.py", line 715, in _execute
    lit = TypeEngine.to_literal(ctx, v, hint, variable.type)
  File "/Users/nlofaso/.virtualenvs/msat/lib/python3.8/site-packages/flytekit/core/type_engine.py", line 692, in to_literal
    transformer.assert_type(python_type, python_val)
  File "/Users/nlofaso/.virtualenvs/msat/lib/python3.8/site-packages/flytekit/core/type_engine.py", line 97, in assert_type
    raise TypeTransformerFailedError(f"Type of Val '{v}' is not an instance of {t}")
flytekit.core.type_engine.TypeTransformerFailedError: Type of Val 'Level2ProxyParameters(id='FlyteRemoteTest', job_i_step=1, job_j_step=300, control_template={'CO2': 'CO2_GGG2020_1x1_MAIR.control', 'H2O': 'H2O_GGG2020_1x1_MAIR.control', 'O2': 'O2_GGG2020_1x1_MAIR.control'}, aggregation=<Aggregation.GROUP_1x1: '1x1'>)' is not an instance of <class 'types.Level2proxyparametersSchema'>
I tried using
proxy_params.schema()
but received
flytekit.core.type_engine.TypeTransformerFailedError: Type of Val '<Level2proxyparametersSchema(many=False)>' is not an instance of <class 'types.Level2proxyparametersSchema'>
n

Niels Bantilan

08/23/2022, 3:07 PM
hey @Nicholas LoFaso can you try providing the
type_hints
argument to the
remote.execute
call?
remote.execute(..., type_hints={"proxy_params": types.Level2proxyparametersSchema})
I’ve come across this issue before, if I recall correctly in my case it happened because the dataclass was defined programmatically and wasn’t accessible in the module-level scope. Let me know what you get
n

Nicholas LoFaso

08/23/2022, 3:28 PM
Incredible that was exactly the issue! Thanks @Niels Bantilan
remote.execute(..., type_hints={"proxy_params": Level2ProxyParameters})
n

Niels Bantilan

08/23/2022, 3:31 PM
@Nicholas LoFaso great! just to confirm, is
Level2ProxyParameters
being defined programmatically (say, within a function), or is it defined in the top-level scope of your module?
n

Nicholas LoFaso

08/23/2022, 3:31 PM
It’s defined in a different package (not within a function)
n

Niels Bantilan

08/23/2022, 4:10 PM
gotcha. @Eduardo Apolinario (eapolinario) @Yee we should probably • look into why the type transformer can handle these cases and raises:
flytekit.core.type_engine.TypeTransformerFailedError: Type of Val '<Level2proxyparametersSchema(many=False)>' is not an instance of <class 'types.Level2proxyparametersSchema'>
• if appropriate, update the error message here to direct the user to use the
type_hints
argument • document this behavior better @Smriti Satyan @Samhita Alla
👀 1
c

Calvin Leather

09/02/2022, 7:47 PM
This workaround fixed things for us too. If other people find this (can add this to an issue if one exists, will look in a minute), be sure to pass default_project and default_domain when creating your remote object. We passed them during fetch_workflow, and got an error during execute about missing project/domain (edit: don't see an issue from https://github.com/flyteorg/flyte/issues?q=Level2proxyparametersSchema, may be tangentially related to https://github.com/flyteorg/flyte/issues/2502, but not trivially the same as this is about dataclasses)
Opened https://github.com/flyteorg/flyte/issues/2844 to help with search in the future
s

Samhita Alla

09/03/2022, 7:55 AM
cc: @Eduardo Apolinario (eapolinario) / @Yee