https://flyte.org
Join Slack
hey gang, new user for the python SDK - are there examples of good patterns for mocking remote tasks...
# flyte-support
m
hey gang, new user for the python SDK - are there examples of good patterns for mocking remote tasks/workflows, a la what 
task_mock
can do for local python tasks? While I can get local patching working sometimes, it runs into issues e.g. with explicit task dependencies set like 
t1() >> t2()
where the return value (mocked as a python type) is missing some flyte node plumbing under the hood.
👀 1
t
do you have repros you can copy/paste please?
we’ll make a ticket out of it and work on it.
it should work, and even if there are cases where they don’t we need to improve our error messaging to explain why.
m
Untitled.py
Untitled.py
hmm, might be harder cause the remotes are all from our internal task warehouse but basically the setup goes like ☝️
for testing, the guidance from our ML infra team was to just patch out the return values to their python types
Untitled.py
Untitled.py
when the remotes return values and done use the 
__rshift__
logic, it seemingly works ok (though the patches are pretty sensitive to imports in the test...) but the above fails with an assertion error
that 
None
lacks the 
ref
attribute needed to actually build the DAG
it should work, and even if there are cases where they don’t we need to improve our error messaging to explain why.
I suspect the issue is actually in our guidance for how to mock these, tbh
@thankful-minister-83577 looking at this some more, I wonder if the problem isn't how that 
handle
function is built in our internal libraries - it looks like the 
RemoteEntity
returned by 
fetch_task
should work with 
flytekit.testing.patch
right?
t
yes it should.
let me spend some time today looking.
m
yeah I think we have one too many layers of abstraction, where our internal lib's wrappers are returning the result of calling the RemoteEntity rather than returning the entity itself, which then would work with the patching
t
let me get the patching working for remote entities first, demonstrate that basic unit tests work
m
👍
t
@mysterious-whale-21778 reviewed the testing code in flytekit. it does look like it’s built currently just for non-fetched tasks. but i’m not sure that really matters…
i’ll add an example of how you might test a fetched task - https://github.com/flyteorg/flytekit/pull/1722/files
you can avoid using the testing stuff in flytekit completely.
see the 
test_mocking_remote
test
m
ok yeah, that squares with my thinking - we've just got the actual remote task buried under one too many layers of abstraction
reckon I could try building a patcher functionality for our wrappers using the logic you've got there
ok yeah I was able to get it working with patching 
fetch_remote
within our internals but that required defining a mock task with matching kwargs for each remote task, which is a little gnarly
would be nicer if we could find a way to 
task_mock
the RemoteEntity it grabs like you've got in yours
hah, actually seems like we get the same effect if we just make a mock task and set that as a side effect of the mocked parent function. same problem with the expected arguments though
t
yeah there’s no way around the expected arguments. that feeds the interface and the interface needs to be correctly defined.
we’ll merge that pr today
m
that feeds the interface and the interface needs to be correctly defined.
yeah, unless we can get at mocking the internal RemoteEntity reckon we're stuck with it
146 Views
Open in Slack
PreviousNext
Powered by

Flyte

Flyte enables production-grade orchestration for machine learning workflows and data processing created to accelerate local workflows to production.

Powered by