https://flyte.org logo
#ask-the-community
Title
# ask-the-community
j

John Walk

06/28/2023, 4:09 PM
hey gang, new user for the python SDK - are there examples of good patterns for mocking remote tasks/workflows, a la what
task_mock
can do for local python tasks? While I can get local patching working sometimes, it runs into issues e.g. with explicit task dependencies set like
t1() >> t2()
where the return value (mocked as a python type) is missing some flyte node plumbing under the hood.
y

Yee

06/28/2023, 4:49 PM
do you have repros you can copy/paste please?
we’ll make a ticket out of it and work on it.
it should work, and even if there are cases where they don’t we need to improve our error messaging to explain why.
j

John Walk

06/28/2023, 7:37 PM
Untitled.py
hmm, might be harder cause the remotes are all from our internal task warehouse but basically the setup goes like ☝️
for testing, the guidance from our ML infra team was to just patch out the return values to their python types
Untitled.py
when the remotes return values and done use the
__rshift__
logic, it seemingly works ok (though the patches are pretty sensitive to imports in the test...) but the above fails with an assertion error
that
None
lacks the
ref
attribute needed to actually build the DAG
it should work, and even if there are cases where they don’t we need to improve our error messaging to explain why.
I suspect the issue is actually in our guidance for how to mock these, tbh
@Yee looking at this some more, I wonder if the problem isn't how that
handle
function is built in our internal libraries - it looks like the
RemoteEntity
returned by
fetch_task
should work with
flytekit.testing.patch
right?
y

Yee

06/30/2023, 4:02 PM
yes it should.
let me spend some time today looking.
j

John Walk

06/30/2023, 4:03 PM
yeah I think we have one too many layers of abstraction, where our internal lib's wrappers are returning the result of calling the RemoteEntity rather than returning the entity itself, which then would work with the patching
y

Yee

06/30/2023, 4:09 PM
let me get the patching working for remote entities first, demonstrate that basic unit tests work
j

John Walk

06/30/2023, 4:10 PM
👍
y

Yee

07/05/2023, 10:07 PM
@John Walk reviewed the testing code in flytekit. it does look like it’s built currently just for non-fetched tasks. but i’m not sure that really matters…
i’ll add an example of how you might test a fetched task - https://github.com/flyteorg/flytekit/pull/1722/files
you can avoid using the testing stuff in flytekit completely.
see the
test_mocking_remote
test
j

John Walk

07/06/2023, 3:54 PM
ok yeah, that squares with my thinking - we've just got the actual remote task buried under one too many layers of abstraction
reckon I could try building a patcher functionality for our wrappers using the logic you've got there
ok yeah I was able to get it working with patching
fetch_remote
within our internals but that required defining a mock task with matching kwargs for each remote task, which is a little gnarly
would be nicer if we could find a way to
task_mock
the RemoteEntity it grabs like you've got in yours
hah, actually seems like we get the same effect if we just make a mock task and set that as a side effect of the mocked parent function. same problem with the expected arguments though
y

Yee

07/06/2023, 8:06 PM
yeah there’s no way around the expected arguments. that feeds the interface and the interface needs to be correctly defined.
we’ll merge that pr today
j

John Walk

07/06/2023, 8:07 PM
that feeds the interface and the interface needs to be correctly defined.
yeah, unless we can get at mocking the internal RemoteEntity reckon we're stuck with it
71 Views