Hello, I have defined a Flyte workflow which calls a few flyte Tasks. All but one tasks return a string, and one task,
ingest_reports
, returns a
List[Dict[str, str]]
. Furthermore in my workflow, I have
>>
operator defining the sequence of task execution.
Now when I test the workflow, I have a couple of questions:
1. When I test the workflow, I want to mock the task calls. Now,
>>
operator defines the sequence of task execution, and it expects the tasks to return a Promise. Since a Promise is expected, so the mocked task should be able to return a Promise, otherwise this operation fails with an error like:
except Exception as exc:
> raise type(exc)(f"Error encountered while executing '{fn_name}':\n {exc}") from exc
E AttributeError: Error encountered while executing 'phishing_reports_workflow':
E 'list' object has no attribute 'ref'
2. To do so I defined the mock as below:
def mock_promise(value: any):
magic_mock_promise = MagicMock(spec=Promise)
mock_output = MagicMock()
mock_output.value = value
magic_mock_promise.output = mock_output
return magic_mock_promise
3. This mocking of tasks to return a FlytePromise works fine as long as task return type is a String. For the task,
ingest_reports
, return type is
List[Dict[str, str]]
. When I try to mock the response of
ingest_reports
using
mock_promise
above, by passing in a list of dict of strings, it fails with exception below:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = Typed List Transforms (<class 'list'>) to Flyte native
ctx = FlyteContext(file_access=<flytekit.core.data_persistence.FileAccessProvider object at 0x103772730>, level=1, flyte_cli...tackframe=<FrameSummary file /Library/Python/3.9/site-packages/flytekit/core/context_manager.py, line 877 in <module>>)
python_val = <MagicMock name='ingest_phishing_reports()' spec='Promise' id='5383586432'>
python_type = typing.List[typing.Dict[str, str]]
expected = <FlyteLiteral collection_type { map_value_type { simple: STRING } }>
def to_literal(self, ctx: FlyteContext, python_val: T, python_type: Type[T], expected: LiteralType) -> Literal:
if type(python_val) != list:
> raise TypeTransformerFailedError("Expected a list")
E flytekit.core.type_engine.TypeTransformerFailedError: Expected a list
/Library/Python/3.9/site-packages/flytekit/core/type_engine.py:1042: TypeTransformerFailedError
Any help on how to solve this?
