cool-lifeguard-49380
09/15/2023, 4:38 PMcool-lifeguard-49380
09/15/2023, 4:38 PMfrom flytekit import task, workflow
@task
def test(inp: tuple) -> tuple:
return inp
@workflow
def wf():
test(inp=(1, 2, 3))
if __name__ == "__main__":
wf()
cool-lifeguard-49380
09/15/2023, 4:39 PMRestrictedTypeError: Transformer for type <class 'tuple'> is restricted currently
broad-monitor-993
09/15/2023, 5:35 PMhigh-park-82026
full-ram-17934
09/15/2023, 6:21 PMhigh-park-82026
full-ram-17934
09/15/2023, 6:23 PMThe current struggle is that named tuples are used to define named outputs... I think it won't be obvious how to distinguish between those and these...I meant to make whatever distinction is needed here
high-park-82026
@task
def test(inp: tuple) -> namedtuple("a", "b"):
return inp
@task
def test2(inp: tuple) -> namedtuple("a", "b"):
return inp
How to tell if the user wanted a single output that's of type tuple or two outputs a
and b
full-ram-17934
09/15/2023, 6:24 PMfull-ram-17934
09/15/2023, 6:25 PMhigh-park-82026
cool-lifeguard-49380
09/16/2023, 9:01 AMfrom flytekit import task, workflow
@task
def test() -> tuple[str, str, tuple[str, str]]:
return "foo", "bar", ("foo-inner", "bar-inner")
@workflow
def wf() -> tuple[str, str, tuple[str, str]]:
return test()
if __name__ == "__main__":
print(wf())
I think it would be ok to treat the “outer or main” DefaultNamedTupleOutput
return value (that is always returned when there are multiple return values) with the existing logic and not with a potential new tuple type transformer but invoke a potential tuple type transformer only for the “inner” tuple here.
Of course it would be nice to have a general solution but I think it’s more important that users can do what is shown in the code snippet.
---
One could argue that a tuple type transformer isn’t very important, one could just return e.g. a list instead.
The problem is that changing the tuple to a list in the example above requires a user to understand that the type engine exists in the first place and what its limitations are.
The type engine works so well that most users at least early on don’t realize its there. I onboarded ~30 people onto Flyte at my previous and current company and would say that there are 3 typical situations where people first learn that the type engine exists. 1) When they try to return tuples, 2) when they put an int into a dict and get a dict with a float on the other side (maybe fixed now?) and 3) when I show them how to build a custom type transformer.
They typically find 3) super cool but also typically are (maybe unreasonably) frustrated when learning about 1 and 2. Something like “returning a tuple is the most basic python thing, why doesn’t even this work?”
I guess when something works well, it doesn’t cause gratefulness but creates the expectation that it works perfectly 🤷♂️full-ram-17934
09/18/2023, 7:33 PMhigh-park-82026
@task
def task() -> int, str:
...
# and:
@task
def task2() -> tuple[int, str]:
...
I think the "right" thing (Perhaps not practical) to do here is perhaps to always treat outputs as "one" output that's indexable (much like what python does)... Support is being added to flyte for indexing into nested fields, right, @cool-lifeguard-49380?cool-lifeguard-49380
09/27/2023, 7:46 AMcool-lifeguard-49380
09/27/2023, 7:46 AMcool-lifeguard-49380
09/27/2023, 7:46 AMhigh-park-82026