Hello, I'd like to use the results of a

map_task

task with a

min_success_ratio

< 1.0 as input to another task. However, as the failed outputs are not filtered out of the results, flytekit is unable to convert them to inputs properly, resulting in errors like this and this. Since this happens in type engine code between tasks, even an intermediate task to filter out none values would fail. This blocks us from using

map_task

effectively in our workflows. Here is a min repro example: https://gist.github.com/sabrinalui/e5478b9557dcf370c84d5e57758b4c87 It seems

min_success_ratio

doesn't work locally but I attached the output when running with inputs

[true,true,false]

from our endpoint below. What would the lift be to (optionally) filter out failed tasks from the

map_task

output?

we append a nilLiteral here, so flytekit failed to deserialize it. @Haytham Abuelfutuh @Dan Rammer (hamersaw) can we remove this empty output?

Then there's no way of tracking which subtask failed right? I'm sure in some cases it doesn't matter, it other it might. Does flytekit support nullable values in any way?

Option

Union

? Would it be possible to declare the maptask output in a way that accepts the

nilLiteral

. Open to discussion on if this is a good idea or not as well.

I just test it with

Option[bool]

, it works for me.

This worked for me. Thanks so much @Kevin Su and @Dan Rammer (hamersaw) 🎉