<#3207 [Core feature] Convert List[Any] to a singl...
# flytekit
a
#3207 [Core feature] Convert List[Any] to a single pickle file Issue created by pingsutw Motivation: Why do you think this is important? Currently, flyte create N (size of list) pickle files if output type is
List[Any]
. This slows down serialization. it takes more than 15 mins to upload the pickles to s3 if the size of list is 1000. People don't care about how we serialize
List[Any]
. We can just convert entire list into a single pickle file, which reduces the time required for serialization. Goal: What should the final outcome look like, ideally? it will make serialization faster Describe alternatives you've considered • Raise an error when using large list • Add a detailed warning Propose: Link/Inline OR Additional context Slack Thread Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte