@ancient-wolf-19325 I have some tasks typed with a List[Union[str,datetime]] but running into this error, how would i type hint and pass lists around? or is passing lists around from task to task not supported Type of Generic List type is not supported, Transformer for type <class 'tuple'> is restricted currently

Passing lists between tasks is supported in Flyte, but only for univariate (single-type) lists, e.g., Liststr] or List[datetime]. Flyte does not support lists with Union types like List[Union[str, datetime]]—this will result in a "Type of Generic List type is not supported" error, as only univariate typing.List[T] is supported by the type engine ([source, source). To work around this, you should use a list of a single type, or if you must mix types, consider using a custom dataclass or serialize your data as strings and handle type conversion within your tasks. Would you like more detail or example code for these workarounds? Sources: - flytekit/core/type_engine.py - BUG: List subtype check for remote task compilation does not work 👍 1 button 👎 0 button 💡 Hint: Mention @ancient-wolf-19325 in the thread for followups.

@ancient-wolf-19325 what about Optional[List[str]], im still getting the error after paring down my function to only use str instead of both str and datetime

Support for OptionalList[str]] is available in recent versions of Flyte, but older versions of flytekit had issues with dataclasses or task inputs using Optional[List[T]]—this caused errors like the one you described ([source, source). This was fixed in flytekit 1.15.0 and later. If you are still seeing errors, ensure you are using flytekit >=1.15.0. If you are, and the error persists, please provide the exact code and error message for further diagnosis. Would you like more detail or upgrade instructions? Sources: - https://github.com/flyteorg/flyte/issues/2953 - https://github.com/flyteorg/flyte/issues/6057 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.

@ancient-wolf-19325 i have flytekit version 1.15.3 on both my local environment and the image, why would i still be getting this error

If you're using flytekit 1.15.3 and still getting "Type of Generic List type is not supported" with OptionalList[str]], it's possible that your code or type hints are not being interpreted as expected, or there may be a mismatch in how the type is declared or passed ([source). Also, ensure you are not using nested optionals or unions (e.g., OptionalDict[str, List[str] | None]]), as these can still cause issues even in recent versions ([source). Please double-check that your task signature is exactly Optional[List[str]] and that you are not passing None or an empty list in a way that could confuse the type engine. Would you like to share your task definition and call for more targeted troubleshooting? Sources: - https://github.com/flyteorg/flyte/issues/6057 - https://github.com/flyteorg/flyte/issues/6474 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.

@ancient-wolf-19325 here's what i am using for my type hints

def list_raw_files(dates_to_rerun: Optional[List[str]] = None) -> List[FileInfo]:

and also

def process_files_in_batches(databricks_files: List[FileInfo]) -> List[FileInfo]:

Your type hints OptionalList[str]] and List[FileInfo] are both supported by Flyte, provided FileInfo is a dataclass or a supported type ([source). If you are still getting the "Type of Generic List type is not supported" error, check that FileInfo is not itself a generic or contains unsupported types (like Union or nested Optionals), as this can cause the type engine to fail (source). If FileInfo is a custom dataclass, ensure it only uses supported field types (e.g., str, int, float, bool, List[T], Dict[str, V], etc.) and does not use Union or Any. Would you like to share the definition of FileInfo for further troubleshooting? Sources: - https://github.com/flyteorg/flyte/issues/6474 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.

@ancient-wolf-19325 it seems like fileinfor is a namedtuple, is that not supported?

FileInfo(namedtuple("FileInfo", ["path", "name", "size", "modificationTime"])

NamedTuple is only supported as a return type for multiple outputs in Flyte tasks and workflows, not as an input or as a type for passing data between tasks. Using NamedTuple as an input or in a List will cause the "Transformer for type class 'tuple' is restricted currently" error (source, source). To pass structured data between tasks, use a @dataclass instead of NamedTuple for FileInfo. Would you like an example of how to convert your NamedTuple to a dataclass for Flyte? Sources: - https://github.com/flyteorg/flyte/issues/1337 - https://github.com/flyteorg/flyte/issues/2720 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.

@ancient-wolf-19325 yes please, an example would be great

Certainly! Replace your NamedTuple with a dataclass like this:

from dataclasses import dataclass
from typing import List, Optional

@dataclass
class FileInfo:
    path: str
    name: str
    size: int
    modificationTime: str

def list_raw_files(dates_to_rerun: Optional[List[str]] = None) -> List[FileInfo]:
    # Example implementation
    return [FileInfo(path="/tmp/a", name="a", size=123, modificationTime="2024-06-01")]

def process_files_in_batches(databricks_files: List[FileInfo]) -> List[FileInfo]:
    # Example implementation
    return databricks_files

This dataclass approach is fully supported for passing lists of FileInfo between Flyte tasks (source). Would you like more detail or a Flyte task/workflow example? Sources: - https://www.union.ai/docs/flyte/user-guide/data-input-output/dataclass/ 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.

Hi Kevin, I'm facing a very similar issue. Were you able to resolve it in the end?

Hi An - I was able to using the example they provided, so i made a function to convert the problematic type hinted class to a dataclass, added it to a task and had it output the dataclass, then used that dataclass as the typehint & passed objects of that dataclass through the workflow without any problem