cuddly-jelly-27016
05/15/2025, 12:12 AMStructuredDatasets
for example:
schema = StructType([
StructField('name', StructType([
StructField('firstname', StringType(), True),
StructField('middlename', StringType(), True),
StructField('lastname', StringType(), True)
])),
StructField('state', StringType(), True),
StructField('gender', StringType(), True)
])
This allows for enforcing nested schema during compile time for a StructuredDataset
Some datasets are naturally nested and it would be good if flyte could support that. Right now the current workaround is to not enforce schema type or flatten the dataset. But native support from Flyte would be the ideal path forward.
### Goal: What should the final outcome look like, ideally?
Flyte supports nested StructuredDataset
### Describe alternatives you've considered
not enforcing compile time checking and returning the Dataframe as is without StructuredDataset
wrapper.
### Propose: Link/Inline OR Additional context
No response
### Are you sure this issue hasn't been raised already?
• Yes
### Have you read the Code of Conduct?
• Yes
flyteorg/flytecuddly-jelly-27016
05/15/2025, 12:12 AM