https://flyte.org logo
#ask-the-community
Title
# ask-the-community
c

Cody Scandore

08/19/2023, 6:13 PM
Flyte will persist a dataframe (parquet) on remote storage such as S3 by passing the
uri
param.
Copy code
def load_data() -> StructuredDataset:
    return StructuredDataset(dataframe=pd.read_csv("..."), uri="<s3://my-bucket/my-dataframe.parquet>")
However, this results in partitions nested under
my-dataframe.parquet
. Is there a way to directly save without the partition structure.
Copy code
|- my-bucket
|  |- my-dataframe.parquet
|  |  |- 0000
k

Kevin Su

08/19/2023, 8:34 PM
0000 is default file name, flytekit didn’t create partition by default
2 Views