Hello, can any one tell me how to import datasets ...
# flyte-supportj
jolly-easter-30455
05/13/2023, 4:46 PMHello, can any one tell me how to import datasets into demo container?
l
loud-belgium-4006
05/14/2023, 12:46 AMHello! You should be able to navigate to the object store at
<http://localhost:30080/minio/browser>j
jolly-easter-30455
05/14/2023, 1:59 PMthank you, but how to import that into flyte, what the path should look like
f
freezing-airport-6809
05/14/2023, 6:15 PM
What do you mean import
freezing-airport-6809
05/14/2023, 6:15 PM
Flyte can access anything from an object store or https etc
l
loud-belgium-4006
05/14/2023, 7:05 PMI actually wrote something about this, see the get_dir task here: https://gist.github.com/pryce-turner/0a67f86febdc812c9a2a9e739c22eeca
You use the s3 path (s3://my-s3-bucket/blah) when creating a FlyteFile/FlyteDirectory
🙇 1
j
jolly-easter-30455
05/15/2023, 8:37 AMI have a csv file in a s3 bucket, how to read that csv inside task and return pandas dataset
g
glamorous-carpet-83516
05/15/2023, 10:20 PM Copy code
@task
def t1(a: FlyteFile) -> pd.DataFrame:
# read csv file
return pd.read_csv(a) # flyte will download csv file from s3, and read it159 Views