Hello, can any one tell me how to import datasets ...
# ask-the-communityk
Khalil Almazaideh
05/13/2023, 4:46 PMHello, can any one tell me how to import datasets into demo container?
p
Pryce
05/14/2023, 12:46 AMHello! You should be able to navigate to the object store at
<http://localhost:30080/minio/browser>k
Khalil Almazaideh
05/14/2023, 1:59 PMthank you, but how to import that into flyte, what the path should look like
k
Ketan (kumare3)
05/14/2023, 6:15 PM
What do you mean import
Ketan (kumare3)
05/14/2023, 6:15 PM
Flyte can access anything from an object store or https etc
p
Pryce
05/14/2023, 7:05 PMI actually wrote something about this, see the get_dir task here: https://gist.github.com/pryce-turner/0a67f86febdc812c9a2a9e739c22eeca
You use the s3 path (s3://my-s3-bucket/blah) when creating a FlyteFile/FlyteDirectory
k
Khalil Almazaideh
05/15/2023, 8:37 AMI have a csv file in a s3 bucket, how to read that csv inside task and return pandas dataset
k
Kevin Su
05/15/2023, 10:20 PM Copy code
@task
def t1(a: FlyteFile) -> pd.DataFrame:
# read csv file
return pd.read_csv(a) # flyte will download csv file from s3, and read it153 Views