Hello, can any one tell me how to import datasets ...
# ask-the-community
k
Hello, can any one tell me how to import datasets into demo container?
p
Hello! You should be able to navigate to the object store at
<http://localhost:30080/minio/browser>
, which has a nice interface that allows you to upload files/objects
k
thank you, but how to import that into flyte, what the path should look like
k
What do you mean import
Flyte can access anything from an object store or https etc
p
I actually wrote something about this, see the get_dir task here: https://gist.github.com/pryce-turner/0a67f86febdc812c9a2a9e739c22eeca You use the s3 path (s3://my-s3-bucket/blah) when creating a FlyteFile/FlyteDirectory
k
I have a csv file in a s3 bucket, how to read that csv inside task and return pandas dataset
k
Copy code
@task
def t1(a: FlyteFile) -> pd.DataFrame:
  # read csv file
  return pd.read_csv(a) # flyte will download csv file from s3, and read it
102 Views