https://flyte.org logo
#ask-the-community
Title
# ask-the-community
f

Frank Shen

01/06/2023, 10:40 PM
Run it local is necessary before you can deploy it to remote, isn’t it?
y

Yee

01/06/2023, 10:45 PM
it’s not strictly necessary, but helpful for testing.
but only if it works of course
you’ve confirmed you have local access to the s3 buckets you want to pull from?
like you can
aws s3 ls/cp …
from your laptop?
f

Frank Shen

01/06/2023, 10:47 PM
yes, the following code works.
Copy code
df = pandas.read_parquet("<s3://bucket/path/test_data_0_0_0.parquet>")
It’s just not scalable because it only reads 1 file at a tile.