Run it local is necessary before you can deploy it...
# ask-the-communityf
Frank Shen
01/06/2023, 10:40 PMRun it local is necessary before you can deploy it to remote, isn’t it?
y
Yee
01/06/2023, 10:45 PM
it’s not strictly necessary, but helpful for testing.
Yee
01/06/2023, 10:45 PM
but only if it works of course
Yee
01/06/2023, 10:45 PM
you’ve confirmed you have local access to the s3 buckets you want to pull from?
Yee
01/06/2023, 10:45 PM
like you can
aws s3 ls/cp …f
Frank Shen
01/06/2023, 10:47 PMyes, the following code works.
Copy code
df = pandas.read_parquet("<s3://bucket/path/test_data_0_0_0.parquet>")Frank Shen
01/06/2023, 10:48 PMIt’s just not scalable because it only reads 1 file at a tile.
152 Views