Run it local is necessary before you can deploy it to remote

Question

Run it local is necessary before you can deploy it to remote isn t it

Yee · Answer

it s not strictly necessary but helpful for testing

Yee · Answer

but only if it works of course

Yee · Answer

you ve confirmed you have local access to the s3 buckets you want to pull from

Yee · Answer

like you can `aws s3 ls cp ` from your laptop

Frank Shen · Answer

yes the following code works ``` df = pandas read parquet <s3 bucket path test data 0 0 0 parquet> ```

Frank Shen · Answer

It s just not scalable because it only reads 1 file at a tile