Flyte enables production-grade orchestration for machine learning workflows and data processing created to accelerate local workflows to production.

Flyte

Run it local is necessary before you can deploy it to remote, isn’t it?

it’s not strictly necessary, but helpful for testing.

you’ve confirmed you have local access to the s3 buckets you want to pull from?

like you can `aws s3 ls/cp …` from your laptop?

yes, the following code works.
```    df = pandas.read_parquet("<s3://bucket/path/test_data_0_0_0.parquet>")```

It’s just not scalable because it only reads 1 file at a time.