Channels
datahub-flyte
scipy-2023-workshop
flyte-school
agent-support-memverge
flyte-build
flyte-users-berlin
scipy-2023-sprint
auth
flyte-bazel
large-language-models
contribute
bioinformatics-computational-biology
great-content
in-flyte-conversations
flyte-on-gcp
show-and-tell
shameless-promotion
linkedin-flyte
random
deployment
hacktoberfest-2023
flyte-github
feature-discussions
linen-test
flytelab
flytekit-java
integrations
ray-on-flyte
conference-talks
release
flyte-ui-ux
workflow-building-ui-proj
writing-w-sfloris
jobs
hacktoberfest-2022
torch-elastic
flyte-console
engineeringlabs
helsing-flyte
flyte-documentation
konan-integration
databricks-integration
ray-integration
wg-gpu-types
flytekit
ecosystem-unionml
scipy-2022-sprint
announcements
ask-the-community
flyte-deployment
introductions
events
Powered by
#ask-the-community
Title
# ask-the-community
f
Frank Shen
01/06/2023, 10:40 PM
Run it local is necessary before you can deploy it to remote, isn’t it?
y
Yee
01/06/2023, 10:45 PM
it’s not strictly necessary, but helpful for testing.
but only if it works of course
you’ve confirmed you have local access to the s3 buckets you want to pull from?
like you can
aws s3 ls/cp …
from your laptop?
f
Frank Shen
01/06/2023, 10:47 PM
yes, the following code works.
Copy code
df = pandas.read_parquet("<s3://bucket/path/test_data_0_0_0.parquet>")
It’s just not scalable because it only reads 1 file at a tile.
Post