Channels
announcements
ask-the-community
auth
conference-talks
contribute
databricks-integration
datahub-flyte
deployment
ecosystem-unionml
engineeringlabs
events
feature-discussions
flyte-bazel
flyte-build
flyte-console
flyte-deployment
flyte-documentation
flyte-github
flyte-ui-ux
flytekit
flytekit-java
flytelab
great-content
hacktoberfest-2022
helsing-flyte
in-flyte-conversations
integrations
introductions
jobs
konan-integration
linkedin-flyte
random
ray-integration
ray-on-flyte
release
scipy-2022-sprint
show-and-tell
sig-large-models
torch-elastic
workflow-building-ui-proj
writing-w-sfloris
Title
k
krishna Yerramsetty
06/21/2022, 5:31 PMHi all. I have been trying Flyte for the past week or so and I love it! I am new to ML Ops style tools and workflows, so I don't think I get all of the details yet. Question on the sandbox?: Is there a way to download the task outputs to the sandbox container? Right now my outputs for a task that creates a directory is this:
{1 item
o0:{2 items
type:"multi-part blob"
uri:"<s3://my-s3-bucket/hi/ag5dp94bs4c846cn847s-n0-0/628c0053bbedf3f1db4d44362f48857e>"
}
}❤️ 3
k
katrina
06/21/2022, 7:38 PMhi @krishna Yerramsetty what are you trying to do with the outputs? you can view them in the console or alternatively use the remote client for programmatic access
k
krishna Yerramsetty
06/21/2022, 8:01 PMThanks @katrina! I am trying to check if the files have the right info in them. Can I use the remote client if I am running on a sandbox locally?
And the ouputs of these tasks are of type FlyteFile, so only their paths show up on the console
k
katrina
06/21/2022, 8:44 PMyup you can absolutely use flytekit remote with the local sandbox! just initialize the client with the path to your sandbox flytekit config (usually
~/.flyte/config.yamlin fact i think the .auto() mode will pick this up for you
i believe you can also use minio (the sandbox blob store) browser to browse the files too if you'd like
k
Ketan (kumare3)
06/22/2022, 1:10 AMSo all intermediate outputs will reside in the configured bucket / s3/gcs or minio in sandbox case
👍 1
k
krishna Yerramsetty
06/22/2022, 5:59 PMThank you, @katrina and @Ketan (kumare3)! I like the remote client. Giving it a shot now
👍 1
So, I was able to figure out where the data is on the minio pod. Thanks for pointing me in that direction! The
remote_client.outputsk
Ketan (kumare3)
06/22/2022, 10:22 PMit should retrieve if you open - cc @Yee?
y
Yee
06/22/2022, 11:03 PM(pinged on dm @krishna Yerramsetty but we can continue here too if you want)