https://flyte.org
Join Slack
Hi all. I have been trying Flyte for the past week or so and I love it! I am new to ML Ops style too...
# flyte-support
s
Hi all. I have been trying Flyte for the past week or so and I love it! I am new to ML Ops style tools and workflows, so I don't think I get all of the details yet. Question on the sandbox?: Is there a way to download the task outputs to the sandbox container? Right now my outputs for a task that creates a directory is this:
Copy code
{1 item
o0:{2 items
type:"multi-part blob"
uri:"<s3://my-s3-bucket/hi/ag5dp94bs4c846cn847s-n0-0/628c0053bbedf3f1db4d44362f48857e>"
}
}
Not sure where this multi-part blob resides
❤️ 3
a
hi @sparse-addition-80036 what are you trying to do with the outputs? you can view them in the console or alternatively use the remote client for programmatic access
s
Thanks @acceptable-policeman-57188! I am trying to check if the files have the right info in them. Can I use the remote client if I am running on a sandbox locally?
And the ouputs of these tasks are of type FlyteFile, so only their paths show up on the console
a
yup you can absolutely use flytekit remote with the local sandbox! just initialize the client with the path to your sandbox flytekit config (usually 
~/.flyte/config.yaml
) and you should be good to go
in fact i think the .auto() mode will pick this up for you
i believe you can also use minio (the sandbox blob store) browser to browse the files too if you'd like
f
So all intermediate outputs will reside in the configured bucket / s3/gcs or minio in sandbox case
👍 1
s
Thank you, @acceptable-policeman-57188 and @freezing-airport-6809! I like the remote client. Giving it a shot now
👍 1
So, I was able to figure out where the data is on the minio pod. Thanks for pointing me in that direction! The 
remote_client.outputs
only shows the path to the files in the minio storage pod and I don't think it retrieves/downloads the files onto the local machine (that hosts the sandbox deployment). Is that correct? Is there a way to use flytekit or flytectl to interact with the minio storage files directly
f
it should retrieve if you open - cc @thankful-minister-83577?
t
(pinged on dm @sparse-addition-80036 but we can continue here too if you want)
162 Views
Open in Slack
PreviousNext
Powered by

Flyte

Flyte enables production-grade orchestration for machine learning workflows and data processing created to accelerate local workflows to production.

Powered by