https://flyte.org logo
#flyte-deployment
Title
# flyte-deployment
j

Jan Fiedler

11/30/2023, 2:16 PM
👋 Flyte Crew 🙂 I am currently playing around with an on premise Flyte Setup. While getting it to run with minio, i questioned my understanding of the whole process what happens when i use
pyflyte register
. My understanding was: hitting pyflyte register --> submitting the workflow artifacts over grcp to flyteadmin --> flyteadmin stores the artifacts in the object storage (minio in my case) --> done. (Is this correct ?)
d

David Espejo (he/him)

11/30/2023, 4:08 PM
HI Jan, yes. When you use "fast registration" the packaged protobuf files, which are the result of the serialization done by FlyteIDL, are then stored on the blob storage bucket that flyteadmin uses
j

Jan Fiedler

11/30/2023, 4:18 PM
Ah ok so flyteadmin is not the instance storing the packaged protobuf files but the FlyteIDL? How does the FlyteIDL retrieved the credentials to the object store than?
d

David Espejo (he/him)

11/30/2023, 4:20 PM
FlyteIDL is not a standalone component, it's part of flyteadmin
j

Jan Fiedler

11/30/2023, 4:24 PM
Got it! So it is Flyteadmin in the end. Than my main question i wanted to come up with is the following: In this on premise setup i experienced, that if i want to register a workflow on another local machine, it failed looking for the configured minio. Only when i port forward the minio on the k8 cluster it works. I was wondering why i need minio network access from this another machine were i want to register the workflow?
d

David Espejo (he/him)

11/30/2023, 4:28 PM
if you move to another machine and want to register to the flyteadmin instance you already deployed, you'll still need access to the bucket to upload the protobuf files. This is true also if your K8s cluster has multiple node machines
j

Jan Fiedler

11/30/2023, 4:30 PM
But may i ask why this is? If flyteadmin is taking care of the upload and lives on the same cluster as the minio?
d

David Espejo (he/him)

11/30/2023, 4:42 PM
in the case of fast-registration, it's the client who will be uploading the compiled artifacts
j

Jan Fiedler

11/30/2023, 4:46 PM
Sry for not stop asking questions but how does the client retrieves like f.e. minio credentials like access-key and secret? Does flyteadmin sends it in some way?
d

David Espejo (he/him)

11/30/2023, 5:00 PM
Sry for not stop asking questions
No need to sorry, happy to help!
Does flyteadmin sends it in some way?
I don't think the control plane sends this to the client but its read from the config. I'm digging in the code
j

Jan Fiedler

11/30/2023, 5:25 PM
Would be really cool to know how this works in detail 🙂
k

Ketan (kumare3)

12/02/2023, 1:06 PM
You do need access to the bucket on the client that is tunneled through admin using signed urls
j

Jan Fiedler

12/03/2023, 11:30 AM
Ah signed URL’s it is - makes a lot of sense. Thank you
2 Views