https://flyte.org logo
Powered by
n

Nada Saiyed

06/23/2022, 11:43 PM
hello.. new flyte user here.. had a question about how can i configure different tasks to use different images in a workflow? i am aware that i can do this by specifying
Copy code
@task(
    container_image="{{.image.img1.fqn }}:{{.image.img1.version}}"
)
and the value of 
image.img1.version
can be specified in a config file.. but how is that config linked to tasks while packaging the workflow?
y

Yee

06/23/2022, 11:48 PM
how are you running the packaging flow @Nada Saiyed?
you should be able to pass a config file to your pyflyte run/package/register command.
you should also be able to specify additional images to those commands on the command line as options
n

Nada Saiyed

06/23/2022, 11:49 PM
pyflyte --pkgs workflows package --image "my-flyte:v1" -f
y

Yee

06/23/2022, 11:49 PM
after registering, run the workflow, and click on the corresponding nodes.
on the right-hand side bar that pops up, click on the task tab. that should bring up the json definition.
the image should be hard-coded there
n

Nada Saiyed

06/23/2022, 11:50 PM
pyflyte --pkgs workflows --config workflows/my.config package -f
would be enough?
y

Yee

06/23/2022, 11:51 PM
mind dumping the config here too? (can redact if you need to)
n

Nada Saiyed

06/23/2022, 11:51 PM
Copy code
[sdk]
workflow_packages=core

[images]
task1 = nsaiyed-flyte:v1
task2 = nsaiyed-flyte:v2
also what is this for?
Copy code
[sdk]
workflow_packages=core
y

Yee

06/23/2022, 11:51 PM
i think you can delete that
yeah that should be fine.
the config should be respected, and the images should get picked up
you’re following the flytesnacks example right?
n

Nada Saiyed

06/23/2022, 11:52 PM
yep
y

Yee

06/23/2022, 11:52 PM
sweet
hopping off for now, will be back on later.
n

Nada Saiyed

06/23/2022, 11:56 PM
i get this error
Copy code
containers with unready status: [ajpsqhm6rzqws2tdrcls-n0-0]|Failed to apply default image tag "{{.image.task1.fqn }}:{{.image.task1.version}}": couldn't parse image reference "{{.image.task1.fqn }}:{{.image.task1.version}}": invalid reference format
tried to specify the config file while serializing the workflow as well, but no luck 
pyflyte --pkgs workflows serialize --in-container-config-path workflows/my.config workflows -f .
y

Yee

06/24/2022, 3:19 AM
no you shouldn’t need it for the in-container path.
this is weird.
ooh i’m sorry yeah the config file is not respected
😞
we will have to add that, for now you have to use the command line
✌️ 1
👍 1
this syntax
Copy code
pyflyte -c tst.config --pkgs core.flyte_basics package --image a.com/b/c:latest --image task1=nsaiyed-flyte:v1 -f
the first one doesn’t have a name and is the default
n

Nada Saiyed

06/24/2022, 3:56 AM
is passing a default image name mandatory? i am guessing no
i also tried passing it via command line.. but i keep getting the same error
also if i register using the above command you gave as an example, i am not able to trigger the workflow 😞 i get this error 
Request failed with status code 500 failed to create workflow in propeller Unauthorized
y

Yee

06/24/2022, 4:13 AM
hmm. i think so actually - the default bit.
👍 1
at least we typically always include it, so not sure what the behavior is if you don’t. can always use a dummy image
👍 1
the other bit is weird though
should not have a 500
don’t suppose you have access to the cluster and can dump out the admin logs
we should improve that error message.
n

Nada Saiyed

06/24/2022, 4:23 AM
my cluster is the sandbox cluster.. and killing the container and restarting it seems to have fixed it
y

Yee

06/24/2022, 4:23 AM
Nice
Turn it off and on again.
😛 1
n

Nada Saiyed

06/24/2022, 4:26 AM
but still specifying 
@task(container_image="{{.image.task2.fqn }}:{{.image.task2.version}}")
throws an error
Copy code
containers with unready status: [axxmntxh2dpm6d742sdp-n0-0]|Failed to apply default image tag "{{.image.task1.fqn }}:{{.image.task1.version}}": couldn't parse image reference "{{.image.task1.fqn }}:{{.image.task1.version}}": invalid reference format
well.. for now i will resort to giving the image name directly since its not able to parse image reference
Copy code
@task(container_image="nsaiyed-flyte:v1")
Thanks a lot for your help
y

Yee

06/24/2022, 5:11 AM
let’s do a video call tomorrow.
will be easier to debug over a screenshare
n

Nada Saiyed

06/24/2022, 2:21 PM
sure.. what time?
y

Yee

06/24/2022, 4:04 PM
you free now?
n

Nada Saiyed

06/24/2022, 4:04 PM
yep
y

Yee

06/24/2022, 4:04 PM
sweet
Copy code
flyte-cli parse-proto -f fdsa.pb -p flyteidl.admin.task_pb2.TaskSpec
n

Nada Saiyed

06/24/2022, 5:03 PM
i have the setup ready if you are available for a call?
y

Yee

06/24/2022, 5:30 PM
hey just got in
are you still free?
n

Nada Saiyed

06/24/2022, 5:30 PM
yes
y

Yee

06/24/2022, 5:30 PM
hop back on?
👍 1
tar -xfzv flyte-package.tgz
n

Niels Bantilan

06/29/2022, 7:35 PM
@Marc Paquette @Smriti Satyan @Samhita Alla we should document how to actually link a 
flyte.config
file to a flyte workflow when serializing and registering in this page: https://docs.flyte.org/projects/cookbook/en/latest/auto/core/containerization/multi_images.html
👀 1
s

Samhita Alla

06/30/2022, 4:47 AM
@Niels Bantilan, since the image config is not being respected, what would we be using 
flyte.config
for?
s

Smriti Satyan

06/30/2022, 5:02 AM
@Nada Saiyed Hi, in the config file, while providing image name, are you using underscore? Like image_name or something? That might be the issue..
n

Nada Saiyed

06/30/2022, 3:14 PM
@Smriti Satyan No. example config here
7 Views
Powered by