Hi Team, I’d like to understand a little more about best practice when it comes to a larger project and building containers. Ideally I could have a CICD pipeline that builds the containers needed and packages and registers the workflows. The trouble I see is that the containers seem to need the task & workflow code copied into them, but the task and workflow code needs to refer to the image by tag (including version). So, when I build the container and

COPY

the tasks into it, I won’t know the version of the image to refer to in the task decorators. It’s a little bit circular, isn’t it? Using

latest

tag wont work in my org because in prod we always used pinned container versions. Plus it seems like that might make it harder to reproduce older runs once the latest tag moves. I must be missing something, how have you worked around this?

Hi Joe, welcome to the community! Our build pipelines currently register Flyte workflows on each commit, which should roughly be what you want. Our approach is roughly: 1. Determine the version of the build. For commits on feature branches we usually use

<branch-name>-<short-commit-hash>

. For commits on main we’re using `main-<short-commit-hash>. For released (tagged) versions we’re usually using the version number. 2. Build and tag and push docker image based on the version from step (1) 3. Serialize and register the Flyte workflows based on the version from (1) and the docker image from (2). This step is dependent on the success of step (2). Depending on whether it’s a feature branch, etc. we’re adjusting the domain to register to accordingly

That’s what I want! I think I’m missing something though. In your stage 3, how can you have individual tasks referring to different images when changing the version of the image in the decorator will change the git hash?

Unless you’ve got a special case, you don’t need to set the image in the decorator, but can use the

--image

option whilst serializing/registering (depends on what command you use?). This would then be the default image used for all your tasks. In case you need the image in the

task

decorator (e.g. in the case of different images in a workflow) - you can follow this doc

Image reference is specified, but the version is derived from the default images version

Thank you for the quick responses! This is exaactly what I was looking for!

@Bernhard Stadlbauer thank you, I think we should put some of this stuff on docs correctly. We are working on redoing flytesnacks like a book. Ccc @Peeter Piegaze / @Samhita Alla