Hi all, I have a project where a simulation SIL is...
# ask-the-communityt
Tim Hebbeler
02/12/2023, 4:11 PMHi all, I have a project where a simulation SIL is run and exported signals from the simulation are processed in python. The simulation happens inside a container. The simulation container is exchanged more frequently than the python scripts are adapted. I read in the Flyte documentation that the task is hard-wired to a container image upon registration. Is it possible to define a container image as input for a task? So that someone can specify the path to the container e.g. within the Flyte GUI? Thank you in advance for your answers
k
Ketan (kumare3)
02/12/2023, 4:44 PM
It is indeed possible - but you will have to use dynamic workflows
Ketan (kumare3)
02/12/2023, 4:45 PM
Firstly welcome to the community and thank you for trying out flytes
t
Tim Hebbeler
02/12/2023, 5:18 PMMany thanks for the extremely fast reply and the friendly acceptance into the community. I might have one more question. The simulation consists of a simulator and the simulation code, compiled as a library. Would it be good to implement the simulation via dynamic workflows or would it make more sense to only provide the simulator as an image and the simulation libraries as a zip file via an input for the task
k
Ketan (kumare3)
02/12/2023, 6:08 PM
You are lost welcome.
If you have Python bindings my recommendation would be to use flytekit to implement it as Flyte native workflows.
Flyte manages to zip and install things to run on new versions - you get a good clean versioning story.
But I do not know your architecture to comment. Why is simulator code a container image? How often what changes the base simulator or the usage of the simulator. Understanding this will
Help us better help you
Ketan (kumare3)
02/12/2023, 6:08 PM
But I also understand that the his could be proprietary
t
Tim Hebbeler
02/12/2023, 7:56 PMCurrently everything is bundled inside one image, simulator and simulation model (sorry i think simulation code was not clear). The simulation model like an FMU and simulator are written in C/C++. We want to provide a workflow in which users can specify the FMU version as input via a GUI. The signals of the FMU will later be analyzed in python. We think Flyte's idea is great that each workflow is versioned and thus each user can work independently
Tim Hebbeler
02/20/2023, 6:09 PM@Ketan (kumare3)
can you give me a hint how to set the container_image dynamically? I tried to instantiate the task decorator in the function but i don‘t think the task resolver „DefaultTaskResolver“ works like that.
Error inside property self.task_resolver.location
python_auto_container.py:151
Copy code
Error looking for LHS in __main__k
Ketan (kumare3)
02/20/2023, 7:12 PM
You will have to use dynamic workflows / tasks
t
Tim Hebbeler
02/20/2023, 7:20 PMI did. I found a solution
Copy code
my_task._container_image = ‘image‘k
Ketan (kumare3)
02/20/2023, 7:27 PM
We can add that as a method
Ketan (kumare3)
02/20/2023, 7:27 PM
The problem is container image is a fixed property to maintain reproducibility
Ketan (kumare3)
02/20/2023, 7:27 PM
But you can also use raw container task
Ketan (kumare3)
02/20/2023, 7:27 PM
Ohh there is one
Ketan (kumare3)
02/20/2023, 7:27 PM
Using your own custom resolver
Ketan (kumare3)
02/20/2023, 7:27 PM
But that is involved
Ketan (kumare3)
02/20/2023, 7:28 PM
It seems you have a raw container
Ketan (kumare3)
02/20/2023, 7:28 PM
Can I ping you later- I am infact out today - it’s a holiday
Ketan (kumare3)
02/20/2023, 7:28 PM
With family now
t
Tim Hebbeler
02/20/2023, 7:44 PMSure, have a nice holiday. I will take a look a raw containers. Thanks a lot for the hint
168 Views