Hi all, I have a project where a simulation SIL is...
# flyte-supportb
breezy-receptionist-39475
02/12/2023, 4:11 PMHi all, I have a project where a simulation SIL is run and exported signals from the simulation are processed in python. The simulation happens inside a container. The simulation container is exchanged more frequently than the python scripts are adapted. I read in the Flyte documentation that the task is hard-wired to a container image upon registration. Is it possible to define a container image as input for a task? So that someone can specify the path to the container e.g. within the Flyte GUI? Thank you in advance for your answers
f
freezing-airport-6809
02/12/2023, 4:44 PM
It is indeed possible - but you will have to use dynamic workflows
freezing-airport-6809
02/12/2023, 4:45 PM
Firstly welcome to the community and thank you for trying out flytes
b
breezy-receptionist-39475
02/12/2023, 5:18 PMMany thanks for the extremely fast reply and the friendly acceptance into the community. I might have one more question. The simulation consists of a simulator and the simulation code, compiled as a library. Would it be good to implement the simulation via dynamic workflows or would it make more sense to only provide the simulator as an image and the simulation libraries as a zip file via an input for the task
f
freezing-airport-6809
02/12/2023, 6:08 PM
You are lost welcome.
If you have Python bindings my recommendation would be to use flytekit to implement it as Flyte native workflows.
Flyte manages to zip and install things to run on new versions - you get a good clean versioning story.
But I do not know your architecture to comment. Why is simulator code a container image? How often what changes the base simulator or the usage of the simulator. Understanding this will
Help us better help you
freezing-airport-6809
02/12/2023, 6:08 PM
But I also understand that the his could be proprietary
b
breezy-receptionist-39475
02/12/2023, 7:56 PMCurrently everything is bundled inside one image, simulator and simulation model (sorry i think simulation code was not clear). The simulation model like an FMU and simulator are written in C/C++. We want to provide a workflow in which users can specify the FMU version as input via a GUI. The signals of the FMU will later be analyzed in python. We think Flyte's idea is great that each workflow is versioned and thus each user can work independently
breezy-receptionist-39475
02/20/2023, 6:09 PM@freezing-airport-6809
can you give me a hint how to set the container_image dynamically? I tried to instantiate the task decorator in the function but i don‘t think the task resolver „DefaultTaskResolver“ works like that.
Error inside property self.task_resolver.location
python_auto_container.py:151
Copy code
Error looking for LHS in __main__f
freezing-airport-6809
02/20/2023, 7:12 PM
You will have to use dynamic workflows / tasks
b
breezy-receptionist-39475
02/20/2023, 7:20 PMI did. I found a solution
Copy code
my_task._container_image = ‘image‘f
freezing-airport-6809
02/20/2023, 7:27 PM
We can add that as a method
freezing-airport-6809
02/20/2023, 7:27 PM
The problem is container image is a fixed property to maintain reproducibility
freezing-airport-6809
02/20/2023, 7:27 PM
But you can also use raw container task
freezing-airport-6809
02/20/2023, 7:27 PM
Ohh there is one
freezing-airport-6809
02/20/2023, 7:27 PM
Using your own custom resolver
freezing-airport-6809
02/20/2023, 7:27 PM
But that is involved
freezing-airport-6809
02/20/2023, 7:28 PM
It seems you have a raw container
freezing-airport-6809
02/20/2023, 7:28 PM
Can I ping you later- I am infact out today - it’s a holiday
freezing-airport-6809
02/20/2023, 7:28 PM
With family now
b
breezy-receptionist-39475
02/20/2023, 7:44 PMSure, have a nice holiday. I will take a look a raw containers. Thanks a lot for the hint
168 Views