https://flyte.org
Join Slack
Hi all, I have a project where a simulation SIL is run and exported signals from the simulation are ...
# flyte-support
b
Hi all, I have a project where a simulation SIL is run and exported signals from the simulation are processed in python. The simulation happens inside a container. The simulation container is exchanged more frequently than the python scripts are adapted. I read in the Flyte documentation that the task is hard-wired to a container image upon registration. Is it possible to define a container image as input for a task? So that someone can specify the path to the container e.g. within the Flyte GUI? Thank you in advance for your answers
f
It is indeed possible - but you will have to use dynamic workflows
Firstly welcome to the community and thank you for trying out flytes
b
Many thanks for the extremely fast reply and the friendly acceptance into the community. I might have one more question. The simulation consists of a simulator and the simulation code, compiled as a library. Would it be good to implement the simulation via dynamic workflows or would it make more sense to only provide the simulator as an image and the simulation libraries as a zip file via an input for the task
f
You are lost welcome. If you have Python bindings my recommendation would be to use flytekit to implement it as Flyte native workflows. Flyte manages to zip and install things to run on new versions - you get a good clean versioning story. But I do not know your architecture to comment. Why is simulator code a container image? How often what changes the base simulator or the usage of the simulator. Understanding this will Help us better help you
But I also understand that the his could be proprietary
b
Currently everything is bundled inside one image, simulator and simulation model (sorry i think simulation code was not clear). The simulation model like an FMU and simulator are written in C/C++. We want to provide a workflow in which users can specify the FMU version as input via a GUI. The signals of the FMU will later be analyzed in python. We think Flyte's idea is great that each workflow is versioned and thus each user can work independently
@freezing-airport-6809 can you give me a hint how to set the container_image dynamically? I tried to instantiate the task decorator in the function but i don‘t think the task resolver „DefaultTaskResolver“ works like that. Error inside property self.task_resolver.location python_auto_container.py:151
Copy code
Error looking for LHS in __main__
f
You will have to use dynamic workflows / tasks
b
I did. I found a solution
Copy code
my_task._container_image = ‘image‘
inside a dynamic task. But doesn‘t look very elegant. Do you have better idea?
f
We can add that as a method
The problem is container image is a fixed property to maintain reproducibility
But you can also use raw container task
Ohh there is one
Using your own custom resolver
But that is involved
It seems you have a raw container
Can I ping you later- I am infact out today - it’s a holiday
With family now
b
Sure, have a nice holiday. I will take a look a raw containers. Thanks a lot for the hint
167 Views
Open in Slack
PreviousNext
Powered by

Flyte

Flyte enables production-grade orchestration for machine learning workflows and data processing created to accelerate local workflows to production.

Powered by