Flyte enables production-grade orchestration for machine learning workflows and data processing created to accelerate local workflows to production.

Flyte

Hi all,
just evaluating flyte right now and looks promising for automating and scaling our container based workflows!
I'm wondering if there are plans to add support for local execution of raw containers?

hi <@U03N8LN2SHX> firstly welcome. Thank you for joining

Raw container execution can be done locally, we have not had an issue or anyone asking, but it should not be too much work

please file an issue - you can even contribute :slightly_smiling_face:

it is purely in python and the execute function today just executes the script

We already support local execution now, but it will always return `None` . Do you want to start a docker container locally to run that raw_container task?
Check <https://github.com/flyteorg/flytekit/blob/386626356ff2cb810733a29a51cf3d6351c8424d/flytekit/core/container_task.py#L72-L81>

Thanks for the quick reply!
Yes, exactly since we already have our tasks containerized and they read/write all their data to minio, I would like to just run them locally when building a new workflow or I don't have production cluster set up yet.

Seems like it should be easy to add very basic local execution via subprocess then (assuming docker, etc).

yes, would you like to help us to update the `execution` function?

The demo/sandbox doesn't fit my basic first steps well, as I "just" need to run the containers (images are large and already locally available, but not in sandbox).

sure, I'll see what basic execution I can come up with.

Thank you, as you said we need to use subprocess to launch a docker container to run a task, and try to fetch the task output from that container and return it.

ok, any pointers on where to check how success/error should be reported?

sorry, I think you’re able to use <https://docs.python.org/3/library/subprocess.html#subprocess.Popen.communicate|Popen.communicate> to get output and error. If we got an error, just raise an exception in flytekit

```bashCommand = f"docker run ...."
process = subprocess.Popen(bashCommand.split(), stdout=subprocess.PIPE)
outs, errs = proc.communicate()```

ok, I'll poke around a bit first to get a bit familiar and then start with just raising an exception