n00b python / flytekit question here! I'm curious about the part of flyte that will take a base docker image, build an image for the specific run with the current code, and then push the image to the docker registry for running later. couple questions: • i found there's this bash script template: https://github.com/flyteorg/flytekit/blob/master/flytekit_scripts/flytekit_build_image.sh. But is this deprecated or are users supposed to use it? • i also see there is

pyflyte build

which I think is perhaps the more recent / canonical code path to use? https://github.com/flyteorg/flytekit/blob/1b35b091210f541d69c95ae2d1bc1e8f89814c8e/flytekit/clis/sdk_in_container/build.py#L29. ◦ related question: can a user somehow do a flytekit

build

and

run

programmatically? • The hello world ( https://docs.flyte.org/en/latest/user_guide/basics/hello_world.html ) talks about running locally and I see there's a small part about

pyflyte run --remote

in the "running a workflow locally" section ( https://docs.flyte.org/en/latest/user_guide/getting_started_with_workflow_development/running_a_workflow_locally.html#id2 ) ... but does

pyflyte run --remote

do the docker build and push stuff implicitly in this part of the doc? Assuming the workflow has an

ImageSpec

Lastly and separately, if i were to start up flytekit sandbox, would that let me run workflow concurrently just on the one machine where I have the sandbox? E.g. i could kick off 10 workflows and maybe they all run concurrently?

the build_image.sh is deprecated afaik. INstea dyou'd use imagespec https://docs.flyte.org/en/latest/user_guide/customizing_dependencies/imagespec.html To run locally you just use

pyflyte run

without the

--remote

flag.

thank you @gentle-tomato-480! @rapid-artist-48509 > if i were to start up flytekit sandbox, would that let me run workflow concurrently just on the one machine where I have the sandbox? Correct, sandbox is intended for demo/evaluation and while it supports concurrency, is largely limited by the local Docker (OCI) daemon resources. > i could kick off 10 workflows and maybe they all run concurrently? There are different options for concurrency: • At the task level, you can use

MapTask

to concurrently map over a set of inputs without bootstraping a container(Pod) per task. (docs) • Launch Plans: this is a construct that lets you parametrize workflow executions. Every workflow comes with a default launchplan and you could adjust schedules on each plan so they run concurrently (docs) • Union comes with `Actors`which creates a long-running execution environment for concurrent executions without the penaly of container boot time. Combined with MapTasks it's much much faster (docs) Does that answer your question or was too much?😬

@average-finland-92144 sweet thank you for the details!! oh sweet I didn't know Flyte had extended concurrency support, i really just need basic for now tho. returning to the part that "builds local stuff into a docker image for remote execution" ... is that perhaps a launch plan then? e.g. i grok that for pure-python there's one way to do it, then for python + custom docker another way, then if you even had native code or custom build step there would be another way ... or is this all wrapped up into

package

perhaps? https://github.com/flyteorg/flytekit/blob/1b35b091210f541d69c95ae2d1bc1e8f89814c8e/flytekit/clis/sdk_in_container/package.py#L127 lastly, is

flytekit.clis.sdk_in_container

a module that third-party code should be using, or is it supposed to be more hidden / just for the pyflyte CLI ?

I think it's much simpler, you can just use

ImageSpec

and provide arguments and then the docker image will be built for you, including the declared dependencies https://github.com/flyteorg/flytekit/blob/1b35b091210f541d69c95ae2d1bc1e8f89814c8e/flytekit/image_spec/image_spec.py#L29 Docs Does that help?

right, ok let me just look at some code paths that actually use

ImageSpec

. thank you! returning to

flytekit.clis.sdk_in_container

real quick, is that mainly just for flyte CLI, or if say I had my own cli / python library that I want to effectively wrap

pyflyte

invocations, could i use that module?