Flyte enables production-grade orchestration for machine learning workflows and data processing created to accelerate local workflows to production.

Flyte

set the channel description: Channel for discussing Flyte &amp; Bazel integration

Just set this up to discuss Bazel &lt;&gt; Flyte integration &amp; any particulars that come up during development, as well as any work to get `rules_flyte` back up to date/upstream some of our own work

I'd be happy to contribute to `rules_flyte` . The implementation I put together this week is fairly encapsulated from our stack simply because its so new -- though that is a double edged sword

The only specifics we use are our internal py_binary/py_library targets. Historically we added linters as sidecar targets to that, though we’ve recently moved those to aspects. So we might be able to ditch the custom targets soon

Other than that we’re close to the same state 

Joining late to the party, but we (Woven by Toyota/ex-level5) have also open sourced some parts of our monorepo.. Happy to conttibute/upstream our work as well.

Would love to contribute as well. We based much of our integration at stripe on the bazel rules but will need some time to remove the stripe specific components

One thing that'd be awesome to share is how each of our implementations is handling serialization, registration &amp; execution. Do you all have separate targets for each, or are any of the steps handled together?

In our case, we handle both serialization &amp; registration in a single `.register` target, and handle execution w/ a `py_binary` target that takes in the launch plan name &amp; user-specified CLI args that are passed to `flyte_remote.execute` inside that binary. One interesting concern is that it requires the user to remember to register prior to attempting to launch an execution, and this is an area we'd really like to clean up and make more seamless. Curious if anyone else has different solutions for this?

Hi <@U03BT8P2XGA>, you got any lead here ? I am also looking forward to make the process one shot, without manually worrying about such dependency. Below is our way of doing it.

we have also combined serialization &amp; registration in a single `.register`  target, My design for launchplan is to pass a list of launchplans defined in workflow file as an attribute to flyte_library rule and it will prepare a separate target for launchplan, which will have dependency on register. So flow will look like
`.push` for image
`.register`  serialize and register (serialization dont produce any executable just the serialized proto consumed by registration)
`.launch` activate the launcplans (deppends on registration but that will only build the executable for registeration will not run it)
All the three steps will use the `commit-id` as version and dirty-repo will result in failure
We have also ensured to make `pyflyte*` and `flytectl` as bazel dependency rather than installing separately in the dev environment

Folks remember Flyte slack is free. Good thing would be to put things in a doc

Hi Guys, I am using bazel to build the flyte target but when I use bazel output binaries to serialize and register the workflows it lead to _*ModuleNotFoundError: No module named 'pyflyte'*_ at the time of task execution.  The error occurred because the value of  `resolver` in task container argument changed from usual  `flytekit.core.python_auto_container.default_task_resolver`  to  `pyflyte.pypi_flytekit.site-packages.flytekit.core.python_auto_container.default_task_resolver` . Did anyone face this issue ? is there a way to control this?

this just sounds like the virtualenv or the local python environment doesn’t have flytekit installed.

pyflyte comes stock with flytekit, it should always be present

I am trying to ship pyflyte as a Bazel dependency, so not installing it externally on the environment. 

<@U04MG1WLA4T> I wound up wrapping pyflyte in a `py_binary` target and adding that to my `py3_image`. That pattern should work for you

<@U04LSAKK6RH> also we figured out how to avoid the container_image step; since the entrypoint in our container delegates to the appropriate `pyflyte-*` command, just using the py3_image as-is worked (w/ the way that Flyte sets `args` for the pods)

This shaved ~45-60s off build time, which adds up over time