Hi all, I'm trying to get my implementation of a Bazel/Flyte integration off the ground and I'm running into an issue on the last bit which is stumping me. I created a bazel macro called

flyte_library

(taking cues from @Rahul Mehta’s talk!). On run, that rule 1. Creates a py3_image with the workflow file, as well as pulling in the

aws

,

flytectl

, and

pyflyte-*

cli's. I wanted to keep things hermetic within the bazel env so I didn't create a base image with `awscli`/`pyflyte` pre-installed. 2. We add the

FLYTE_INTERNAL_IMAGE

tag, and then push this image to ECR. I'm still not 100% sure what

FLYTE_INTERNAL_IMAGE

does, but followed the examples I could find. 3. We then have a genrule which runs

docker run

using the image we just created, and calls a custom register script which wraps

pyflyte register

to register the workflow, and uses

flytectl

to enable/optionally execute any launchplans registered alongside the workflow. Registration works correctly as far as I can tell. The objects are created and viewable in the Console, but all tasks fail with:

[1/1] currentAttempt done. Last Error: UNKNOWN::Outputs not generated by task execution

I can see the pod starting, pulling the correct image, and the

pyflyte-fast-execute

command exiting successfully via

kubectl

. No logs are created before the script exits so I'm having a bit of trouble identifying the issue. Weirder still, the exact same

pyflyte-fast-execute

command runs fine if I run it in a docker container locally.

what i really wanted was to use python directly in bazel - but maybe thats not possible. otherwise -

flytekit.remote

might be cool

Yeah... I didn't see

FlyteRemote

until a few hours ago -- will probably switch over to that

Ah interesting, our

register

genrule just executes `pyflyte`/`flytectl` in a shell script. we also had to jump through a few hoops to actually construct a

pyflyte*-execute

entrypoint that worked w/ our py3_image, we wound up making an entrypoint that could launch pyflyte but include all of the workflow deps inside its runfiles tree

I basically wrap each of the

pyflyte*

scripts as a

py_binary

using the pip_parse/entry_point rules and then add symlinks to them in the container

Makes sense, that's a different approach to get to the same outcome

it may make sense that I remove that requirement -- but the fact it runs fine locally in the same image means the issue is probably something else

@Sam Eckert did you need to use the

py3_image

as the

base

in a normal

container_image

in order to unset the entrypoint?

Yeah thats right.

Yeah...in general it's an annoying amount of overhead, especially from a cold cache