I have a mystery that is either about mixing Flyte Task from multiple repos or is about how Flyte creates a version string for tasks that are registered without an explicit

--version

option. I have two python packages/repos that define python tasks. Each repo also creates a versioned docker image for use with its own Flyte Tasks and the docker image for outer_package contains and installation of inner_package. Each registers all of its Task and Workflows via

pyflyte register --version ${git_tag} ${package_folder}

. One package (call it outer_package) imports the other (call it inner_package) and defines a workflow from Tasks defined in both inner_package and outer_package. When running this workflow from outer_package, via

pyflyte run --remote

I see errors like

Running Execution on Remote.
Request rejected by the API, due to Invalid input.
RPC Failed, with Status: StatusCode.INVALID_ARGUMENT
        Details: <python import path to an inner_package_task> task with different structure already exists. (Please register a new version of the task):
/template/Target/Container/image:
        - <docker_uri>:v0.1.17
        + <docker_uri>:v0.1.18

I think this means that identical code for the inner_package function is already registered, but with a different

container_image

setting in

@task()

. This is true. I can get past this error by making an arbitrary change to the code of the inner_package function. Question: during registration, when Flyte creates a version string, does it use the hash value of the code being registered, not including the config passed to

@task()

? Question: Is there a preferred pattern for doing this kind of thing? I really hope the answer is not "mono-repo" 😉

What's the reason for having two repos for your tasks? Is it because different teams are working on different parts?

We have a core repo of general tasks for the team and a dozen other repos for individual projects.

🤔 Wouldn't using

reference task

make more sense? instead of importing tasks as code?

Aha! Excellent, thanks, @gentle-tomato-480! That sounds like the solution. I'll give it a go. Thank you for thinking about this problem and helping me out!