Hi everyone, I was trying to use clusterAssignment values passed through the pyflyte flag, however it did not seem to work for me. On further examination i see the clusterAssignment value is being read/updated in flyteadmin, but never actually used while creating executions (a quick lookup for clusterassignment). I might be missing something. value being added to exec parameters no reference of the value being used in executor

seems like a bug to me. @average-finland-92144 can you confirm, please?

cc: @careful-analyst-27506

@hundreds-baker-75079 currently the mechanism to route executions to particular K8s clusters is the Execution Cluster Label. A change just merged to enable a more convenient way to express this on workflow execution: https://github.com/flyteorg/flyte/pull/4998 It confirms that the cluster pool parameter doesn't seem to be used. Does that help?

yes, thanks a lot for that!

I'm not sure I follow, I've used the labels with multiple clusters, using fast registration (pyflyte run). Pinging @high-accountant-32689 in case more context is needed re:flytekit

Oh I wasn't aware of that. Maybe I need a little handholding here. I was looking at the documentation here: https://docs.flyte.org/en/latest/api/flytekit/pyflyte.html and I only see

--cluster_pool

flag, but we validated above that wasn't being used in admin. Additionally the

ClusterAssignment

value also doesn't get passed to

ExecutionClusterLabel

either. I'm probably missing something obvious here.

You're not missing anything obvious, @hundreds-baker-75079. In fact, this is the combination of two similar features, so let's clarify some of the confusion here.

ExecutionClusterLabel

is a recent addition which was not implemented in

pyflyte run

yet. And

cluster-pool

was never fully implemented in the backend.

Got it, so as of now, does there exist a way to specify

ExecutionClusterLabel

using flytekit? If not and nobody has taken that up, I'll happily take it up on my end

please, go ahead. More than happy to review a PR.

thanks! i'll start working on it soon

Sorry, my bad mentioning pyflyte, I was hallucinating (maybe wishful thinking 😅 )

No worries! I have a fundamental question.

ExecutionClusterLabel (ecl)

is a

MatchingAttribute (ma)

. So during execution, it'd also make sense to specify other ma's such as

ExecutionQueue

or

PluginOverrides

as well? Would there be a reason to not support them? Imo adding all of them separately seems redundant. We already have a wrapper and we should ideally leverage that functionality. I don't have a lot of experience with a lot of these attributes like

ClusterResource

so I'm yet not sure whether they'd seem to make sense on an execution level. I'd love to hear your thoughts. Update: I was looking at the

ExecutionSpec

and just realized we already have

QoS

,

ClusterAssignment

and now

ExecutionClusterLabel

already a part of the struct. I do foresee some other fields to be added as well.

@hundreds-baker-75079 did you make any progress? I think this would be a really useful feature.

let me send out a pr soon. we were able get this working on our setup. i totally missed pushing this change upstream