https://flyte.org logo
#ask-the-community
Title
# ask-the-community
v

Vinayak

03/22/2024, 8:45 PM
Hi everyone, I was trying to use clusterAssignment values passed through the pyflyte flag, however it did not seem to work for me. On further examination i see the clusterAssignment value is being read/updated in flyteadmin, but never actually used while creating executions (a quick lookup for clusterassignment). I might be missing something. value being added to exec parameters no reference of the value being used in executor
s

Samhita Alla

03/25/2024, 7:15 AM
seems like a bug to me. @David Espejo (he/him) can you confirm, please?
v

Vinayak

03/25/2024, 4:51 PM
cc: @shardool
d

David Espejo (he/him)

03/26/2024, 12:16 PM
@Vinayak currently the mechanism to route executions to particular K8s clusters is the Execution Cluster Label. A change just merged to enable a more convenient way to express this on workflow execution: https://github.com/flyteorg/flyte/pull/4998 It confirms that the cluster pool parameter doesn't seem to be used. Does that help?
v

Vinayak

03/26/2024, 4:34 PM
yes, thanks a lot for that!
@David Espejo (he/him) would you know if someone is already working on the relevant changes for flytekit: https://github.com/flyteorg/flytekit/blame/master/flytekit/models/execution.py#L167? this part doesn't yet seem to support the execution cluster labels.
d

David Espejo (he/him)

04/05/2024, 3:08 PM
I'm not sure I follow, I've used the labels with multiple clusters, using fast registration (pyflyte run). Pinging @Eduardo Apolinario (eapolinario) in case more context is needed re:flytekit
v

Vinayak

04/05/2024, 5:56 PM
Oh I wasn't aware of that. Maybe I need a little handholding here. I was looking at the documentation here: https://docs.flyte.org/en/latest/api/flytekit/pyflyte.html and I only see
--cluster_pool
flag, but we validated above that wasn't being used in admin. Additionally the
ClusterAssignment
value also doesn't get passed to
ExecutionClusterLabel
either. I'm probably missing something obvious here.
e

Eduardo Apolinario (eapolinario)

04/05/2024, 10:02 PM
You're not missing anything obvious, @Vinayak. In fact, this is the combination of two similar features, so let's clarify some of the confusion here.
ExecutionClusterLabel
is a recent addition which was not implemented in
pyflyte run
yet. And
cluster-pool
was never fully implemented in the backend.
v

Vinayak

04/05/2024, 10:36 PM
Got it, so as of now, does there exist a way to specify
ExecutionClusterLabel
using flytekit? If not and nobody has taken that up, I'll happily take it up on my end
e

Eduardo Apolinario (eapolinario)

04/05/2024, 10:39 PM
please, go ahead. More than happy to review a PR.
v

Vinayak

04/05/2024, 10:41 PM
thanks! i'll start working on it soon
d

David Espejo (he/him)

04/05/2024, 11:30 PM
Sorry, my bad mentioning pyflyte, I was hallucinating (maybe wishful thinking 😅 )
v

Vinayak

04/09/2024, 11:40 PM
No worries! I have a fundamental question.
ExecutionClusterLabel (ecl)
is a
MatchingAttribute (ma)
. So during execution, it'd also make sense to specify other ma's such as
ExecutionQueue
or
PluginOverrides
as well? Would there be a reason to not support them? Imo adding all of them separately seems redundant. We already have a wrapper and we should ideally leverage that functionality. I don't have a lot of experience with a lot of these attributes like
ClusterResource
so I'm yet not sure whether they'd seem to make sense on an execution level. I'd love to hear your thoughts. Update: I was looking at the
ExecutionSpec
and just realized we already have
QoS
,
ClusterAssignment
and now
ExecutionClusterLabel
already a part of the struct. I do foresee some other fields to be added as well.
@Eduardo Apolinario (eapolinario) wondering if you have any thoughts on that?