acoustic-van-93414
09/15/2023, 8:11 PMNodeSelector
field in the ExecutionSpec
in flyteidl/admin? Labels and annotations are already exposedacoustic-van-93414
09/15/2023, 8:11 PMglamorous-carpet-83516
09/15/2023, 8:25 PMfreezing-airport-6809
freezing-airport-6809
freezing-airport-6809
agreeable-flower-8989
09/25/2023, 6:09 PMagreeable-flower-8989
09/25/2023, 6:11 PMagreeable-flower-8989
09/25/2023, 6:12 PMfreezing-airport-6809
agreeable-flower-8989
09/26/2023, 2:17 PMteamA
will have nodeSelector <http://company.com/team|company.com/team>: teamA
and the node should have a corresponding label.
◦ This is actually Karpenter standard pattern "to provide workload segregation across nodes", where provisioner will create new nodes with label corresponding to pending pods value of the NodeSelector
• We apply nodeSelector at execution time, and expect the scheduling behavior to be per execution. Essentially, we are ok with different registered workflow/task having different nodeSelector. (Unlike pod template name)
• We have admin-proxy backend, where we have a reliable user identity and we would like to inject NodeSelector into the Execution request spec (similarly to how labels/annotation customizable on Flyte UI is applied to all tasks)
Using pod template names
• i know Dan works with the community for this useful feature, but i think it'll be hard for us..
◦ we want nodeSelector per user, so we will need to create that many number of podTemplates when new users in the company get onboarded..
◦ Pod template name is per-task, and we want to apply this scheduling for all tasks in a workflow.
◦ More importantly, users can choose any podTemplate meant for other users, so for our use case to achieve security objectives on user-specific workload isolation, during runtime, i can specify pod template for nodeSelector of any other userschilly-ram-6579
09/26/2023, 2:45 PMagreeable-flower-8989
09/26/2023, 3:34 PMNot all plugins allow flyte to customize the pods, fully, that end up getting created..1/ i see. Would "labels" added as part of execution request also get propagated to all plugins? But what you are saying is rather, the nodeSelector is less general than labels such that the selector might be dropped in some cases?
pod template (default) that says "nodeselector: pod.labels.user_identity"
...
K8s doesn't allow downstream APIs (referencing pod labels) in node selectors2/ Just to confirm, you are saying
nodeselector: pod.labels.user_identity
as in idea right? Implementation-wise, you are suggesting the admission webhook.
admission webhook (flyte's )3/ Are there ways we can add on to Flyte's admission webhook?
chilly-ram-6579
09/26/2023, 3:36 PMagreeable-flower-8989
09/26/2023, 3:37 PMchilly-ram-6579
09/26/2023, 3:38 PMagreeable-flower-8989
09/26/2023, 3:39 PMagreeable-flower-8989
09/26/2023, 3:39 PM