<#7274 [V2] Scope-aware secret encoding end-to-end...
# flyte-github
a
#7274 [V2] Scope-aware secret encoding end-to-end Pull request opened by pingsutw Why are the changes needed? Secrets written through
SecretService
and secrets looked up by the pod-admission webhook used different encodings, so project/domain-scoped secrets created via the CLI were never found at task admission — lookups fell through to the global scope. On top of that: • Pods had no
organization
label, so
validateRequiredFieldsExist
rejected every scoped lookup in OSS v2. • The K8s injector emitted both
_UNION_<group>_<key>
and the user's
EnvVar
name, producing duplicate env vars in the PodSpec. • The Embedded secret manager pre-checked the cache at every scope before any fetch, letting a stale broader-scope cache entry (e.g. global) mask a newly created narrower-scope secret (e.g. project+domain). • The webhook required in-cluster credentials at startup, which made local
make run
fail with
open /var/run/secrets/kubernetes.io/serviceaccount/token: no such file or directory
. What changes were proposed in this pull request?
SecretService
(secret/service/secret_service.go)
• Encode secret IDs as
EncodeSecretName("flyte", domain, project, name)
so the hashed Kubernetes resource name matches what the fetcher computes from pod labels. The encoded ID is stored as the data map key so
K8sSecretFetcher
reads it back by the same key. • Single namespace for all flyte-managed secrets (
config.GetConfig().Kubernetes.Namespace
). • Scope validation: project requires domain; domain-only and global are allowed.
organization
is assumed to be
flyte
since v2 has no org concept. • List filters by
app.flyte.org/project
/
app.flyte.org/domain
labels and reconstructs the response
SecretIdentifier
via
DecodeSecretName
. Executor (executor/pkg/plugin/task_exec_metadata.go) • Inject
organization=flyte
,
project=<spec.Project>
,
domain=<spec.Domain>
pod labels so the Embedded fetcher's
deriveSecretNameComponents
succeeds. Embedded injector (flyteplugins/.../secret/embedded_secret_manager.go) •
lookUpSecret
now walks scopes in priority order (project+domain → domain → org). For each scope it checks the cache first, then falls through to the fetcher loop for the same scope. Only when that scope is confirmed absent from both cache and all fetchers does it move to the next broader scope. This prevents a stale higher-scope cache entry from masking a fresh lower-scope secret. K8s injector (flyteplugins/.../secret/k8s_secrets.go) • In
Secret_ENV_VAR
mode, when the user sets
Secret.EnvVar
, use that name directly on the single env var instead of emitting the auto-generated
_UNION_<group>_<key>
alongside it. Local-dev kubeconfig (flyteplugins/.../secret/kube_config.go, secrets_injector.go, secret_fetcher.go) • New
resolveKubeConfig(ctx)
tries
InClusterConfig()
first, then falls back to the default kubeconfig loader. Used by both the Embedded injector and the K8s embedded fetcher so the webhook runs locally. Config •
manager/config.yaml
and
docker/devbox-bundled/manifests/complete.yaml
switched the webhook to
secretManagerTypes: [Embedded]
with
embeddedSecretManagerConfig.type: K8s
, pointed at the
flyte
namespace. How was this patch tested? Tested locally end-to-end: •
flyte create secret --project flytesnacks --domain development my-secret
writes a Secret in the
flyte
namespace with the scope labels. •
flyte get secret
returns the scoped Secret. • A task that references
my-secret
is admitted; the pod receives the value from the project+domain scope (cache-then-fetch per scope prevents the old global cache entry from leaking through). • Verified
_UNION_…
env var is no longer emitted when
Secret.EnvVar
is set. •
go build ./...
and
go vet ./...
pass across
flyteplugins
,
executor
, and
secret
. Labels • changed Check all the applicable boxes • I updated the documentation accordingly. • All new and existing tests passed. • All commits are signed-off. flyteorg/flyte