https://flyte.org logo
#ask-the-community
Title
# ask-the-community
a

Amadeusz Lisiecki

11/22/2023, 12:39 AM
Hey, did anyone succeed enabling AWS Secrets Manager in
flyte-binary
? I've added this to my values.yaml:
Copy code
configuration:
  inline:
    webhook:
      secretManagerType: "AWS"
and it seemed to override the proppeler configurtion correctly but no sidecar is being created for any of the tasks. And I get this error on task execution:
Copy code
Please make sure to add secret_requests=[Secret(group=arn:aws:secretsmanager:***:***:secret:, key=flyte/***)] in @task. Unable to find secret for key flyte/*** in group arn:aws:secretsmanager:***:***:secret: in Env Var:_FSEC_ARN:AWS:SECRETSMANAGER:***:***:SECRET:_FLYTE/*** and FilePath: /etc/secrets/arn:aws:secretsmanager:***:***:secret:/flyte/***
k

Kevin Su

11/22/2023, 12:40 AM
is your flyte-webhook running in the flyte-binary
a

Amadeusz Lisiecki

11/22/2023, 12:43 AM
I've used the default config so it's seems to be enabled, I can see an exposed webhook port and active webhook service. Can I test it somehow?
k

Kevin Su

11/22/2023, 12:51 AM
could you test it by using k8s secret?
a

Amadeusz Lisiecki

11/22/2023, 1:06 AM
I've modified my config to
Copy code
configuration:
  inline:
    webhook:
      secretManagerType: "K8s"
and run the example from the link. It worked for k8s secret - I was able to retrieve the secret inside the task.
k

Kevin Su

11/22/2023, 1:09 AM
investigating. cc @Prafulla Mahindrakar if you know how to fix it
p

Prafulla Mahindrakar

11/22/2023, 1:26 AM
Can you dump the pod webhook logs and check to see if it tried to add the sidecar. Also does the task pod have right role annotations to load the secrets from aws secrets manager ,
a

Amadeusz Lisiecki

11/22/2023, 1:29 AM
flyte-binary didn't output any logs for the webhook - is there some kind of config value to make it more verbose?
a

Amadeusz Lisiecki

11/22/2023, 7:46 PM
Maybe this is it
Copy code
{"json":{"src":"secrets.go:51"},"level":"info","msg":"Failed to inject a secret using injector [Global]. Error: secrets not found - Env [FLYTE_SECRET_FLYTE-FLYTE-BINARY-5BC97CC4CF-R7QPBARN:AWS:SECRETSMANAGER:E***:***:SECRET:_FLYTE/NEPTUNE-ZVYJTX], file [/etc/secrets/flyte-flyte-binary-5bc97cc4cf-r7qpbarn:aws:secretsmanager:***:***:secret:/flyte/neptune-zvyjTX]","ts":"2023-11-22T19:38:23Z"}
{"json":{"src":"secrets.go:45"},"level":"info","msg":"Skipping SecretManager [K8s] since it's not enabled.","ts":"2023-11-22T19:38:23Z"}
{"json":{"src":"aws_secret_manager.go:116"},"level":"error","msg":"unrecognized mount requirement [ENV_VAR] for secret [flyte/neptune-zvyjTX]","ts":"2023-11-22T19:38:23Z"}
{"json":{"src":"secrets.go:51"},"level":"info","msg":"Failed to inject a secret using injector [AWS]. Error: unrecognized mount requirement [ENV_VAR] for secret [flyte/neptune-zvyjTX]","ts":"2023-11-22T19:38:23Z"}
{"json":{"src":"secrets.go:45"},"level":"info","msg":"Skipping SecretManager [GCP] since it's not enabled.","ts":"2023-11-22T19:38:23Z"}
{"json":{"src":"secrets.go:45"},"level":"info","msg":"Skipping SecretManager [Vault] since it's not enabled.","ts":"2023-11-22T19:38:23Z"}
{"json":{"src":"pod.go:122"},"level":"info","msg":"Failed to mutate using [secrets]. Since it's not a required mutator, skipping. Error: unrecognized mount requirement [ENV_VAR] for secret [flyte/neptune-zvyjTX]","ts":"2023-11-22T19:38:23Z"}
Removing mount_requirement helped for some reason
Copy code
Secret(
        group=SECRET_GROUP,
        key=SECRET_NAME,
        # mount_requirement=Secret.MountType.ENV_VAR,
    ),
Thx for help!
Turns out AWS secrets plugin works only with
Secret.MountType.FILE
but not with
Secret.MountType.ENV_VAR
y

Yee

11/24/2023, 1:01 AM
i’m not sure there’s a good reason for this. mind putting in a ticket for this?
if that’s really the case it should probably fail or at least warn earlier. (like at registration time)
g

Garret Cook

12/01/2023, 3:51 AM
I had the same question, and I think I understand the answer. Only, where do I specify the AWS credentials?
Copy code
"AWS_ACCESS_KEY_ID": "some-value"
            "AWS_SECRET_ACCESS_KEY": "some-value",
            "AWS_DEFAULT_REGION": "some-value"
and may I specify different credentials on different project/environment combinations?
y

Yee

12/01/2023, 7:00 AM
can you open a new question plz? i don’t think this is the same.
11 Views