ancient-wolf-19325
09/06/2024, 6:21 PMflyte-copilot-downloader init container is embedding secrets directly into the manifest. This might be worse for us than other folks since the manifests are stored in our batch compute engine and displayed on UIs for everyone to see but this doesn't seem great?
initContainers:
- name: flyte-copilot-downloader
image: <redacted>/flytecopilot:v1.13.0-stack.14
command:
- /bin/flyte-copilot
- '--storage.limits.maxDownloadMBs=0'
- '--storage.container=<redacted>'
- '--storage.type=stow'
- '--storage.stow.config'
- secret_key=<redacted>
- '--storage.stow.config'
- access_key_id=<redacted>
- '--storage.stow.config'
- authtype=accesskey
- '--storage.stow.config'
- disable_ssl=0
- '--storage.stow.config'
- endpoint=<redacted>
- '--storage.stow.config'
- region=us-east-1
- '--storage.stow.kind=s3'
# Answer
To avoid embedding secrets directly into the flyte-copilot-downloader init container manifest, you can use Flyte's secret management feature. This allows you to securely pass sensitive information to your tasks without exposing it in the UI or logs.
1. Define Secrets: Use the Secret class to define the secrets required by your task.
2. Access Secrets: Retrieve the secrets within your task using flytekit.current_context().secrets.get.
Example:
import flytekit
from flytekit import Secret, task, workflow
SECRET_NAME = "user_secret"
SECRET_GROUP = "user-info"
@task(secret_requests=[Secret(group=SECRET_GROUP, key=SECRET_NAME)])
def secret_task() -> str:
secret_val = flytekit.current_context().secrets.get(SECRET_GROUP, SECRET_NAME)
# Use the secret value securely
return secret_val
@workflow
def wf() -> str:
return secret_task()
Would you like more detail on this approach?
Sources:
- Is there any way to mask secrets that are passed between tasks?
- Obfuscate sensitive data in TaskConfig
Slack Link
https://flyte-org.slack.com/archives/CP2HDHKE1/p1725646907.029379 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.