https://flyte.org logo
#ask-the-community
Title
# ask-the-community
c

Chris Grass

01/24/2024, 12:31 AM
Hello - i'm testing a flow using
ContainerTask
. can i inject credentials directly into
flyte-copilot-downloader
(or any other initContainer)? i tried adding to the task
PodTemplate
but that creates a conflict:
failed to execute handle for plugin [container]: [Invalid] failed to create resource, caused by: Pod "{podname}" is invalid: spec.initContainers[1].name: Duplicate value: "flyte-copilot-downloader"
s

Samhita Alla

01/24/2024, 5:08 AM
would you mind elaborating on your use case?
c

Chris Grass

01/24/2024, 2:54 PM
certainly - i have a a
ContainerTask
defined that is used by a workflow:
Copy code
transforminput = ContainerTask(
    name="flowtest",
    input_data_dir="/app/data/inputs",
    output_data_dir="/app/data/outputs",
    inputs=kwtypes(inputpath=str, outputpath=str),
    image="{imagelocation}/flowtest:latest",
    pod_template_name="default-pod-template",
    command=[{stuff}],
)
creating and using a
ContainerTask
in a remote wf appears to also create an `InitContainer`:
flyte-copilot-downloader
that init container attempts to pull data from our s3 metadata bucket. I see this in the task pod details:
Copy code
Init Containers:
  flyte-copilot-downloader:
    Container ID:  <containerd://561fe38310bb2f5af11e8e145816d9d4a983613924beea4223c6e237780a667>9
    Image:         <http://cr.flyte.org/flyteorg/flytecopilot-release:v1.10.0|cr.flyte.org/flyteorg/flytecopilot-release:v1.10.0>
    Image ID:      <http://cr.flyte.org/flyteorg/flytecopilot-release@sha256:8f1ae57f51c5c0f68c0a6ddf4986967d106c596497b86c8eb333cca27203ed52|cr.flyte.org/flyteorg/flytecopilot-release@sha256:8f1ae57f51c5c0f68c0a6ddf4986967d106c596497b86c8eb333cca27203ed52>
  Args:
      download
      --from-remote
      s3://{containerName}/n0/data/inputs.pb
however, it fails because the credentials injected into the credentials injected via env vars into the task container are not merged with the spawned initcontainer
so the initcontainer fails
I tried adding this block to the
PodTemplate
, but then a name collision pops:
Copy code
initContainers:
    - envFrom:
      - secretRef:
          name: default-aws-creds  
      name: flyte-copilot-downloader
      image: <http://cr.flyte.org/flyteorg/flytecopilot-release:v1.10.0|cr.flyte.org/flyteorg/flytecopilot-release:v1.10.0>
apparently some mechanism is trying to create the initContainer twice rather than merge the podtemplate config.
Copy code
failed to execute handle for plugin [container]: [Invalid] failed to create resource, caused by: Pod "fcaf1c64bb68945f7b67-n0-0" is invalid: spec.initContainers[1].name: Duplicate value: "flyte-copilot-downloader"
s

Samhita Alla

01/25/2024, 9:15 AM
are you providing credentials (secrets) via your container task? cc @David Espejo (he/him)
c

Chris Grass

01/25/2024, 1:42 PM
I am providing a
PodTemplate
to the
ContainerTask
, and that template contains a reference to the secret. but i don't know how to pass credentials specifically. can you point me towards documentation for that use case?
s

Samhita Alla

01/25/2024, 3:24 PM
you should be able to provide
secret_requests
directly to the
ContainerTask
https://docs.flyte.org/en/latest/flytesnacks/examples/productionizing/use_secrets.html#secrets
c

Chris Grass

01/29/2024, 3:31 PM
@Samhita Alla - i tried using
secret_requests
a few different ways but ran into a couple problems 1. the secrets get injected into the
InitContainer
but have a prepended
FLYTE_SECRETS_ENV_PREFIX
which is set to
_FSEC_
. a. The init for the stow container doesn't seem to be configurable to look for envvars with a specific prefix 2. the documentation suggests I can set
FLYTE_SECRETS_ENV_PREFIX
to customize the behavior - that is true for
Containers
but that configuration doesn't seem to get propagated to
InitContainers
any other ideas or suggested workarounds?
s

Samhita Alla

01/29/2024, 3:55 PM
@Eduardo Apolinario (eapolinario) would you mind chiming in?
c

Chris Grass

01/30/2024, 1:33 PM
Hello. We switched to using Azure Blob Storage with a Managed Identity because those credentials are injected at the pod level. Then Terence noticed a different exception in the InitContainer, which ended up being a permissions issue. The identity injected into the inicontainer wasn't able to read from azure blob storage
s

Samhita Alla

01/30/2024, 3:23 PM
is it working now?
c

Chris Grass

01/30/2024, 3:25 PM
yes, injecting a managed identity works. we have not yet found a way to inject envvars (sans
_FSEC_
prefix) into the
InitContainer
. that would be helpful for the future but we are not currently blocked
4 Views