https://flyte.org logo
Powered by
#ask-the-community
Title
# ask-the-community
a

Ariel Kaspit

06/26/2023, 1:03 PM
We are running Flyte on GKE cluster, we use podTemplate to configure env vars and secrets. After updading the podTemplate, we were trying to run workflows using 
pyflyte --image
but we’re getting this error (occasionally) - 
Failed to apply default image tag "{{.image.demo.fqn}}:{{.image.demo.version}}": couldn't parse image reference "{{.image.demo.fqn}}:{{.image.demo.version}}": invalid reference format
Does anyone have any idea why it happens? Tnx
j

jeev

06/26/2023, 1:22 PM
Looks like it didn’t render the image uri at all. What was the full command you ran? Please redact as necessary.
n

Nizar Hattab

06/26/2023, 1:36 PM
Run was done using the gui, same tasks were registered using this command:
Copy code
pyflyte register -p project_name -d development --image demo=someimeage:latest workflows/wf.py
Issue happens after updating podTemplate. same wfs run normally before.
j

jeev

06/26/2023, 1:43 PM
Hmm. Can you paste the pod template as well? What do you mean when you say the error happens occasionally?
a

Ariel Kaspit

06/26/2023, 1:51 PM
Copy code
apiVersion: v1
kind: PodTemplate
metadata:
  name: default
  namespace: dataengineer-development
template:
  metadata:
  spec:
    containers:
      - name: primary
        image: <http://docker.io/rwgrim/docker-noop|docker.io/rwgrim/docker-noop> # This value will always be overridden by Flyte
        env:
        - name: DB_USER
          valueFrom:
            secretKeyRef:
              name: db-non-prod
              key: username
        - name: DB_PASSWORD
          valueFrom:
            secretKeyRef:
              name: db-non-prod
              key: password
This is the pod template. It sometime works, sometimes not. We noticed happens after changing the pod tempalte
k

Ketan (kumare3)

06/26/2023, 1:54 PM
Why do you have a dummy value? Also, any reason why you don’t prefer secrets request?
a

Ariel Kaspit

06/26/2023, 1:55 PM
We prefer to configure env and secret in k8s level, not in the workflows (the developers don’t have access to the k8s secret we create)
j

jeev

06/26/2023, 1:57 PM
Hmm the pod template looks reasonable as well. Not sure why this would be flaky. It should either work or not. I’ll try to repro.
Flyte and flytekit version @Ariel Kaspit?
a

Ariel Kaspit

06/26/2023, 1:59 PM
The dummy value for pod template is because we set the image with the pyflyte command
Flyte chart version: 1.6.2 Flytekit version; 1.7.0
j

jeev

06/27/2023, 3:50 AM
I can’t seem to repro. What code is throwing the error? flytekit?
a

Ariel Kaspit

06/27/2023, 10:32 AM
I guess it’s the flyteadmin/flytepropeller. We get the error while creating the task (= pod in GKE cluster)
j

jeev

06/27/2023, 7:54 PM
Powered by