Flyte enables production-grade orchestration for machine learning workflows and data processing created to accelerate local workflows to production.

Flyte

We are running Flyte on GKE cluster, we use podTemplate to configure env vars and secrets.
After updading the podTemplate, we were trying to run workflows using `pyflyte --image` but we’re getting this error (occasionally) - `Failed to apply default image tag "{{.image.demo.fqn}}:{{.image.demo.version}}": couldn't parse image reference "{{.image.demo.fqn}}:{{.image.demo.version}}": invalid reference format`
Does anyone have any idea why it happens?
Tnx

Looks like it didn’t render the image uri at all. What was the full command you ran? Please redact as necessary.

Run was done using the gui, same tasks were registered using this command:
``` pyflyte register -p project_name -d development --image demo=someimeage:latest workflows/wf.py```
Issue happens after updating podTemplate. same wfs run normally before.

Hmm. Can you paste the pod template as well? What do you mean when you say the error happens occasionally?

```apiVersion: v1
kind: PodTemplate
metadata:
  name: default
  namespace: dataengineer-development
template:
  metadata:
  spec:
    containers:
      - name: primary
        image: <http://docker.io/rwgrim/docker-noop|docker.io/rwgrim/docker-noop> # This value will always be overridden by Flyte
        env:
        - name: DB_USER
          valueFrom:
            secretKeyRef:
              name: db-non-prod
              key: username
        - name: DB_PASSWORD
          valueFrom:
            secretKeyRef:
              name: db-non-prod
              key: password```

This is the pod template. It sometime works, sometimes not. We noticed happens after changing the pod tempalte

Why do you have a dummy value? Also, any reason why you don’t prefer secrets request?

We prefer to configure env and secret in k8s level, not in the workflows (the developers don’t have access to the k8s secret we create)

Hmm the pod template looks reasonable as well. Not sure why this would be flaky. It should either work or not. I’ll try to repro.

Flyte and flytekit version <@U059U7X5DB5>?

The dummy value for pod template is because we set the image with the pyflyte command

Flyte chart version: 1.6.2
Flytekit version; 1.7.0

I can’t seem to repro. What code is throwing the error? flytekit?

I guess it’s the flyteadmin/flytepropeller. We get the error while creating the task (= pod in GKE cluster)

i can't seem to reproduce this at all: <https://github.com/jeevb/flyte-examples/tree/jeev/debug-image-template-render-fail/debug-image-template-render-failure-with-pod-templates>

can you try the above minimal example on GKE?