https://flyte.org
Join Slack
We are running Flyte on GKE cluster, we use podTemplate to configure env vars and secrets. After upd...
# flyte-support
e
We are running Flyte on GKE cluster, we use podTemplate to configure env vars and secrets. After updading the podTemplate, we were trying to run workflows using 
pyflyte --image
but we’re getting this error (occasionally) - 
Failed to apply default image tag "{{.image.demo.fqn}}:{{.image.demo.version}}": couldn't parse image reference "{{.image.demo.fqn}}:{{.image.demo.version}}": invalid reference format
Does anyone have any idea why it happens? Tnx
f
Looks like it didn’t render the image uri at all. What was the full command you ran? Please redact as necessary.
i
Run was done using the gui, same tasks were registered using this command:
Copy code
pyflyte register -p project_name -d development --image demo=someimeage:latest workflows/wf.py
Issue happens after updating podTemplate. same wfs run normally before.
f
Hmm. Can you paste the pod template as well? What do you mean when you say the error happens occasionally?
e
Copy code
apiVersion: v1
kind: PodTemplate
metadata:
  name: default
  namespace: dataengineer-development
template:
  metadata:
  spec:
    containers:
      - name: primary
        image: <http://docker.io/rwgrim/docker-noop|docker.io/rwgrim/docker-noop> # This value will always be overridden by Flyte
        env:
        - name: DB_USER
          valueFrom:
            secretKeyRef:
              name: db-non-prod
              key: username
        - name: DB_PASSWORD
          valueFrom:
            secretKeyRef:
              name: db-non-prod
              key: password
This is the pod template. It sometime works, sometimes not. We noticed happens after changing the pod tempalte
f
Why do you have a dummy value? Also, any reason why you don’t prefer secrets request?
e
We prefer to configure env and secret in k8s level, not in the workflows (the developers don’t have access to the k8s secret we create)
f
Hmm the pod template looks reasonable as well. Not sure why this would be flaky. It should either work or not. I’ll try to repro.
Flyte and flytekit version @elegant-toddler-67101?
e
The dummy value for pod template is because we set the image with the pyflyte command
Flyte chart version: 1.6.2 Flytekit version; 1.7.0
👍 1
f
I can’t seem to repro. What code is throwing the error? flytekit?
e
I guess it’s the flyteadmin/flytepropeller. We get the error while creating the task (= pod in GKE cluster)
👍 1
f
191 Views
Open in Slack
PreviousNext
Powered by

Flyte

Flyte enables production-grade orchestration for machine learning workflows and data processing created to accelerate local workflows to production.

Powered by