https://flyte.org logo
c

Chris Antenesse

08/26/2022, 12:57 PM
hi there, i’m trying to pull an image from a private registry. i modified the default service account to look like this:
kubectl edit serviceaccount/default -n flyte
Copy code
apiVersion: v1
imagePullSecrets:
- name: ghcr
kind: ServiceAccount
metadata:
  creationTimestamp: "2022-08-18T20:42:23Z"
  name: default
  namespace: flyte
  resourceVersion: "13612623"
  uid: 5bc39079-a6a5-4455-ae27-31eaed46c368
secrets:
- name: default-token-mfw8r
then created a secret resource to look like this:
kubectl edit secret/ghcr -n flyte
Copy code
apiVersion: v1
data:
  .dockerconfigjson: <REDACTED>
kind: Secret
metadata:
  annotations:
    <http://kubectl.kubernetes.io/last-applied-configuration|kubectl.kubernetes.io/last-applied-configuration>: |
      {"apiVersion":"v1","kind":"Secret","metadata":{"annotations":{},"name":"ghcr","namespace":"flyte"},"stringData":{".dockerconfigjson":"{\"auths\":{\"<http://ghcr.io|ghcr.io>\":{\"username\":\"<REDACTED>\",\"password\":\"<REDACTED>\",\"email\":\"<REDACTED>\",\"auth\":\"<REDACTED>\"}}}"},"type":"<http://kubernetes.io/dockerconfigjson|kubernetes.io/dockerconfigjson>"}
  creationTimestamp: "2022-08-25T21:02:59Z"
  name: ghcr
  namespace: flyte
  resourceVersion: "13628687"
  uid: 24dc477e-71cc-42eb-a0be-1bc61af20f5c
type: <http://kubernetes.io/dockerconfigjson|kubernetes.io/dockerconfigjson>
i used the
config.json
locally and was able to push and pull images to and from the private registry. but when executing a workflow, i get an image pull error in the UI (below) and when i describe the pod, i get this:
Copy code
Normal   Scheduled  18m                   default-scheduler  Successfully assigned flytesnacks-development/ajjf7v4swhdz7jxzvhwc-n0-0 to ip-10-15-147-196.ec2.internal
  Normal   Pulling    17m (x4 over 18m)     kubelet            Pulling image "<http://ghcr.io/predictap/symphony_hall:v0.0.14|ghcr.io/predictap/symphony_hall:v0.0.14>"
  Warning  Failed     17m (x4 over 18m)     kubelet            Failed to pull image "<http://ghcr.io/predictap/symphony_hall:v0.0.14|ghcr.io/predictap/symphony_hall:v0.0.14>": rpc error: code = Unknown desc = Error response from daemon: Head "<https://ghcr.io/v2/predictap/symphony_hall/manifests/v0.0.14>": unauthorized
  Warning  Failed     17m (x4 over 18m)     kubelet            Error: ErrImagePull
  Warning  Failed     17m (x6 over 18m)     kubelet            Error: ImagePullBackOff
  Normal   BackOff    3m26s (x65 over 18m)  kubelet            Back-off pulling image "<http://ghcr.io/predictap/symphony_hall:v0.0.14|ghcr.io/predictap/symphony_hall:v0.0.14>"
is there a good way to troubleshoot this? it seems like the docker config may not be present on the node that’s trying to pull the image (maybe one of the ec2 nodes associated with the EKS cluster?)
k

Kevin Su

08/26/2022, 1:41 PM
you should patch the
default
service account under the
PROJECTNAME-development
namespace instead of “Flyte”. https://github.com/flyteorg/flyte/discussions/1107#discussioncomment-842776
c

Chris Antenesse

08/26/2022, 1:42 PM
ah nice. ok, and probably should create the secret in that namespace too.
i’ll give it a try. thanks!
k

Kevin Su

08/26/2022, 1:42 PM
yes, correct. 👍
7 Views