hello, I am trying to use spark-operator with flyte. However, I cannot pull the image.

docker pull gcr.io/spark-operator/spark-operator:v1beta2-1.2.0-3.0.0
Error response from daemon: manifest for gcr.io/spark-operator/spark-operator:v1beta2-1.2.0-3.0.0 not found: manifest unknown: Failed to fetch "v1beta2-1.2.0-3.0.0" from request "/v2/spark-operator/spark-operator/manifests/v1beta2-1.2.0-3.0.0".
(base) ➜  mlforge git:(flyte-deployment-2) ✗ docker pull gcr.io/spark-operator/spark-operator
Using default tag: latest
Error response from daemon: manifest for gcr.io/spark-operator/spark-operator:latest not found: manifest unknown: Failed to fetch "latest" from request "/v2/spark-operator/spark-operator/manifests/latest".

I got the image repo url from https://github.com/helm/charts/blob/master/incubator/sparkoperator/values.yaml, and image tag from https://googlecloudplatform.github.io/spark-on-k8s-operator/ Could anyone provide a working image tag? Thanks!

@Frank Shen are you following the steps outlined in https://docs.flyte.org/en/latest/deployment/plugins/k8s/index.html doc?

If you have run it before, can you update your repo?

helm repo update

I think the latest version is:

v1beta2-1.3.8-3.1.1

.. can you give this a shot? from here: https://googlecloudplatform.github.io/spark-on-k8s-operator/index.yaml

Hi @Haytham Abuelfutuh, The same error:

docker pull <http://gcr.io/spark-operator/spark-operator:v1beta2-1.3.8-3.1.1|gcr.io/spark-operator/spark-operator:v1beta2-1.3.8-3.1.1>
Error response from daemon: manifest for <http://gcr.io/spark-operator/spark-operator:v1beta2-1.3.8-3.1.1|gcr.io/spark-operator/spark-operator:v1beta2-1.3.8-3.1.1> not found: manifest unknown: Failed to fetch "v1beta2-1.3.8-3.1.1" from request "/v2/spark-operator/spark-operator/manifests/v1beta2-1.3.8-3.1.1".

@Frank Shen the image should be:

<http://ghcr.io/googlecloudplatform/spark-operator:v1beta2-1.3.8-3.1.1|ghcr.io/googlecloudplatform/spark-operator:v1beta2-1.3.8-3.1.1>

I wonder why is it your installation is trying to install from a non-existing repo... or maybe that was a more recent change from ghcr to gcr?

@Haytham Abuelfutuh, The spark-operator helm-chart is automatically using gcr from it’s templates which is wrong! I had to override to ghcr.io in my chart.