hi team, is there any example/dockerfile for buidling flyte spark plugin task image , with imagespec it works but is there any way to manually build it ? tried using same dockerfile as of flyte python task with installing flytekitplugins-spark but it fails creating driver pod with below

│   Warning  Failed       1s    kubelet            Error: failed to create containerd task: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error duri │
│ ng container init: exec: "driver": executable file not found in $PATH: unknown

which image are you using? We publish official flytekit spark plugin images. Are you using them?

hi @high-accountant-32689 thanks, i found the images for flytekit spark plugin but it has python 3.9, does it only supports 3.9 ?

@square-carpet-13590, you'll have to build a similar image as https://github.com/flyteorg/flytekit/blob/master/plugins/flytekit-spark/Dockerfile. I checked and the latest apache/spark-py image was published 2 years ago and runs python 3.10.6.

Thanks a lot @high-accountant-32689