https://flyte.org logo
#ask-the-community
Title
# ask-the-community
f

Frank Shen

10/21/2022, 5:57 PM
Hello, Regarding I found this https://docs.flyte.org/projects/cookbook/en/latest/auto/integrations/kubernetes/k8s_spark/index.html#spark-docker-image, and its example Dockerfile.
Copy code
RUN python3 -m venv ${VENV}
ENV PATH="${VENV}/bin:$PATH"

RUN pip3 install wheel

# Install Python dependencies
COPY k8s_spark/requirements.txt /root
RUN pip install -r /root/requirements.txt

RUN flytekit_install_spark3.sh
k

Kevin Su

10/21/2022, 6:33 PM
you could follow this guide to setup the cluster and write a docker image for spark job.
and here is an spark job example in flytesnack
f

Frank Shen

10/21/2022, 8:50 PM
@Kevin Su, thank you. That’s exactly what I was looking for.