Hello Team, I want to specify a GPU pool for my m...
# announcements
v
Hello Team, I want to specify a GPU pool for my map task. I went through this documentation and it is not very clear. Could you please help with this ?
🖖 1
v
Hi @katrina - We use this spark config to specify in which pool to launch a spark task in
"<http://spark.kubernetes.node.selector.l5.lyft.com/pool|spark.kubernetes.node.selector.l5.lyft.com/pool>": eks-pdx-pool-gpu,
Now, @Vijay Saravana is trying to convert the spark task to a map task which requires GPU, we are wondering how to do that. Should we add it to task’s Resource?
k
you should definitely add GPU to the task resource decorator but please go through the above documentation (specifically with regards to the configmap changes) in order for the map tasks to be scheduled on machines with gpus
v
Sure @katrina = Thank you, we will look into that!
you should definitely add GPU to the task resource decorator
Can you show an example for this?
please go through the above documentation
Where should we add that config? in propeller/admin ? Bear with me for all noob questions! 🙂
k
no problem, we love questions 😄 (and finding ways to improve our documentation) you'll need to update the propeller configmap
for adding gpu to the map task instances, you can use the with_overrides syntax: https://docs.flyte.org/projects/flytekit/en/latest/generated/flytekit.map_task.html#flytekit-map-task
v
Copy code
k8s:
        co-pilot:
          name: "flyte-copilot-"
          image: ${ecr_repository}:flytecopilot-${flytecopilot_version}
          start-timeout: "30s"
        scheduler-name: flyte-scheduler
        inject-finalizer: true
        default-annotations:
          - <http://flyte.lyft.com/deployment|flyte.lyft.com/deployment>: flyte-l5
        resource-tolerations:
          <http://nvidia.com/gpu|nvidia.com/gpu>:
            key: <http://lyft.com/gpu|lyft.com/gpu>
            value: dedicated
            operator: Equal
            effect: NoSchedule
Looks like we already have this added to our propeller config Map 🙂
k
great
v
That helped @katrina
We launched Map tasks on gpu successfully!
🙌 1
How can we edit/update flyte documentation?
s
@varsha Parthasarathy, here’s a detailed guide to update docs. In a nutshell, • Fork the
flytesnacks
repo • Update the relevant Python file •
pip install -r docs-requirements.txt
in the
cookbook
folder • Run
make html
in the
docs
folder • Open HTML pages present in the
docs/_build
directory in the browser to verify if everything’s rendered correctly
115 Views