https://flyte.org
Join Slack
Hello Team, I want to specify a GPU pool for my map task. I went through this <documentation> and i...
# announcements
o
Hello Team, I want to specify a GPU pool for my map task. I went through this documentation and it is not very clear. Could you please help with this ?
🖖 1
a
s
Hi @acceptable-policeman-57188 - We use this spark config to specify in which pool to launch a spark task in 
"<http://spark.kubernetes.node.selector.l5.lyft.com/pool|spark.kubernetes.node.selector.l5.lyft.com/pool>": eks-pdx-pool-gpu,
Now, @orange-umbrella-99251 is trying to convert the spark task to a map task which requires GPU, we are wondering how to do that. Should we add it to task’s Resource?
a
you should definitely add GPU to the task resource decorator but please go through the above documentation (specifically with regards to the configmap changes) in order for the map tasks to be scheduled on machines with gpus
s
Sure @acceptable-policeman-57188 = Thank you, we will look into that!
you should definitely add GPU to the task resource decorator
Can you show an example for this?
please go through the above documentation
Where should we add that config? in propeller/admin ? Bear with me for all noob questions! 🙂
a
no problem, we love questions 😄 (and finding ways to improve our documentation) you'll need to update the propeller configmap
for adding gpu to the map task instances, you can use the with_overrides syntax: https://docs.flyte.org/projects/flytekit/en/latest/generated/flytekit.map_task.html#flytekit-map-task
s
Copy code
k8s:
        co-pilot:
          name: "flyte-copilot-"
          image: ${ecr_repository}:flytecopilot-${flytecopilot_version}
          start-timeout: "30s"
        scheduler-name: flyte-scheduler
        inject-finalizer: true
        default-annotations:
          - <http://flyte.lyft.com/deployment|flyte.lyft.com/deployment>: flyte-l5
        resource-tolerations:
          <http://nvidia.com/gpu|nvidia.com/gpu>:
            key: <http://lyft.com/gpu|lyft.com/gpu>
            value: dedicated
            operator: Equal
            effect: NoSchedule
Looks like we already have this added to our propeller config Map 🙂
a
great
s
That helped @acceptable-policeman-57188
We launched Map tasks on gpu successfully!
🙌 1
How can we edit/update flyte documentation?
t
@sticky-art-97180, here’s a detailed guide to update docs. In a nutshell, • Fork the 
flytesnacks
repo • Update the relevant Python file • 
pip install -r docs-requirements.txt
in the 
cookbook
folder • Run 
make html
in the 
docs
folder • Open HTML pages present in the 
docs/_build
directory in the browser to verify if everything’s rendered correctly
164 Views
Open in Slack
PreviousNext
Powered by

Flyte

Flyte enables production-grade orchestration for machine learning workflows and data processing created to accelerate local workflows to production.

Powered by