I couldn’t find this, but is there a way to specif...
# flytekit
e
I couldn’t find this, but is there a way to specify the machine type in the task? I see options for cpu, gpu, and mem, but not for which types. I think we could train using AWS Sagemaker where the machine type in configurable, but seems unnecessary if I can just do it in Flyte. Let me know what I am missing! Thank you
n
so Flyte folllows K8s philosophy of abstracting away specifics about compute like machine type of the underlying cloud interface, hence options for cpu, gpu, mem. The AWS sagemaker integration allows for this (see the
SagemakerTrainingJobConfig
here) @Kevin Su @Samhita Alla not sure if you input here on how to configure machine types at the Flyte cluster configuration level.
e
Ah okay. I think Kubeflow lets you do this and it, at least to me, it is a pretty important feature for running jobs natively. Otherwise, I might need databricks and sagemaker to get the optimized compute machines. It is possible that the speed up is negligible for our jobs, but for some jobs it could make a big difference
n
what machine type are you trying to specify?
e
I think it would be the m5 and c5 types. I honestly am not sure what our cluster runs on, but I will find out.
We might want some workers with a lot of memory for inference spark jobs as well.
I don’t know what the limits are when the machine type is abstracted away
k
Requests and limits are the mechanisms Kubernetes uses to control resources IIRC, we don’t support specify the machine type in the task for now. cc @Ketan (kumare3) @Dan Rammer (hamersaw) is there any other way to run the task on a specific machine type?
e
Ah okay yeah this looks like vertex ai doesn’t do this either. They will choose the appropriate type depending on the job. I don’t really know what that means, but does Flyte do something similar if you request like 96 cpus?
k
@Evan Sadler how can kubeflow do it - I do not know how to do it in K8s - I think you might want to choose the right node-pools and that can be done using
labels and annotations
and or node selectors
e
@Ketan (kumare3) I am speaking from an area of very little experience so I apologize. I think we setup multiple node pools and picked between them. I will double check with my old colleagues and see.
Also appreciate all the knowledge
k
absolutely, ohh no harm. I just wanted to explain my ignorance
d
Another thought is taints / tolerations. Not sure if there is anyway to interface with K8s machine types there. May be worth looking into.
138 Views