https://flyte.org logo
#ask-the-community
Title
# ask-the-community
t

Terry

01/24/2024, 7:39 AM
Hello experts. I had a general question about flyte. I'm coming from authoring pytorchjobs as yaml files where the pytorchjobs just wrap a script. I see examples where the main logic of a task is defined in Python and wrapped with a decorator, and I presume it is injected into a pytorchjobs. Does flyte support just simply running a python/bash script in a pytorchjob (or any arbitrary custom resource) under the task decorator?
s

Samhita Alla

01/24/2024, 11:44 AM
hi Terry, i don't think that's possible today but it should be possible to use subprocess within a python task. there's shell task but you cannot use pytorchjob with shell task as per this docstring.
But yes using shell tasks is an option. Remember using all gpus setting things up is more complicated
t

Terry

01/24/2024, 4:44 PM
Thanks for the response. Shell task looks nearly like what I'm looking for, with the exception of it not allowing me to specify an arbitrary job spec in the task_config. Any idea of when that option will be available? As for elastic, that's helpful; tho I think that still requires the python code to be wrapped in the task decorator, and won't work with shell
k

Ketan (kumare3)

01/24/2024, 4:49 PM
What do you mean by arbitrary job spec?
Do you want a task config to be passed into the shell task
t

Terry

01/24/2024, 6:17 PM
By arbitrary, I just meant like a pytorchjob or mpijob. Re: shell task Yea, since I'm trying to migrate users who wrap a shell script in a pytorchjob