3 task types: • SlurmTask: most simple, runs an existing script on the Slurm cluster • SlurmShellTask: define a script in the task body, which gets copied and run on Slurm cluster, supports input/output substitution • SlurmFunctionTask: passed in to regular @task config, allows defining the python function as you normally would and runs it on the Slurm cluster. This is the one I have the most questions about. ◦ It seems to require a container, I assume this is just for the calling env i.e. no docker deps on Slurm cluster ◦ The example defines a script in the config, is this required? If so, what are the gains over SlurmShellTask? I've uncovered some bugs with the basic ShellTask as I'm still trying to create a workflow out of those. I'll follow up in another thread once I verify my understanding of the above. The code looks much more fully-formed and production-ready overall though! I'm thinking we want to feature freeze around the above and work on stability / usability / documentation though. Let me know your thoughts!

Slurm function task is a superset of slurm shell task and SlurmTask, we can use slurm function task only

@freezing-airport-6809 absolutely this, workspaces + notebooks, and the folding

for other task type

I'll do it with abao this week later

I'm working on running gpu task on a slurm gpu cluster

> we can use slurm function task only I mean we can use slurm function task to achieve what

slurm shell task

and

slurm task

can do

Also I see SlurmShellTask and SlurmTask aren't included in that PR that merged. Are we going to orphan both then since FunctionTask is supposed to cover their functionality?

the gpu setup is done on an aws ec2 cluster

Also I see SlurmShellTask and SlurmTask aren't included in that PR that merged. Are we going to orphan both then since FunctionTask is supposed to cover their functionality?

yes we are going to merge them after more testing

thank you Pryce