creamy-shampoo-53278
12/16/2024, 3:43 PMcreate and get methods works fine locally. At this early stage, we setup a single-host (ubuntu 20.04) machine for our local development and testing. Some takeaways are:
1. Run controller daemons (slurmctld, slurmdbd), one compute node daemon (slurmd), and also the REST API daemon (slurmrestd) on the same machine.
a. The slurm agent can interact with slurmrestd through the host base url <http://localhost:6820>.
b. Authentication is done by JWT.
2. Test the slurm agent locally based on this guide to mimic FlytePropeller’s behavior.
We'll keep pushing forward to make this feature a reality!creamy-shampoo-53278
12/16/2024, 3:44 PMimport os
from typing import Any, Dict
from flytekit import workflow
from flytekitplugins.slurm import SlurmTask
slurm_tiny_job = SlurmTask(
name="demo-slurm",
slurm_config={
"script": "#!/bin/bash\necho Hello Slurm Agent!",
"account": "flyte",
"partition": "debug",
"name": "hello-slurm-agent",
"environment": ["PATH=/bin/:/sbin/:/usr/bin/:/usr/sbin/"],
"current_working_directory": "/tmp"
},
)
@workflow
def hi_slurm(dummy: str) -> Dict[str, Any]:
"""Return slurm job information."""
res = slurm_tiny_job(dummy=dummy)
return res
if __name__ == "__main__":
from flytekit.clis.sdk_in_container import pyflyte
from click.testing import CliRunner
runner = CliRunner()
path = os.path.realpath(__file__)
# Local run
print(f">>> LOCAL EXEC <<<")
result = runner.invoke(pyflyte.main, ["run", path, "hi_slurm", "--dummy", "dummy_input"])
print(result.output)creamy-shampoo-53278
12/16/2024, 3:45 PMcreamy-shampoo-53278
12/16/2024, 3:46 PMfreezing-airport-6809
freezing-airport-6809
freezing-airport-6809
sbatch and aruncreamy-shampoo-53278
12/16/2024, 11:41 PMfreezing-airport-6809
asyncssh library, we actually discussed this. cc @damp-lion-88352. this is because most people do not use Rest apidamp-lion-88352
12/17/2024, 1:16 PMdamp-lion-88352
12/17/2024, 1:16 PMcreamy-shampoo-53278
12/17/2024, 2:35 PM