https://flyte.org
Join Slack
Hello Team! I have some problems, I made a workflow that has one shelltask. This shelltask have pod ...
# flyte-support
m
Hello Team! I have some problems, I made a workflow that has one shelltask. This shelltask have pod task config. But workflow gives errors that
Copy code
failed at Node[n0]. RuntimeExecutionError: failed during plugin execution, caused by: failed to execute
│  handle for plugin [sidecar]: [BadTaskSpecification] invalid TaskSpecification [<nil>], Err: [nil Struct Object passed]
How can I handle it?
Copy code
train = ShellTask(
⚠ name="train",
      debug=True,
      script="""
      set -ex
      NPROC_PER_NODE=$(nvidia-smi --query-gpu=name --format=csv,noheader | wc -l)
      python -m torch.distributed.launch --nproc_per_node $NPROC_PER_NODE pocs/wav2vec2/flyte.py --cfg {inp>
      """,
      inputs=kwtypes(cfg=str, save_directory=str),
      task_config=Pod(pod_spec=generate_gpu_pod_spec_for_task(), primary_container_name="primary"),
  )

⚠ @workflow
  def wf(cfg: str, save_directory: str) -> None:
      train(cfg=cfg, save_directory=save_directory)
my workflow
generate_gpu_pod_spec_for_task() method works on another tasks.
with 
@task(task_config=Pod(…))
annotation, task’s config has 
k8sPod
item, however, shelltask with pod task_config, task’s config has 
container
item, not 
k8sPod
q
maybe this is missing for ShellTask, just like it is missing for ContainerTask: https://github.com/flyteorg/flyte/issues/3055
h
@millions-librarian-58501 currently the 
Pod
task_config
is only supported in python task definitions. There has been some great conversation on the restrictions on this and how we might address it. Basically, we decided to support a 
pod_template
argument for all task definitions that will work exactly as you expected in your example above. We are tracking progress in this issue. The backend changes are close to being complete (hopefully this week) and then after completing the flytekit and testing end 2 end we will make this support available!
thx 2
m
Great! Thanks for noticing me!
253 Views
Open in Slack
PreviousNext
Powered by

Flyte

Flyte enables production-grade orchestration for machine learning workflows and data processing created to accelerate local workflows to production.

Powered by