Hello Team! I have some problems, I made a workflo...
# ask-the-community
s
Hello Team! I have some problems, I made a workflow that has one shelltask. This shelltask have pod task config. But workflow gives errors that
Copy code
failed at Node[n0]. RuntimeExecutionError: failed during plugin execution, caused by: failed to execute
│  handle for plugin [sidecar]: [BadTaskSpecification] invalid TaskSpecification [<nil>], Err: [nil Struct Object passed]
How can I handle it?
Copy code
train = ShellTask(
⚠ name="train",
      debug=True,
      script="""
      set -ex
      NPROC_PER_NODE=$(nvidia-smi --query-gpu=name --format=csv,noheader | wc -l)
      python -m torch.distributed.launch --nproc_per_node $NPROC_PER_NODE pocs/wav2vec2/flyte.py --cfg {inp>
      """,
      inputs=kwtypes(cfg=str, save_directory=str),
      task_config=Pod(pod_spec=generate_gpu_pod_spec_for_task(), primary_container_name="primary"),
  )

⚠ @workflow
  def wf(cfg: str, save_directory: str) -> None:
      train(cfg=cfg, save_directory=save_directory)
my workflow
generate_gpu_pod_spec_for_task() method works on another tasks.
with
@task(task_config=Pod(…))
annotation, task’s config has
k8sPod
item, however, shelltask with pod task_config, task’s config has
container
item, not
k8sPod
f
maybe this is missing for ShellTask, just like it is missing for ContainerTask: https://github.com/flyteorg/flyte/issues/3055
d
@Seung-Woo Lee currently the
Pod
task_config
is only supported in python task definitions. There has been some great conversation on the restrictions on this and how we might address it. Basically, we decided to support a
pod_template
argument for all task definitions that will work exactly as you expected in your example above. We are tracking progress in this issue. The backend changes are close to being complete (hopefully this week) and then after completing the flytekit and testing end 2 end we will make this support available!
s
Great! Thanks for noticing me!
127 Views