#356 Don't add master replica log link when doing elastic pytorch training Pull request opened by fg91 TL;DR When doing torch elastic training, there is no so-called master replica in the resulting

PytorchJob

as opposed to when doing non-elastic pytorch distributed training. Flyteplugins, however, still generates a log link for the non-existing master replica in case of elastic training. This PR fixes this. Type ☑︎ Bug Fix ☐ Feature ☐ Plugin Are all requirements met? ☑︎ Code completed ☑︎ Smoke tested ☑︎ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue I built a propeller image and tested that the correct log links are shown both for elastic and the original non-elastic pytorch tasks. Complete description When doing "normal" non-elastic training, a flyte task looks like this:

@task(
    task_config=PyTorch(
        num_workers=3,
    )
)
def train():

The pytorch job that is created from this task definition looks like this:

apiVersion: "<http://kubeflow.org/v1|kubeflow.org/v1>"
kind: PyTorchJob
metadata:
  ...
spec:
  pytorchReplicaSpecs:
    Master:
      replicas: 1
      ...
    Worker:
      replicas: 3
      ...

Notice that there is a so-called "master" replica and multiple workers. In the Flyte console, a link to the master replica and to the 3 worker replicas logs is shown. When using the new elastic training task (torchrun) ...

@task(
    task_config=Elastic(
        nnodes=3,
        nproc_per_node=1,
    ),
)
def train():

... the resulting pytorch job looks like this:

apiVersion: "<http://kubeflow.org/v1|kubeflow.org/v1>"
kind: PyTorchJob
metadata:
  ...
spec:
  elasticPolicy:
    ...
  pytorchReplicaSpecs:
    Worker:
      replicas: 3
      ...

Notice that there is no-more "master" replica. Even though there is no "master" replica, currently the Flyte console still shows a log link for the master replica that doesn't exist. This PR fixes this. Tracking Issue NA Follow-up issue NA flyteorg/flyteplugins ✅ All checks have passed 7/7 successful checks