Flyte enables production-grade orchestration for machine learning workflows and data processing created to accelerate local workflows to production.

Flyte

Hey I'm writing some e2e tests for my workflows that have failure nodes.
I'm structuring them in a similar way to these remote tests here <https://github.com/flyteorg/flytekit/blob/989eb678f1f119471c302ba71ea4cf71a263f977/tests/flytekit/integration/remote/test_remote.py#L121-L128>
During the sync that happens in the call to `remote.wait` , I'm getting the following error:
```ValueError: Missing node from mapping: fn0```

Running this with
```flyteidl==1.15.1
flytekit==1.15.0```

My tests' actual code is:
```@pytest.mark.timeout(120)
def test_workflow_e2e():
  storage_path = "xxx"
  OUTPUT_BUCKET= "xxx"
  # Trigger workflow using local code
  execution_id = run("workflow.py","workflow", "--storage_path", storage_path, "--output_bucket", OUTPUT_BUCKET)
  # Observe it
  remote = FlyteRemote(Config.auto(config_file=CONFIG), PROJECT, DOMAIN)
  execution = remote.fetch_execution(name=execution_id)
  # Assert that it ran under 120 seconds. Wait throws an exception in case of timeout
  try:
    execution = remote.wait(execution=execution, timeout=datetime.timedelta(minutes=2))
  except FlyteTimeout as timeout:
     assert False, timeout
  print("Execution Error:", execution.error)
  assert execution.closure.phase == WorkflowExecutionPhase.SUCCEEDED, f"Execution failed with phase: {execution.closure.phase}"```


This seems to only occur when the workflow fails and failure node is scheduled.
If the workflow succeeds, then `remote.wait` seems to work just fine/the Error is not thrown

Another related error. When calling `remote.wait` which calls `remote.sync` on a workflow that has branches (so `conditionals`, I assume), I get the following error:
```E           AttributeError: 'FlyteBranchNode' object has no attribute 'interface'

../../../../../anaconda3/envs/flyte/lib/python3.10/site-packages/flytekit/remote/remote.py:2648: AttributeError```

The full stacktrace:
```    execution = remote.wait(execution=execution, timeout=timeout)
../../../../../anaconda3/envs/flyte/lib/python3.10/site-packages/flytekit/remote/remote.py:2384: in wait
    execution = self.sync_execution(execution, sync_nodes=sync_nodes)
../../../../../anaconda3/envs/flyte/lib/python3.10/site-packages/flytekit/remote/remote.py:2479: in sync_execution
    node_execs[n.id.node_id] = self.sync_node_execution(n, node_mapping)  ```

I see this as well when using `on_failure`, I think the missing node error is a bug but may not be reported yet. I see it in the flytekit/pyflyte output

<https://github.com/flyteorg/flyte/issues/6490> Now reported