#1837 Fix: Make sure decks created in elastic task workers are transferred to parent process Pull request opened by fg91 TL;DR Currently, Flyte decks don't work with the elastic tasks. This PR fixes this. Type ☑︎ Bug Fix ☐ Feature ☐ Plugin Are all requirements met? ☑︎ Code completed ☑︎ Smoke tested ☑︎ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue <- no pending items Complete description The reason is that the elastic task starts worker processes (as

torchrun

does). When a user creates a deck, this deck is appended to

flytekit.current_context().decks

in the worker processes. However, the upload of the deck happens based on the

ctx

in the parent process where these decks are never appended. The decks are, thus, never shown in the UI. This PR fixes this by returning the list of decks from the worker processes and, for rank

0

, attaches those decks to the parent processes context. Tracking Issue NA Follow-up issue NA flyteorg/flytekit GitHub Actions: build-plugins (3.8, flytekit-kf-tensorflow) GitHub Actions: build-plugins (3.8, flytekit-kf-pytorch) GitHub Actions: build-plugins (3.8, flytekit-kf-mpi) GitHub Actions: build-plugins (3.8, flytekit-k8s-pod) GitHub Actions: build-plugins (3.8, flytekit-huggingface) GitHub Actions: build-plugins (3.8, flytekit-hive) GitHub Actions: build-plugins (3.8, flytekit-greatexpectations) GitHub Actions: build-plugins (3.8, flytekit-envd) GitHub Actions: build-plugins (3.8, flytekit-duckdb) GitHub Actions: build-plugins (3.8, flytekit-dolt) GitHub Actions: build-plugins (3.8, flytekit-deck-standard) GitHub Actions: build-plugins (3.8, flytekit-dbt) GitHub Actions: build-plugins (3.8, flytekit-data-fsspec) GitHub Actions: build-plugins (3.8, flytekit-dask) GitHub Actions: build-plugins (3.8, flytekit-bigquery) GitHub Actions: build-plugins (3.8, flytekit-aws-sagemaker) GitHub Actions: build-plugins (3.8, flytekit-aws-batch) GitHub Actions: build-plugins (3.8, flytekit-aws-athena) GitHub Actions: docs GitHub Actions: build-integration (ubuntu-latest, 3.11) ✅ 10 other checks have passed 10/30 successful checks

#1837 Fix: Make sure decks created in elastic task workers are transferred to parent process Pull request merged by fg91 flyteorg/flytekit