#485 Move caching to node executor for fast cache hits Pull request opened by hamersaw TL;DR Currently cache lookups require 3 node-level phase changes, which equates to 3 streak rounds of flytepropeller execution. The overhead of updating the k8s FlyteWorkflow CRD after each of these rounds may make it seem like cache lookups are slow. This PR updates cache lookups to happen in the node executor, which allows us to transition nodes directly from NotYetStarted to Success and immediately process downstream nodes in the same FlytePropeller round making cache lookup overhead minimal. Type ☐ Bug Fix ☑︎ Feature ☐ Plugin Are all requirements met? ☐ Code completed ☐ Smoke tested ☐ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description In addition to the performance improvements this PR refactors the code to support future implementation of caching other Flyte node types (ie. subworkflow / launchplan and array nodes when they are implemented). Tracking Issue fixes flyteorg/flyte#2886 Follow-up issue NA flyteorg/flytepropeller GitHub Actions: Build & Push Flytepropeller Image GitHub Actions: Goreleaser GitHub Actions: Bump Version ✅ 11 other checks have passed 11/14 successful checks

1 new commit pushed to

<https://github.com/flyteorg/flytekit/tree/master|master>

by pingsutw

<https://github.com/flyteorg/flytekit/commit/6530f1205cb676e28b38429afc0d3514fc1300b2|6530f120>

- Automatically remove unused import (#1574) flyteorg/flytekit

#3572 Update Flyte components Pull request opened by flyte-bot Updated flyte deployment • Updated GCP Flyte kustomize generated manifest file • Updated EKS Flyte kustomize generated manifest file • Updated Sandbox Flyte kustomize generated manifest file • Updated TEST Flyte kustomize generated manifest file • Updated GCP Flyte helm generated manifest file • Updated EKS Flyte helm generated manifest file • Updated Sandbox Flyte helm generated manifest file • Updated TEST Flyte helm generated manifest file • Auto-generated by [flyte-bot] flyteorg/flyte ✅ All checks have passed 11/11 successful checks

1 new commit pushed to

<https://github.com/flyteorg/flytekit-python-template/tree/main|main>

by cosmicBboy

<https://github.com/flyteorg/flytekit-python-template/commit/c0fee0e9cf539f2de91321920ba6f412fc983e58|c0fee0e9>

- Continuous Deployment of OCI template default images (#23) flyteorg/flytekit-python-template

#24 ghcr secrets Pull request opened by zeryx This PR adds the correct ghcr secrets for flyteorg https://github.com/flyteorg/flytekit/blob/master/.github/workflows/pythonpublish.yml#L90-L96 flyteorg/flytekit-python-template ✅ All checks have passed 2/2 successful checks

1 new commit pushed to

<https://github.com/flyteorg/flytekit-python-template/tree/main|main>

by zeryx

<https://github.com/flyteorg/flytekit-python-template/commit/a9d779930b0a3308f429e77b2d15ea28cdbea5a0|a9d77993>

- added the correct org wide secrets for ghcr.io (#24) flyteorg/flytekit-python-template

#1577 Disallow partial lists in map tasks Pull request opened by eapolinario TL;DR As the title says Type ☐ Bug Fix ☑︎ Feature ☐ Plugin Are all requirements met? ☑︎ Code completed ☑︎ Smoke tested ☑︎ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description How did you fix the bug, make the feature etc. Link to any design docs etc Tracking Issue flyteorg/flyte#3403 Follow-up issue NA flyteorg/flytekit Codecov: 69.76% (-0.17%) compared to 9658b02 GitHub Actions: build-plugins (3.11, flytekit-sqlalchemy) GitHub Actions: build-plugins (3.11, flytekit-snowflake) GitHub Actions: build-plugins (3.11, flytekit-polars) GitHub Actions: build-plugins (3.11, flytekit-papermill) GitHub Actions: build-plugins (3.11, flytekit-pandera) GitHub Actions: build-plugins (3.11, flytekit-kf-tensorflow) GitHub Actions: build-plugins (3.11, flytekit-kf-pytorch) GitHub Actions: build-plugins (3.11, flytekit-kf-mpi) GitHub Actions: build-plugins (3.11, flytekit-k8s-pod) GitHub Actions: build-plugins (3.11, flytekit-hive) GitHub Actions: build-plugins (3.11, flytekit-dolt) GitHub Actions: build-plugins (3.11, flytekit-dbt) GitHub Actions: build-plugins (3.10, flytekit-vaex) ✅ 16 other checks have passed 16/30 successful checks

#222 Mock validation Pull request opened by bergman TL;DR Add some simple validation to mocks ☑︎ throw an exception if a mock was not used ☑︎ throw an exception if a duplicate mock was already defined ☐ throw an exception if a mock was not found and not explicitly configured to run actual task ☑︎ test behaviour of unmocked SdkRunnableTask tasks (they run by default if task type has no mocks defined) Improve error messages around mocks, Type ☐ Bug Fix ☐ Feature ☐ Plugin Are all requirements met? ☐ Code completed ☐ Smoke tested ☐ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description How did you fix the bug, make the feature etc. Link to any design docs etc Tracking Issue _Remove the '_fixes_' keyword if there will be multiple PRs to fix the linked issue_ fixes https://github.com/flyteorg/flyte/issues/ Follow-up issue NA OR https://github.com/flyteorg/flyte/issues/ flyteorg/flytekit-java GitHub Actions: build-snapshot ✅ 2 other checks have passed 2/3 successful checks

1 new commit pushed to

<https://github.com/flyteorg/flytekit/tree/master|master>

by eapolinario

<https://github.com/flyteorg/flytekit/commit/10bd38f69c78d778232f307360833924fec590e8|10bd38f6>

- Disallow partial lists in map tasks (#1577) flyteorg/flytekit

#980 Add an example of how to use BatchSize Pull request opened by Yicheng-Lu-llll Add an example of how to use BatchSize. See flyteorg/flytekit#1535. Please note that the example can only be run once flyteorg/flytekit#1535 has been merged. flyteorg/flytesnacks GitHub Actions: Mark github pre-release as Release GitHub Actions: Publish artifacts to github release GitHub Actions: Create Prerelease GitHub Actions: Bump Version GitHub Actions: Serialize & Register Flytesnacks workflow / Register example to sandbox GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/kubernetes/ray_example) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/kubernetes/pod) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/kubernetes/kftensorflow) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/kubernetes/kfpytorch) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/kubernetes/kfmpi) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/kubernetes/k8s_spark) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/kubernetes/k8s_dask) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/flytekit_plugins/sql) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/flytekit_plugins/pandera_examples) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/flytekit_plugins/onnx_examples) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/flytekit_plugins/mlflow_example) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/core) ✅ 13 other checks have passed 13/30 successful checks

#1578 Remove duplicate reporting logic Pull request opened by wild-endeavor TL;DR This used to be a problem but is now less so. In early iterations of flytekit, we would run into issues with multiple definitions of flyte entities. They tended to have different definitions and so would trigger a 'duplicate entity with different definition' error in flyte admin. This occurs much less frequently now and this check is causing an issue with map tasks. We will investigate post 1.5 release how to cache map task definitions so that multiple map_task() declarations on the same underlying task do not result in a new object being created. Type ☑︎ Bug Fix ☐ Feature ☐ Plugin Are all requirements met? ☑︎ Code completed ☐ Smoke tested ☑︎ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description flyteorg/flytekit Codecov: 69.90% (-0.02%) compared to 9658b02 ✅ 29 other checks have passed 29/30 successful checks

1 new commit pushed to

<https://github.com/flyteorg/flytekit-java/tree/master|master>

by github-actions[bot]

<https://github.com/flyteorg/flytekit-java/commit/cf964b63aae2758a1ee78bf8222d9b57d402d2f9|cf964b63>

- [maven-release-plugin] prepare release 0.4.16 flyteorg/flytekit-java

1 new commit pushed to

<https://github.com/flyteorg/flytekit-java/tree/master|master>

by github-actions[bot]

<https://github.com/flyteorg/flytekit-java/commit/09907cef70c10f23e7f3dd161c03d8b097826f3b|09907cef>

- [maven-release-plugin] prepare for next development iteration flyteorg/flytekit-java

Release - 0.4.16 New release published by pablocasares What's Changed • Explicitly fall back to real test execution when testing by @honnix in #218 Full Changelog: 0.4.15...0.4.16 flyteorg/flytekit-java

1 new commit pushed to

<https://github.com/flyteorg/flytesnacks/tree/master|master>

by samhita-alla

<https://github.com/flyteorg/flytesnacks/commit/990a0582d9e3494cc7703f85a2f9473db8b43e57|990a0582>

- Improve the multi-images example (#976) flyteorg/flytesnacks

Release - flytesnacks v0.3.189 New release published by flyte-bot Changelog • 990a058 Improve the multi-images example (#976) flyteorg/flytesnacks

#25 Add getting started template Pull request opened by ppiegaze Adding a template based on the wine dataset example from the Flyte Getting Started doc. flyteorg/flytekit-python-template ✅ All checks have passed 2/2 successful checks

1 new commit pushed to

<https://github.com/flyteorg/flyteplugins/tree/master|master>

by hamersaw

<https://github.com/flyteorg/flyteplugins/commit/e84d27a350596cdc14f780803c0b58890d12ecd3|e84d27a3>

- Validate multiple input values in map task (#334) flyteorg/flyteplugins

Release - v1.0.46 New release published by flyte-bot Changelog • e84d27a Validate multiple input values in map task (#334) flyteorg/flyteplugins

#26 [WIP] CI testing of templates w/ their built default OCI Images Pull request opened by zeryx flyteorg/flytekit-python-template GitHub Actions: test WIP: Title contains "WIP" ✅ 1 other check has passed 1/3 successful checks

1 new commit pushed to

<https://github.com/flyteorg/community/tree/main|main>

by davidmirror-ops

<https://github.com/flyteorg/community/commit/f88bb830f6c4a25ad44c868b736d5c552b91f7b9|f88bb830>

- Add new adopter flyteorg/community

1 new commit pushed to

<https://github.com/flyteorg/flytekit/tree/master|master>

by wild-endeavor

<https://github.com/flyteorg/flytekit/commit/b0cf91818c16867c6ba6d3a052c259771617ca1d|b0cf9181>

- Remove duplicate reporting logic (#1578) flyteorg/flytekit

1 new commit pushed to

<https://github.com/flyteorg/flytekit/tree/master|master>

by pingsutw

<https://github.com/flyteorg/flytekit/commit/0646bc3448f6f2cbf6bc0ceed9536bdfa6214428|0646bc34>

- [Core feature] Convert List[Any] to a single pickle file (#1535) flyteorg/flytekit

#1579 [WIP] Eager workflows to support async workflows Pull request opened by cosmicBboy TL;DR Implement an experimental API for "eager workflows": workflows that enable the use of Python async code. Type ☐ Bug Fix ☑︎ Feature ☐ Plugin Are all requirements met? ☐ Code completed ☐ Smoke tested ☐ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description How did you fix the bug, make the feature etc. Link to any design docs etc Tracking Issue flyteorg/flyte#3396 Follow-up issue NA flyteorg/flytekit GitHub Actions: build-plugins (3.8, flytekit-onnx-pytorch) GitHub Actions: build-plugins (3.8, flytekit-modin) GitHub Actions: build-plugins (3.8, flytekit-mlflow) GitHub Actions: build-plugins (3.8, flytekit-kf-tensorflow) GitHub Actions: build-plugins (3.8, flytekit-kf-pytorch) GitHub Actions: build-plugins (3.8, flytekit-kf-mpi) GitHub Actions: build-plugins (3.8, flytekit-k8s-pod) GitHub Actions: build-plugins (3.8, flytekit-hive) GitHub Actions: build-plugins (3.8, flytekit-greatexpectations) GitHub Actions: build-plugins (3.8, flytekit-duckdb) GitHub Actions: build-plugins (3.8, flytekit-dolt) GitHub Actions: build-plugins (3.8, flytekit-deck-standard) GitHub Actions: build-plugins (3.8, flytekit-dbt) GitHub Actions: build-plugins (3.8, flytekit-data-fsspec) GitHub Actions: build-plugins (3.8, flytekit-dask) GitHub Actions: build-plugins (3.8, flytekit-bigquery) GitHub Actions: build-plugins (3.8, flytekit-aws-sagemaker) GitHub Actions: build-plugins (3.8, flytekit-aws-batch) GitHub Actions: build-plugins (3.8, flytekit-aws-athena) GitHub Actions: docs GitHub Actions: build (windows-latest, 3.11) GitHub Actions: build (windows-latest, 3.9) GitHub Actions: build (windows-latest, 3.8) GitHub Actions: build (ubuntu-latest, 3.11) GitHub Actions: build (ubuntu-latest, 3.10) GitHub Actions: build (ubuntu-latest, 3.9) GitHub Actions: build (ubuntu-latest, 3.8) GitHub Actions: lint GitHub Actions: Docs Warnings WIP: Title contains "WIP" ✅ No checks have passed 0/30 successful checks

#981 clean up warnings in getting started docs Pull request opened by cosmicBboy This PR cleans up unnecessary warnings that are printed out in the getting started docs. flyteorg/flytesnacks ✅ All checks have passed 3/3 successful checks

#3573 [BUG] Reference tasks cannot be mapped over Issue created by samhita-alla Describe the bug When a reference task is mapped over in a workflow, it's resulting in

Failed with Unknown Exception <class 'AttributeError'> Reason: 'ReferenceTask' object has no attribute '_task_function'

error. Expected behavior This operation shouldn't result in any errors. Additional context to reproduce 1. Clone the flytesnacks repo 2. Register the decorating tasks example 3. Trigger the following code by specifying the registered version of task

t1

from flytekit import map_task, reference_task, workflow


@reference_task(
    project="flytesnacks",
    domain="development",
    name="core.flyte_basics.decorating_tasks.t1",
    version="<version>",
)
def t1(x: int) -> int:
    ...


@workflow
def wf(x: list[int] = [3, 2]):
    return map_task(t1)(x=x)

Screenshots No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

1 new commit pushed to

<https://github.com/flyteorg/flytekit/tree/master|master>

by samhita-alla

<https://github.com/flyteorg/flytekit/commit/0063b29763458b65da359a9569fc43a07c69cd6f|0063b297>

- Improve authoring structure documentation (#1572) flyteorg/flytekit

1 new commit pushed to

<https://github.com/flyteorg/flytesnacks/tree/master|master>

by cosmicBboy

<https://github.com/flyteorg/flytesnacks/commit/fa892028f998774b8ec828fae46963ea94d1e55c|fa892028>

- clean up warnings in getting started docs (#981) flyteorg/flytesnacks

#223 [Snyk] Upgrade com.google.api.grpc:proto-google-common-protos from 2.14.2 to 2.14.3 Pull request opened by snyk-bot Snyk has created this PR to upgrade com.google.api.grpc:proto-google-common-protos from 2.14.2 to 2.14.3. ℹ️ Keep your dependencies up-to-date. This makes it easier to fix existing vulnerabilities and to more quickly identify and fix newly disclosed vulnerabilities when they affect your project. * * * • The recommended version is 1 version ahead of your current version. • The recommended version was released 22 days ago, on 2023-03-14. * * * Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open upgrade PRs. For more information: https://camo.githubusercontent.com/34240eee664fe1d5c9566e1627948fe1ffa5ec6fae14486315c8ca68b6ee6bd0/68747470733a2f2f6170692e7365676d656e742e696f2f76312f706978656c2f747261636b3f646174613d65794a33636d6c305a55746c65534936496e4a79576d785a634564485932527954485a7362306c596430645563566734576b4652546e4e434f5545774969776959573576626e6c746233567a535751694f6949314d7a6c6d5a57526a4e433168596a4d7a4c54526c59546b74596a4a694f53307a4f574a6d59325a6b4d6d45324d6d49694c434a6c646d567564434936496c425349485a705a58646c5a434973496e42796233426c636e52705a584d694f6e736963484a4a5a434936496a557a4f575a6c5a474d304c5746694d7a4d744e4756684f5331694d6d49354c544d35596d5a6a5a6d51795954597959694a3966513d3d 🧐 View latest project report 🛠️ Adjust upgrade PR settings 🔕 Ignore this dependency or unsubscribe from future upgrade PRs flyteorg/flytekit-java GitHub Actions: build-snapshot DCO: DCO ✅ 1 other check has passed 1/3 successful checks

#545 Update resource manager to return TaskResourceAttributes Pull request opened by wild-endeavor TL;DR Add task resources to the list of matchable attributes that we fall back to system settings for. Changed the resource manager to take in the very top level

runtimeInterfaces.Configuration

interface instead of just

ApplicationConfiguration

. Type ☐ Bug Fix ☑︎ Feature ☐ Plugin Are all requirements met? ☑︎ Code completed ☑︎ Smoke tested ☐ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description Added

GetAsWorkflowExecutionAttribute()

and

GetAsAttribute()

to the

ApplicationConfiguration

and

TaskResourceConfiguration

interfaces respectively. Tracking Issue flyteorg/flyte#3467 flyteorg/flyteadmin GitHub Actions: Build & Push Flyteadmin Image GitHub Actions: Build & Push Flytescheduler Image GitHub Actions: Goreleaser GitHub Actions: Bump Version Codecov: 33.57% of diff hit (target 58.78%) Codecov: 58.56% (-0.22%) compared to cca9e17 ✅ 8 other checks have passed 8/14 successful checks