#2995 [RFC] Performance Benchmarking Pull request opened by hamersaw Signed-off-by: Daniel Rammer daniel@union.ai flyteorg/flyte ✅ All checks have passed 6/6 successful checks

#542 Bumping flyteplugins to 1.0.42 Pull request opened by hamersaw TL;DR Picks up external cluster configuration for Ray plugin. Type ☐ Bug Fix ☑︎ Feature ☐ Plugin Are all requirements met? ☑︎ Code completed ☑︎ Smoke tested ☐ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description ^^^ Tracking Issue NA Follow-up issue NA flyteorg/flytepropeller GitHub Actions: Goreleaser GitHub Actions: Build & Push Flytepropeller Image GitHub Actions: Bump Version ✅ 11 other checks have passed 11/14 successful checks

#3507 Update Flyte components Pull request opened by flyte-bot Updated flyte deployment • Updated GCP Flyte kustomize generated manifest file • Updated EKS Flyte kustomize generated manifest file • Updated Sandbox Flyte kustomize generated manifest file • Updated TEST Flyte kustomize generated manifest file • Updated GCP Flyte helm generated manifest file • Updated EKS Flyte helm generated manifest file • Updated Sandbox Flyte helm generated manifest file • Updated TEST Flyte helm generated manifest file • Auto-generated by [flyte-bot] flyteorg/flyte ✅ All checks have passed 11/11 successful checks

#3508 [Core feature] Improve error message when using promises like regular python values Issue created by cosmicBboy Motivation: Why do you think this is important? New flytekit users will often try to manipulate promises expecting them to be regular python values. This throws off the user experience, so error messages need to be absolutely clear when doing so. Goal: What should the final outcome look like, ideally? The error message should provide a clear explanation as to why promises can only be operated on in specific circumstances. Describe alternatives you've considered NA Propose: Link/Inline OR Additional context No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#3509 [Core feature] Improved error message with maptasks: using it within a task should error out Issue created by cosmicBboy Motivation: Why do you think this is important? Flytekit should raise a runtime error when map tasks are written in any of the following contexts: • When a user tries to write a

map_task

within a

task

function body

@task
def t1():
    map_task(...)(...)

• When a user tries to define a

map_task

at the module-level scope

@map_task
@task
def t1(): ...

or

@task
def t1(): ...


my_map_task = map_task()(t1)

Goal: What should the final outcome look like, ideally? The error should make it clear that `map_task`s can only be defined in workflow function bodies. Describe alternatives you've considered NA Propose: Link/Inline OR Additional context No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#3510 Re-detect Flyte Admin version Pull request opened by wild-endeavor This was previously hard-coded to deploy a specific version of Admin. Now reverting to older behavior. flyteorg/flyte ✅ All checks have passed 6/6 successful checks

1 new commit pushed to

<https://github.com/flyteorg/flyte/tree/master|master>

by wild-endeavor

<https://github.com/flyteorg/flyte/commit/cfbc88900f0c32097ba41de64b574c8dbd4cc2ca|cfbc8890>

- Re-detect Flyte Admin version (#3510) flyteorg/flyte

#3511 Update Flyte components Pull request opened by flyte-bot Updated flyte deployment • Updated GCP Flyte kustomize generated manifest file • Updated EKS Flyte kustomize generated manifest file • Updated Sandbox Flyte kustomize generated manifest file • Updated TEST Flyte kustomize generated manifest file • Updated GCP Flyte helm generated manifest file • Updated EKS Flyte helm generated manifest file • Updated Sandbox Flyte helm generated manifest file • Updated TEST Flyte helm generated manifest file • Auto-generated by [flyte-bot] flyteorg/flyte ✅ All checks have passed 11/11 successful checks

1 new commit pushed to

<https://github.com/flyteorg/flytekit/tree/master|master>

by eapolinario

<https://github.com/flyteorg/flytekit/commit/a19043116f507d149c7ff7858ab97100717e51ee|a1904311>

- Make

FlyteFile

compatible with

Annotated[..., HashMethod]

(#1544) flyteorg/flytekit

1 new commit pushed to

<https://github.com/flyteorg/flyteconsole/tree/master|master>

by jsonporter

<https://github.com/flyteorg/flyteconsole/commit/c968593b2385935ffd2f31b73a67d18decb55bf7|c968593b>

- fix: show correct i/o in details panel (#727) flyteorg/flyteconsole

1 new commit pushed to

<https://github.com/flyteorg/flyteidl/tree/master|master>

by hamersaw

<https://github.com/flyteorg/flyteidl/commit/bdb76306558b64645f095bc34b32ac487222fc3f|bdb76306>

- Create service for runtime metrics (#367) flyteorg/flyteidl

Release - v1.3.14 New release published by flyte-bot Changelog • bdb7630 Create service for runtime metrics (#367) flyteorg/flyteidl

1 new commit pushed to

<https://github.com/flyteorg/flyte/tree/master|master>

by eapolinario

<https://github.com/flyteorg/flyte/commit/733e39549228854d85c5610f4e9f94f31b2332f1|733e3954>

- Update Flyte components (#3511) flyteorg/flyte

Release - v1.5.3 New release published by flyte-bot 1.5.3 (2023-03-21) Bug Fixes • show correct i/o in details panel (#727) (c968593) flyteorg/flyteconsole

1 new commit pushed to

<https://github.com/flyteorg/flytepropeller/tree/master|master>

by hamersaw

<https://github.com/flyteorg/flytepropeller/commit/b4cde1f63873c88800e869f25070176ac068292a|b4cde1f6>

- bumping flyteplugins to 1.0.42 to pick up external cluster configuration for ray (#542) flyteorg/flytepropeller

#3512 [BUG] Workflow Conditional Operator conflict with task generated promises Issue created by zeryx Describe the bug When using the Conditional operator here a bug is detected where when a task generated output is passed into the task call inside of a

then()

closure, an error is reported server side. When this is run locally, this error is not reported and the output is successfully generated. The error message when run on-cloud is the following:

failed at Node[n3-n0]. BindingResolutionError: Error binding Var [wf].[dataset], caused by: failed at Node[n0]. CausedByError: Failed to GetPrevious data from outputDir [<s3://union-oc-production-demo/metadata/propeller/zeryx-demo-development-a9vk7kbf8ptkdcdwqrq6/n0/data/0/outputs.pb>], caused by: path:<s3://union-oc-production-demo/metadata/propeller/zeryx-demo-development-a9vk7kbf8ptkdcdwqrq6/n0/data/0/outputs.pb>: not found

The node graph diagram displays the incorrect sequence of operations: Expected behavior Conditional Then closures within a Workflow should be able to accept tasks and task generated inputs as parameters for tasks. Instead, only workflow level inputs can be provided to a task as a parameter within a Conditional Then closure, without seeing an error. Additional context to reproduce 1. create a new pyflyte project 2. Inspect the gist here https://gist.github.com/zeryx/1a786f2041d840f08175ea19b741b68f 3. copy broken_example into the workflows/example.py file 4. update the requirements.txt from the gist 5. pip install the dependencies to run locally with pyflyte 6. run the following:

pyflyte run workflows/example.py train_mnist_model --n_epoch 1

7. then run the following, pointing to your remote cluster of choice:

pyflyte --config ~/.uctl/config.yaml run -p your_project -d development --image zeryx1211/mnist _gpu:latest workflows/example.y train_mnist_model --n_epoch1

8. View the failed workflow and execution 9. Rerun the same with working_example from the gist, repeat steps 6-8 Screenshots No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#3495 Split flyte-binary services into http and grpc, and some cleanups Pull request opened by jeevb flyteorg/flyte ✅ All checks have passed 14/14 successful checks

1 new commit pushed to

<https://github.com/flyteorg/flyte/tree/master|master>

by jeevb

<https://github.com/flyteorg/flyte/commit/a68d3b584e015a109f464f7478055f0df4208628|a68d3b58>

- Split flyte-binary services into http and grpc, and some cleanups (#3495) flyteorg/flyte

Release - v1.1.71 New release published by flyte-bot Changelog • b4cde1f bumping flyteplugins to 1.0.42 to pick up external cluster configuration for ray (#542) flyteorg/flytepropeller

#542 Add noop migrations Pull request opened by wild-endeavor TL;DR The existing migrations are not correctly specified. This PR basically rewrites the existing migrations in the form that gorm recommends by adding a series of migrations that should 1. Be a noop for existing Admin users (i.e. users who are up to date with the current set of migrations) 2. Perfectly replicate existing Admin database in the case of a new database. We intend to keep the current migrations around for one release. Type ☐ Bug Fix ☐ Feature ☐ Plugin ☑︎ Refactor Are all requirements met? ☑︎ Code completed ☑︎ Smoke tested ☐ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description We never followed the recommended migration pattern in Gorm. To have an easier time with the work necessary to add mysql support, it'd be better to have existing model migrations cleaned up. Testing Procedure Sandbox Testing using the

flytectl demo

sandbox was done • Spin up flytectl demo cluster, confirm the existing migrations were all run, then run new migrations: confirm no alter commands were run. • Create two new databases, run the old migrations on one, and the new migrations on the second: do a pg_dump and confirm they are identical. The combination of these tests implies that the new migrations map one to one and onto the old migrations. The tests were deleted but can be found here: https://gist.github.com/wild-endeavor/cf2336ebf059a087334a8d8e7efe35b1 The pg_dumps are legacy.txt and noop.txt Tracking Issue This is the first step in fulfilling flyteorg/flyte#3277 flyteorg/flyteadmin GitHub Actions: Check Go Generate / Go Generate GitHub Actions: Docker Build Images / Build Docker Image ✅ 2 other checks have passed 2/4 successful checks

#3514 [Core feature] Possibility to check for `None` values Issue created by MartinTschechne Motivation: Why do you think this is important? Additionally to

is_true()

and

is_false()

it would be nice to have a functionality to check whether

None

is returned by a task. This could be useful in conditionals for example. Goal: What should the final outcome look like, ideally? The function

is_none()

should simply return

True

or

False

and logically work like

if val is None:
    return True
else:
    return False

Describe alternatives you've considered Either returning the string

'None'

or

False

and explicitly check for that. Propose: Link/Inline OR Additional context No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#729 gate node in dynamic task Pull request opened by james-union flyteorg/flyte#3354 How to reproduce

flytesnacks/development/human_in_the_loop.wf_dyn

Launch the above workflow and it should show

Paused

for gate nodes. How to fix Dynamic nodes are included in

subWorkflows

under

compiledWorkflowClosure

instead of

primary

. We only considered

priamry

to check the compiled nodes and that's why the ui couldn't understand the gate nodes. Another issue was that compiled nodes are not only based on execution node id. We should also compare with the metadata.specNodeId Type ☑︎ Bug Fix ☐ Feature ☐ Plugin Are all requirements met? ☐ Code completed ☐ Smoke tested ☐ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description How did you fix the bug, make the feature etc. Link to any design docs etc Tracking Issue _Remove the '_fixes_' keyword if there will be multiple PRs to fix the linked issue_ fixes flyteorg/flyte#3354 flyteorg/flyteconsole GitHub Actions: Build & Push FlyteConsole Image GitHub Actions: Get Release Tag GitHub Actions: Generate Release ✅ 8 other checks have passed 8/11 successful checks

1 new commit pushed to

<https://github.com/flyteorg/flyteadmin/tree/master|master>

by hamersaw

<https://github.com/flyteorg/flyteadmin/commit/83e8694e04f4e75f4b8bf71f32de620fba375c39|83e8694e>

- Tracking reasons time-series (#540) flyteorg/flyteadmin

Release - v1.1.85 New release published by flyte-bot Changelog • 83e8694 Tracking reasons time-series (#540) flyteorg/flyteadmin

#1557 support python 3.11 Pull request opened by cosmicBboy TL;DR Fixes flyteorg/flyte#3312. This PR makes the necessary changes to support python 3.11 Type ☐ Bug Fix ☑︎ Feature ☐ Plugin Are all requirements met? ☑︎ Code completed ☑︎ Smoke tested ☑︎ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description See TLDR Tracking Issue flyteorg/flyte#3312 Follow-up issue NA flyteorg/flytekit GitHub Actions: lint GitHub Actions: build-plugins (3.11, flytekit-whylogs) GitHub Actions: build-plugins (3.11, flytekit-vaex) GitHub Actions: build-plugins (3.11, flytekit-sqlalchemy) GitHub Actions: build-plugins (3.11, flytekit-spark) GitHub Actions: build-plugins (3.11, flytekit-snowflake) GitHub Actions: build-plugins (3.11, flytekit-ray) GitHub Actions: build-plugins (3.11, flytekit-polars) GitHub Actions: build-plugins (3.11, flytekit-papermill) GitHub Actions: build-plugins (3.11, flytekit-pandera) GitHub Actions: build-plugins (3.11, flytekit-modin) GitHub Actions: build-plugins (3.11, flytekit-kf-tensorflow) GitHub Actions: build-plugins (3.11, flytekit-kf-pytorch) GitHub Actions: build-plugins (3.11, flytekit-kf-mpi) GitHub Actions: build-plugins (3.11, flytekit-k8s-pod) GitHub Actions: build-plugins (3.11, flytekit-hive) GitHub Actions: build-plugins (3.11, flytekit-greatexpectations) GitHub Actions: build-plugins (3.11, flytekit-duckdb) GitHub Actions: build-plugins (3.11, flytekit-dolt) GitHub Actions: build-plugins (3.11, flytekit-deck-standard) GitHub Actions: build-plugins (3.11, flytekit-dbt) GitHub Actions: build-plugins (3.11, flytekit-data-fsspec) GitHub Actions: build-plugins (3.11, flytekit-dask) GitHub Actions: build-plugins (3.11, flytekit-bigquery) GitHub Actions: build-plugins (3.11, flytekit-aws-sagemaker) GitHub Actions: build-plugins (3.11, flytekit-aws-batch) GitHub Actions: build-plugins (3.11, flytekit-aws-athena) GitHub Actions: build-plugins (3.10, flytekit-vaex) GitHub Actions: build-plugins (3.10, flytekit-sqlalchemy) GitHub Actions: build-plugins (3.10, flytekit-spark) ✅ No checks have passed 0/30 successful checks

#3516 [Docs] Clarify what Resource is used when doing FlyteFile/Directory.download() or current_context().working_dir Issue created by cosmicBboy Description We need to mention in flytekit and flytesnacks that the

Resource(ephemeral_storage=...)

request is relevant when the user writes data to the task pod's disk. • flytesnacks: https://docs.flyte.org/projects/cookbook/en/latest/auto/deployment/customizing_resources.html • flytekit: • https://docs.flyte.org/projects/flytekit/en/latest/generated/flytekit.Resources.html#flytekit-resources • https://docs.flyte.org/projects/flytekit/en/latest/generated/flytekit.types.file.FlyteFile.html#flytekit.types.file.FlyteFile.download • https://docs.flyte.org/projects/flytekit/en/latest/generated/flytekit.types.directory.FlyteDirectory.html#flytekit.types.directory.FlyteDirectory: this is missing documentation on the

download

method • https://docs.flyte.org/projects/flytekit/en/latest/generated/flytekit.ExecutionParameters.html#flytekit.ExecutionParameters: this is missing documentation on the

working_directory

property Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

1 new commit pushed to

<https://github.com/flyteorg/flyte/tree/master|master>

by wild-endeavor

<https://github.com/flyteorg/flyte/commit/7479b56888d34f2c131e1f16c313ad34bed90484|7479b568>

- Revert "Split flyte-binary services into http and grpc, and some cleanups (#3495)" (#3517) flyteorg/flyte

Release - Flyte v1.4.3 milestone release New release published by flyte-bot Flyte 1.4.3 release This patch release pulls in the compiler changes necessary to compile gate nodes and the fix for StructuredDataset in flyteorg/flyteadmin#541. flyteorg/flyte

Deployment to github-pages by flyte-bot flyteorg/flyte

1 new commit pushed to

<https://github.com/flyteorg/flyteconsole/tree/master|master>

by ursucarina

<https://github.com/flyteorg/flyteconsole/commit/35c3b8c992690590327e7d0dcdcc1ff89c729fa0|35c3b8c9>

- chore: task observability (#720) flyteorg/flyteconsole