#1618 Read polar dataframe without copying to local Pull request opened by pingsutw TL;DR fixed https://unionai.slack.com/archives/C02CMUNT4PQ/p1683221575139199 Read polar dataframe without copying to local Type ☑︎ Bug Fix ☐ Feature ☐ Plugin Are all requirements met? ☑︎ Code completed ☑︎ Smoke tested ☐ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description ^^^ Tracking Issue https://unionai.slack.com/archives/C02CMUNT4PQ/p1683221575139199 Follow-up issue NA flyteorg/flytekit GitHub Actions: build-plugins (3.8, flytekit-aws-batch) GitHub Actions: build-plugins (3.8, flytekit-aws-athena) GitHub Actions: build (windows-latest, 3.11) GitHub Actions: build (windows-latest, 3.9) GitHub Actions: build (windows-latest, 3.8) GitHub Actions: build (ubuntu-latest, 3.11) GitHub Actions: build (ubuntu-latest, 3.10) GitHub Actions: build (ubuntu-latest, 3.9) GitHub Actions: build (ubuntu-latest, 3.8) GitHub Actions: lint GitHub Actions: Docs Warnings GitHub Actions: docs ✅ 2 other checks have passed 2/14 successful checks

#989 update sphinx youtube plugin Pull request opened by cosmicBboy Use a maintained package for the sphinx youtube plugin flyteorg/flytesnacks GitHub Actions: Mark github pre-release as Release GitHub Actions: Publish artifacts to github release GitHub Actions: Create Prerelease GitHub Actions: Bump Version ✅ 26 other checks have passed 26/30 successful checks

#3640 update sphinx youtube plugin Pull request opened by pingsutw Use a maintained package for the sphinx youtube plugin flyteorg/flyte ✅ All checks have passed 6/6 successful checks

#1619 update sphinx youtube plugin Pull request opened by pingsutw Use a maintained package for the sphinx youtube plugin flyteorg/flytekit GitHub Actions: build-plugins (3.10, flytekit-vaex) ✅ 29 other checks have passed 29/30 successful checks

1 new commit pushed to

<https://github.com/flyteorg/flytekit/tree/master|master>

by pingsutw

<https://github.com/flyteorg/flytekit/commit/c03635db118a4612afe823b9a7fa83f928c034c3|c03635db>

- update sphinx youtube plugin (#1619) flyteorg/flytekit

1 new commit pushed to

<https://github.com/flyteorg/flytekit/tree/master|master>

by pingsutw

<https://github.com/flyteorg/flytekit/commit/b1406f8c838d61574bb4c706175fa8d46d3edb66|b1406f8c>

- Read polar dataframe without copying to local (#1618) flyteorg/flytekit

1 new commit pushed to

<https://github.com/flyteorg/flytetools/tree/master|master>

by pingsutw

<https://github.com/flyteorg/flytetools/commit/76058add494b503b493d12ea30f548cc7d23eb7e|76058add>

- Login to GitHub registry only when push is enabled (#73) flyteorg/flytetools

#73 Login to GitHub registry only when push is enabled Pull request opened by pingsutw Publish workflow will try to run docker login first. However, this task will fail if people create a pr from fork repo. Here is an example. Login to GitHub registry if needed. I've tested here. flyteorg/flyteconsole@606d54e flyteorg/flytetools ✅ All checks have passed 2/2 successful checks

#3641 [BUG] Launch plan icon doesn't display correctly Issue created by pingsutw Describe the bug I use latest flyteConsole To reproduce, go to execution detail page, and click previous page button. correct one: Expected behavior display correctly Additional context to reproduce No response Screenshots No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

1 new commit pushed to

<https://github.com/flyteorg/flytekit/tree/master|master>

by pingsutw

<https://github.com/flyteorg/flytekit/commit/f2cfef0ec098d4ae8f042ab915b0b30d524092c6|f2cfef0e>

- Add default spark image (#1616) flyteorg/flytekit

1 new commit pushed to

<https://github.com/flyteorg/flyte/tree/master|master>

by pingsutw

<https://github.com/flyteorg/flyte/commit/7e0c3edc6781c4d0e909efc64bfd77d863f88771|7e0c3edc>

- update sphinx youtube plugin (#3640) flyteorg/flyte

1 new commit pushed to

<https://github.com/flyteorg/flyteconsole/tree/master|master>

by ursucarina

<https://github.com/flyteorg/flyteconsole/commit/e95faa89bf53c9b20ad34a89c7aededabe8e724e|e95faa89>

- chore: propagate dynamic parent id (#751) flyteorg/flyteconsole

#556 Add environment variables to execution spec Pull request opened by pingsutw TL;DR blocked by flyteorg/flyteidl#400 Add environment variables to execution spec, allowing people to add env at execution time • Create execution with environment variables • Override environment variables when relaunching • Override environment variables when recovering Type ☐ Bug Fix ☑︎ Feature ☐ Plugin Are all requirements met? ☑︎ Code completed ☐ Smoke tested ☐ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description ^^^ Tracking Issue flyteorg/flyte#2447 Follow-up issue NA flyteorg/flyteadmin GitHub Actions: Build & Push Flytescheduler Image GitHub Actions: Goreleaser GitHub Actions: Build & Push Flyteadmin Image GitHub Actions: Bump Version ✅ 10 other checks have passed 10/14 successful checks

1 new commit pushed to

<https://github.com/flyteorg/flytesnacks/tree/master|master>

by pingsutw

<https://github.com/flyteorg/flytesnacks/commit/1a54dfcd0d7d8226e05b8d842760bdb08a4273ee|1a54dfcd>

- update sphinx youtube plugin (#989) flyteorg/flytesnacks

Release - flytesnacks v0.3.192 New release published by flyte-bot Changelog • 1a54dfc update sphinx youtube plugin (#989) • 67c8c41 Update vault secret manager documentation (#986) • 094211a fix whylogs docs (#985) flyteorg/flytesnacks

#752 Support uploading files to Flyte blob store in the Flyte Console launch form Pull request opened by james-union flyteorg/flyte#3317 TL;DR Added FileInput Type • [] Bug Fix ☑︎ Feature ☐ Plugin Are all requirements met? ☐ Code completed ☐ Smoke tested ☐ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Tracking Issue fixes flyteorg/flyte#3317 flyteorg/flyteconsole GitHub Actions: Build Flyteconsole Image / Push to Github Registry GitHub Actions: extract_branch GitHub Actions: unit_tests_with_coverage GitHub Actions: lint_project ✅ 2 other checks have passed 2/6 successful checks

Release - Add default spark image New release published by pingsutw #1616 flyteorg/flytekit

1 new commit pushed to

<https://github.com/flyteorg/flytekit/tree/master|master>

by pingsutw

<https://github.com/flyteorg/flytekit/commit/a1636a89daee2301a056b3eba41fe4dcea1257b7|a1636a89>

- Lazy load modules (#1590) flyteorg/flytekit

#971 Doc for external plugin service Pull request opened by pingsutw on <!date^1678905603^{date_short}|2023-03-15T18:40:03Z> This doc includes • Overview • how to register a backend plugin in python • how to build a image for the external plugin service • How to enable external plugin service https://flyte--971.org.readthedocs.build/projects/cookbook/en/971/auto/core/extend_flyte/external_plugin_service.html flyteorg/flytesnacks GitHub Actions: Mark github pre-release as Release GitHub Actions: Publish artifacts to github release GitHub Actions: Create Prerelease GitHub Actions: Bump Version ✅ 26 other checks have passed 26/30 successful checks

1 new commit pushed to

<https://github.com/flyteorg/flytepropeller/tree/master|master>

by hamersaw

<https://github.com/flyteorg/flytepropeller/commit/f4cadb0062320086096306c8d6e5a1e412090ef8|f4cadb00>

- Fast fail if task resource requests exceed k8s resource limits (#488) flyteorg/flytepropeller

#2933 [Core feature] Fast fail tasks if resource quota request exceeds k8s quota limits Issue created by hamersaw Motivation: Why do you think this is important? Currently, FlytePropeller detects resource quota issues using k8s error messages with code 403 and "exceeds quota" in the message. However, this approach is unable to differentiate between scenarios where: (1) the requested quotas + the currently used quotas are more than k8s resource quota limits (2) the requested quotas exceed k8s resource quota limits When detecting resource quota issues, FlytePropeller periodically attempts to schedule the task. In the former case, this will succeed once other tasks complete and resources become available. However, in the later case, the resources will never be available and FlytePropeller leaves the task in "waiting" state until the node is terminated based on the

node-active-deadline

(which defaults to 48h). Goal: What should the final outcome look like, ideally? Tasks which request resources larger than the available k8s resource quota limits should immediately fail rather than periodically attempt to schedule. Describe alternatives you've considered FlyteAdmin exposes a number of configuration options (ex.

task-resource-defaults

) which stop users from registering tasks with resource quotas larger than the configured values. However, these are typically register-time checks and do not apply if FlyteAdmin (1) is not configured to manage cluster resources or (2) k8s resource quotas (or admin configuration) is changed after task registration. Additionally, handling these issues at runtime means diverse environments (many projects / domains) can be handled independently. Propose: Link/Inline OR Additional context The FlytePropeller backoff handler code has a GetComputeResourceAndQuantityRequested function which parses the exceeded resource quota error messages to expose requests, used, and limits on resource quotas. If FlytePropeller detects an attempted task execution exceeds resource quotas (ex. here) it should then check if the task resource quotas exceed k8s resource quota limits, and if so fail immediately rather than attempt to execute when more resources are available. Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#488 Fast fail if task resource requests exceed k8s resource limits Pull request opened by hamersaw TL;DR When encountering a "ResourceExceedsLimits" error from k8s we validate that the task resource requests and limits are below the k8s resource quota. Otherwise the task will never be schedule-able and will forever hang until FlytePropeller terminates it based on

node-active-duration

. Type ☐ Bug Fix ☑︎ Feature ☐ Plugin Are all requirements met? ☑︎ Code completed ☑︎ Smoke tested ☑︎ Unit tests added ☑︎ Code documentation added ☑︎ Any pending items have an associated Issue Complete description ^^^ Tracking Issue fixes flyteorg/flyte#2933 Follow-up issue NA flyteorg/flytepropeller GitHub Actions: Build & Push Flytepropeller Image GitHub Actions: Goreleaser GitHub Actions: Bump Version ✅ 11 other checks have passed 11/14 successful checks

1 new commit pushed to

<https://github.com/flyteorg/flytekit/tree/master|master>

by pingsutw

<https://github.com/flyteorg/flytekit/commit/0b0de279d8eab44442027de5b11615f1963a7e98|0b0de279>

- Override node name with a name containing underscore (#1608) flyteorg/flytekit

1 new commit pushed to

<https://github.com/flyteorg/flyteadmin/tree/master|master>

by wild-endeavor

<https://github.com/flyteorg/flyteadmin/commit/950ad1522aa8697777b86bb39e5df73eaf64e530|950ad152>

- Address resolution (#546) flyteorg/flyteadmin

Release - v1.1.88 New release published by flyte-bot Changelog • f4cadb0 Fast fail if task resource requests exceed k8s resource limits (#488) flyteorg/flytepropeller

Release - v1.1.91 New release published by flyte-bot Changelog • 950ad15 Address resolution (#546) flyteorg/flyteadmin

#1621 Include traceback in errors from admin Pull request opened by wild-endeavor TL;DR We were returning too early and not printing the actual error. Now we do. Type ☑︎ Bug Fix ☐ Feature ☐ Plugin Are all requirements met? ☑︎ Code completed ☑︎ Smoke tested ☐ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description Also make things bold. flyteorg/flytekit Codecov: 0.00% of diff hit (target 70.17%) ✅ 29 other checks have passed 29/30 successful checks

#1622 Add support for copying all the files in source root Pull request opened by pingsutw TL;DR Fixes https://flyte-org.slack.com/archives/CP2HDHKE1/p1683114107382779 Add a new flag (--copy-all) to pyflyte run. it allows user to copy all the files in the source root directory to remote. Type ☑︎ Bug Fix ☐ Feature ☐ Plugin Are all requirements met? ☑︎ Code completed ☑︎ Smoke tested ☐ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description ^^^ Tracking Issue https://flyte-org.slack.com/archives/CP2HDHKE1/p1683114107382779 Follow-up issue NA flyteorg/flytekit Codecov: 50.00% of diff hit (target 71.16%) ✅ 29 other checks have passed 29/30 successful checks

1 new commit pushed to

<https://github.com/flyteorg/flyteidl/tree/master|master>

by eapolinario

<https://github.com/flyteorg/flyteidl/commit/44b6bc64337cb40b08865ab6c1c8762f4d9619e4|44b6bc64>

- add user_identifier (#388) flyteorg/flyteidl

#388 Add userid to security context Pull request opened by ByronHsu TL;DR Add execution userid to security context Type ☐ Bug Fix ☑︎ Feature ☐ Plugin Tests Did an e2e test with flyteorg/flyteadmin#549

user_identifier

was successfully injected to flyteCRD.

securityContext:
  run_as:
    user_identifier: byhsu

Are all requirements met? ☐ Code completed ☐ Smoke tested ☐ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description How did you fix the bug, make the feature etc. Link to any design docs etc Tracking Issue _Remove the '_fixes_' keyword if there will be multiple PRs to fix the linked issue_ flyteorg/flyte#3566 flyteorg/flyteidl ✅ All checks have passed 13/13 successful checks