Release - v1.1.0 New release published by flyte-bot Changelog • 53e63bc Feat: Add pod start and finish time in RFC3339 time format to logging link templating variables #minor (#360) flyteorg/flyteplugins

Release - Flyte v1.7.0 milestone release New release published by flyte-bot Flyte v1.7.0 release In this release we're announcing support for Flyte Agents, a new way of writing backend plugins, only now with a much more tightly integrated developer experience. Also lots of bug fixes all around in a buch of first-time contributors. Flytekit • FLYTECTL_CONFIG env var should take highest precedence in flyteorg/flytekit#1662 by @wild-endeavor • Change flytekit Pytorch, TFJob and MPI plugins to use new kubeflow config in flyteorg/flytekit#1627 by @yubofredwang • Root cert should be byte string when loading from caCertFilePath in flyteorg/flytekit#1669 by @wild-endeavor • Explicitly set the content type for flyte deck by in flyteorg/flytekit#1658 @pingsutw • Set a less strict deadline for hypothesis tests in flyteorg/flytekit#1682 by @eapolinario • Dep: Use protos of new

kubeflow.pytorch

plugin instead of legacy

pytorch

plugin in flyteorg/flytekit#1678 by @fg91 • More time info for time line deck in flyteorg/flytekit#1680 by @Yicheng-Lu-llll Flyteadmin • Add logs and stats to sync loop in flyteorg/flyteadmin#573 by Haytham Abuelfutuh • Fix utf-8 encoding issues with trimmed error messages in flyteorg/flyteadmin#569 by Katrina Rogan • Tiny url improvements in flyteorg/flyteadmin#565 by Yee Hing Tong • Update startedAt timestamp only if not set in flyteorg/flyteadmin#567 pmahindrakar-oss • Save execution namespace in system metadata in flyteorg/flyteadmin#568 by Katrina Rogan Flyteconsole • fix: navlink in flyteorg/flyteconsole#772 by 4nalog • fix: bump version for console in flyteorg/flyteconsole#769 by 4nalog • fix: preserve domain when navigating using sidebar in flyteorg/flyteconsole#768 by 4nalog • fix: dynamic-node-tasks in flyteorg/flyteconsole#765 by 4nalog • chore: hide map task runtime info in https://github.com/flyteorg/flyteconsole7/66 by Carina Ursu • Bug: union val missing for LP relaunch in flyteorg/flyteconsole#762 by Frank Flitton • Feature: Fullview Flyte Deck modal in flyteorg/flyteconsole#764 by Frank Flitton • chore: add item when mapped task in flyteorg/flyteconsole#761 by Jason Porter • Bug: Execution Page's back button returns Workflows route from Launch Plan route https://github.com/flyteorg/flyteconsolepatch in flyteorg/flyteconsole#760 by Frank Flitton Flytepropeller • Register gRPC plugin after reading configmap in flyteorg/flytepropeller#564 by Kevin Su • Not stripping structure from literal types in flyteorg/flytepropeller#571 by Dan Rammer • Bump flyteplugins to v1.0.63 in flyteorg/flytepropeller#568 by bstadlbauer • bumped flyteplugins in flyteorg/flytepropeller#566 by Dan Rammer • Use correct k8 client in flyteorg/flytepropeller#563 by sonjaer flyteorg/flyte

Deployment to github-pages by flyte-bot flyteorg/flyte

1 new commit pushed to

<https://github.com/flyteorg/flytekit/tree/master|master>

by eapolinario

<https://github.com/flyteorg/flytekit/commit/a68cb34799237f6fd52396013b11bfcf02fbab7d|a68cb347>

- upload deck.html only with deck enable (#1693) flyteorg/flytekit

1 new commit pushed to

<https://github.com/flyteorg/flytekit/tree/master|master>

by eapolinario

<https://github.com/flyteorg/flytekit/commit/9977aac26242ebbede8e00d476c2fbc59ac5487a|9977aac2>

- Local agent (#1687) flyteorg/flytekit

#1683 Upload Flytekit runtime metrics to S3 Pull request opened by Yicheng-Lu-llll TL;DR This is part of Flytekit Metrics Exploration. This PR uploads all the flytekit metrics to the remote storage(highlighted in green). Flytekit Metrics Exploration includes: • flytekit: #1683 • flyteplugins: flyteorg/flyteplugins#361 • flyteidl: flyteorg/flyteidl#415 • flytepropeller: flyteorg/flytepropeller#577 • flyteadmin: flyteorg/flyteadmin#575 Checks ran task in the sand box, the span file is under the same folder with deck.html. flyteorg/flytekit ✅ All checks have passed 2/2 successful checks

#1694 get_transformer returns pickle transformer if type is unsupported Pull request opened by pingsutw TL;DR wip Type ☐ Bug Fix ☐ Feature ☐ Plugin Are all requirements met? ☐ Code completed ☐ Smoke tested ☐ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description How did you fix the bug, make the feature etc. Link to any design docs etc Tracking Issue https://github.com/flyteorg/flyte/issues/ Follow-up issue NA OR https://github.com/flyteorg/flyte/issues/ flyteorg/flytekit ✅ All checks have passed 1/1 successful checks

Release - v1.7.0 New release published by eapolinario What's Changed • FLYTECTL_CONFIG env var should take highest precedence by @wild-endeavor in #1662 • Change flytekit Pytorch, TFJob and MPI plugins to use new kubeflow config by @yubofredwang in #1627 • Root cert should be byte string when loading from caCertFilePath by @wild-endeavor in #1669 • Explicitly set the content type for flyte deck by @pingsutw in #1658 • Set a less strict deadline for hypothesis tests by @eapolinario in #1682 • Dep: Use protos of new

kubeflow.pytorch

plugin instead of legacy

pytorch

plugin by @fg91 in #1678 • More time info for time line deck by @Yicheng-Lu-llll in #1680 • Rename external plugin to agent by @pingsutw in #1666 • feat: Add Auth0/audience support for ClientCredentials flow by @PudgyPigeon in #1639 • pyflyte run remote file by @ChungYujoyce in #1670 • Upload 'deck.html' only when the deck is enabled by @Yicheng-Lu-llll in #1693 • Local agent by @pingsutw in #1687 New Contributors • @yubofredwang made their first contribution in #1627 • @PudgyPigeon made their first contribution in #1639 • @ChungYujoyce made their first contribution in #1670 Full Changelog: v1.6.2...v1.7.0 flyteorg/flytekit

#361 Add GetSpanPath and SpanExists Pull request opened by Yicheng-Lu-llll TL;DR This is part of Flytekit Metrics Exploration. This PR helps propeller to get span uri(highlighted in green). Flytekit Metrics Exploration includes: • flytekit: flyteorg/flytekit#1683 • flyteplugins: #361 • flyteidl: flyteorg/flyteidl#415 • flytepropeller: flyteorg/flytepropeller#577 • flyteadmin: flyteorg/flyteadmin#575 flyteorg/flyteplugins ✅ All checks have passed 2/2 successful checks

1 new commit pushed to

<https://github.com/flyteorg/flyte/tree/master|master>

by katrogan

<https://github.com/flyteorg/flyte/commit/bdce6e6f0afeeb62acb2173649f5d69820d6f9b2|bdce6e6f>

- Allow setting resources for cluster resource controller deployment (#3762) flyteorg/flyte

#576 Bump github.com/lestrrat-go/jwx from 1.1.6 to 1.2.26 Pull request opened by dependabot[bot] Bumps github.com/lestrrat-go/jwx from 1.1.6 to 1.2.26. Release notes Sourced from github.com/lestrrat-go/jwx's releases.

[SECURITY] v1.2.26

```

v1.2.26 - 14 Jun 2023

[Security]

* Potential Padding Oracle Attack Vulnerability and Timing Attack Vulnerability

for JWE AES-CBC encrypted payloads affecting all v2 releases up to v2.0.10,

all v1 releases up to v1.2.25, and all v0 releases up to v0.9.2 have been reported by

@shogo82148.

Please note that v0 versions will NOT receive fixes.

This release fixes these vulnerabilities for the v1 series.

```

v1.2.25

```

v1.2.25 23 May 2022

[Bug Fixes][Security]

* [jwe] An old bug from at least 7 years ago existed in handling AES-CBC unpadding,

where the unpad operation might remove more bytes than necessary ([#744](https://github.com/lestrrat-go/jwx/issues/744))

This affects all jwx code that is available before v2.0.2 and v1.2.25.

```

v1.2.24

```

v1.2.24 05 May 2022

[Security]

* Upgrade golang.org/x/crypto ([#724](https://github.com/lestrrat-go/jwx/issues/724))

```

v1.2.23

```

v1.2.23 13 Apr 2022

[Bug fixes]

* [jwk] jwk.AutoRefresh had a race condition when

Configure()

was

called concurrently ([#686](https://github.com/lestrrat-go/jwx/issues/686))

(It has been patched correctly, but we may come back to revisit

the design choices in the near future)

```

v1.2.22

```

v1.2.22 08 Apr 2022

[Bug fixes]

* [jws] jws.Verify was ignoring the

b64

header when it was present

in the protected headers ([#681](https://github.com/lestrrat-go/jwx/issues/681)). Now the following should work:

jws.Sign(..., jws.WithDetachedPayload(payload))

// previously payload had to be base64 encoded

jws.Verify(..., jws.WithDetachedPayload(payload))

```

... (truncated) Changelog Sourced from github.com/lestrrat-go/jwx's changelog.

v1.2.26 - 14 Jun 2023 [Security]

• Potential Padding Oracle Attack Vulnerability and Timing Attack Vulnerability for JWE AES-CBC encrypted payloads affecting all v2 releases up to v2.0.10, all v1 releases up to v1.2.25, and all v0 releases up to v0.9.2 have been reported by `@​shogo82148`.

Please note that v0 versions will NOT receive fixes. This release fixes these vulnerabilities for the v1 series.

[Miscellaneous]

• JWE tests now only run algorithms that are supported by the underlying

jose

tool

v1.2.25 23 May 2022 [Bug Fixes][Security]

• [jwe] An old bug from at least 7 years ago existed in handling AES-CBC unpadding, where the unpad operation might remove more bytes than necessary (#744) This affects all jwx code that is available before v2.0.2 and v1.2.25.

v1.2.24 05 May 2022 [Security]

• Upgrade golang.org/x/crypto (#724)

v1.2.23 13 Apr 2022 [Bug fixes]

• [jwk] jwk.AutoRefresh had a race condition when

Configure()

was called concurrently (#686) (It has been patched correctly, but we may come back to revisit the design choices in the near future)

v1.2.22 08 Apr 2022 [Bug fixes]

• [jws] jws.Verify was ignoring the

b64

header when it was present in the protected headers (#681). Now the following should work:

jws.Sign(..., jws.WithDetachedPayload(payload)) // previously payload had to be base64 encoded jws.Verify(..., jws.WithDetachedPayload(payload))

(note: v2 branch was not affected)

v1.2.21 30 Mar 2022 [Bug fixes]

• [jwk] RSA keys without p and q can now be parsed.

v1.2.20 03 Mar 2022 [Miscellaneous]

• Dependency on golang.org/x/crypto has been upgraded to v0.0.0-20220214200702-86341886e292 to address

... (truncated) Commits • `d9ddbc8` merge v1 (#936) • `ad8c29d` merge develop/v1 (#747) • `e38f677` Merge develop/v1 (#727) • `baba561` Merge branch 'develop/v1' into v1 • `8ff6c75` Update Changes • `ea97e8c` Fix race in

jwk.AutoRefresh

(#686) • `f4701e1` Update Changes • `e831228` Fix jws.Verify not respecting the b64 header in the protected headers (#683) • `b66a2cb` backport: Update golangci lint (#679) (#680) • `4899c32` reword error • Additional commits viewable in compare view Dependabot compatibility score Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting

@dependabot rebase

. * * * Dependabot commands and options You can trigger Dependabot actions by commenting on this PR: •

@dependabot rebase

will rebase this PR •

@dependabot recreate

will recreate this PR, overwriting any edits that have been made to it •

@dependabot merge

will merge this PR after your CI passes on it •

@dependabot squash and merge

will squash and merge this PR after your CI passes on it •

@dependabot cancel merge

will cancel a previously requested merge and block automerging •

@dependabot reopen

will reopen this PR if it is closed •

@dependabot close

will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually •

@dependabot ignore this major version

will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) •

@dependabot ignore this minor version

will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) •

@dependabot ignore this dependency

will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the Security Alerts page. flyteorg/flyteadmin GitHub Actions: End2End Test / End to End tests GitHub Actions: Integration Test / Integration tests ✅ 8 other checks have passed 8/10 successful checks

#3781 Adjust MPI operator installation doc Pull request opened by iaroslav-ciupin Tracking issue Describe your changes Check all the applicable boxes ☑︎ I updated the documentation accordingly. ☐ All new and existing tests passed. ☑︎ All commits are signed-off. Screenshots Note to reviewers flyteorg/flyte ✅ All checks have passed 6/6 successful checks

1 new commit pushed to

<https://github.com/flyteorg/flyte/tree/master|master>

by iaroslav-ciupin

<https://github.com/flyteorg/flyte/commit/201a8e1450d86b72a67be938ce7502ee2621cadb|201a8e14>

- adjust MPI operator installation doc (#3781) flyteorg/flyte

#575 Implement GetFlyteKitMetrics endpoint Pull request opened by Yicheng-Lu-llll TL;DR This is part of Flytekit Metrics Exploration. This PR Implements GetFlyteKitMetrics endpoint: • Download flytekit spans for a particular node from remote storage. • Send flytekit spans to console. Type ☐ Bug Fix ☐ Feature ☐ Plugin Are all requirements met? ☐ Code completed ☐ Smoke tested ☐ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description How did you fix the bug, make the feature etc. Link to any design docs etc Tracking Issue _Remove the '_fixes_' keyword if there will be multiple PRs to fix the linked issue_ fixes https://github.com/flyteorg/flyte/issues/ Follow-up issue NA OR https://github.com/flyteorg/flyte/issues/ flyteorg/flyteadmin ✅ All checks have passed 2/2 successful checks

#577 Add span uri to node event Pull request opened by Yicheng-Lu-llll TL;DR This is part of Flytekit Metrics Exploration. This PR adds span uri to node event(highlighted in green). Flytekit Metrics Exploration includes: • flytekit: flyteorg/flytekit#1683 • flyteplugins: flyteorg/flyteplugins#361 • flyteidl: flyteorg/flyteidl#415 • flytepropeller: #577 • flyteadmin: flyteorg/flyteadmin#575 flyteorg/flytepropeller ✅ All checks have passed 2/2 successful checks

#415 Add span uri Pull request opened by Yicheng-Lu-llll TL;DR This is part of Flytekit Metrics Exploration This PR(also highlighted in green): • adds span uri field to NodeExecutionEvent and NodeExecutionClosure. • adds a new endpoint

GetFlyteKitMetrics

. Flytekit Metrics Exploration includes: • flytekit: flyteorg/flytekit#1683 • flyteplugins: flyteorg/flyteplugins#361 • flyteidl: #415 • flytepropeller: flyteorg/flytepropeller#577 • flyteadmin: flyteorg/flyteadmin#575 Type ☐ Bug Fix ☐ Feature ☐ Plugin Are all requirements met? ☐ Code completed ☐ Smoke tested ☐ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description How did you fix the bug, make the feature etc. Link to any design docs etc Tracking Issue _Remove the '_fixes_' keyword if there will be multiple PRs to fix the linked issue_ fixes https://github.com/flyteorg/flyte/issues/ Follow-up issue NA OR https://github.com/flyteorg/flyte/issues/ flyteorg/flyteidl ✅ All checks have passed 2/2 successful checks

1 new commit pushed to

<https://github.com/flyteorg/flyteplugins/tree/master|master>

by hamersaw

<https://github.com/flyteorg/flyteplugins/commit/318fa6b0e86495c144517f2a852362a31765bdc6|318fa6b0>

- fix v1 pytorch job plugin with elastic policy (#359) flyteorg/flyteplugins

Release - v1.1.1 New release published by flyte-bot Changelog • 318fa6b fix v1 pytorch job plugin with elastic policy (#359) flyteorg/flyteplugins

#362 Feat: Make pod start time available als logging link template variable in kubeflow plugin Pull request opened by fg91 TL;DR I'd like to add the filter

timestamp >= podStartTime

to our stackdriver log link template. This is important because currently our users have to manually extend the time range (default 1h) each time they visit the stackdriver log page. (Unless the job started and finished within the past hour.) To this aim, I recently made the pod start and finish time available as logging link template variables in RFC3339 time format which is required by google cloud logging (#360). Currently, the kubeflow plugin doesn't pass the pod start time to the log plugin though. In this PR I extend the kubeflow plugin to do so. Type ☐ Bug Fix ☑︎ Feature ☐ Plugin Are all requirements met? ☑︎ Code completed ☑︎ Smoke tested ☑︎ Unit tests added ☐ Code documentation added ☑︎ Any pending items have an associated Issue Complete description NA Tracking Issue NA Follow-up issue NA flyteorg/flyteplugins ✅ All checks have passed 7/7 successful checks

#232 Bump guava from 31.1-jre to 32.0.0-jre Pull request opened by dependabot[bot] Bumps guava from 31.1-jre to 32.0.0-jre. Release notes Sourced from guava's releases.

32.0.0

Maven

```

<dependency>

<groupId>com.google.guava</groupId>

<artifactId>guava</artifactId>

<version>32.0.0-jre</version>

<!-- or, for Android: -->

<version>32.0.0-android</version>

</dependency>

```

Jar files

• 32.0.0-jre.jar

• 32.0.0-android.jar

Guava requires one runtime dependency, which you can download here:

• failureaccess-1.0.1.jar

Javadoc

• 32.0.0-jre

• 32.0.0-android

JDiff

• 32.0.0-jre vs. 31.1-jre

• 32.0.0-android vs. 31.1-android

• 32.0.0-android vs. 32.0.0-jre

Changelog

Security fixes

• Reimplemented

Files.createTempDir

and

FileBackedOutputStream

to further address CVE-2020-8908 (#4011) and CVE-2023-2976 (#2575). (feb83a1c8f)

While CVE-2020-8908 was officially closed when we deprecated

Files.createTempDir

in Guava 30.0, we've heard from users that even recent versions of Guava have been listed as vulnerable in other databases of security vulnerabilities. In response, we've reimplemented the method (and the very rarely used

FileBackedOutputStream

class, which had a similar issue) to eliminate the insecure behavior entirely. This change could technically affect users in a number of different ways (discussed under "Incompatible changes" below), but in practice, the only problem users are likely to encounter is with Windows. If you are using those APIs under Windows, you should skip 32.0.0 and go straight to 32.0.1 which fixes the problem. (Unfortunately, we didn't think of the Windows problem until after the release. And while we warn that `common.io` in particular may not work under Windows, we didn't intend to regress support.) Sorry for the trouble.

Incompatible changes

Although this release bumps Guava's major version number, it makes no binary-incompatible changes to the

guava

artifact.

One change could cause issues for Widows users, and a few other changes could cause issues for users in more usual situations:

• The new implementations of

Files.createTempDir

and

FileBackedOutputStream

throw an exception under Windows. This is fixed in 32.0.1. Sorry for the trouble.

• This release makes a binary-incompatible change to a

@Beta

API in the separate artifact

guava-testlib

. Specifically, we changed the return type of

TestingExecutors.sameThreadScheduledExecutor

to

ListeningScheduledExecutorService

. The old return type was a package-private class, which caused the Kotlin compiler to produce warnings. (dafaa3e435)

• This release adds two methods to the Android flavor of Guava:

Invokable.getAnnotatedReturnType()

and

Parameter.getAnnotatedType()

. Those methods do not work under an Android VM; we added them only to help our tests of the Android flavor (since we also run those tests under a JRE). Android VMs tolerate such methods as long as the app does not call them or perform reflection on them, and builds tolerate them because of our new Proguard configurations (discussed below). Thus, we expect no impact to most users. However, we could imagine build problems for users who have set up their own build system for the Android flavor of Guava. Please report any problems so that we can judge how safely we might be able to add other methods to the Android flavor in the future, such as APIs that use Java 8 classes like

Stream

. (b30e73cfa81ad15c1023c17cfd083255a3df0105)

... (truncated) Commits • See full diff in compare view Dependabot compatibility score Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting

@dependabot rebase

. * * * Dependabot commands and options You can trigger Dependabot actions by commenting on this PR: •

@dependabot rebase

will rebase this PR •

@dependabot recreate

will recreate this PR, overwriting any edits that have been made to it •

@dependabot merge

will merge this PR after your CI passes on it •

@dependabot squash and merge

will squash and merge this PR after your CI passes on it •

@dependabot cancel merge

will cancel a previously requested merge and block automerging •

@dependabot reopen

will reopen this PR if it is closed •

@dependabot close

will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually •

@dependabot ignore this major version

will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) •

@dependabot ignore this minor version

will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) •

@dependabot ignore this dependency

will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the Security Alerts page. flyteorg/flytekit-java GitHub Actions: build-snapshot ✅ 2 other checks have passed 2/3 successful checks

#233 [Snyk] Upgrade com.google.auto.service:auto-service-annotations from 1.0.1 to 1.1.0 Pull request opened by EngHabu This PR was automatically created by Snyk using the credentials of a real user. Snyk has created this PR to upgrade com.google.auto.service:auto-service-annotations from 1.0.1 to 1.1.0. ℹ️ Keep your dependencies up-to-date. This makes it easier to fix existing vulnerabilities and to more quickly identify and fix newly disclosed vulnerabilities when they affect your project. * * * • The recommended version is 2 versions ahead of your current version. • The recommended version was released 22 days ago, on 2023-05-23. * * * Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open upgrade PRs. For more information: https://camo.githubusercontent.com/94cb4da59e745f1118c0010e1d4f7585e6ce5ff781820970baf5965792b90b65/68747470733a2f2f6170692e7365676d656e742e696f2f76312f706978656c2f747261636b3f646174613d65794a33636d6c305a55746c65534936496e4a79576d785a634564485932527954485a7362306c596430645563566734576b4652546e4e434f5545774969776959573576626e6c746233567a535751694f694a684d324d32595451774d7930784e5449784c5452694d5745744f57497859693079596d5a69595755305a6d4934595459694c434a6c646d567564434936496c425349485a705a58646c5a434973496e42796233426c636e52705a584d694f6e736963484a4a5a434936496d457a597a5a684e44417a4c5445314d6a45744e47497859533035596a46694c544a695a6d4a685a54526d596a68684e694a3966513d3d 🧐 View latest project report 🛠️ Adjust upgrade PR settings 🔕 Ignore this dependency or unsubscribe from future upgrade PRs flyteorg/flytekit-java GitHub Actions: build-snapshot DCO: DCO ✅ 1 other check has passed 1/3 successful checks

1 new commit pushed to

<https://github.com/flyteorg/flytekit/tree/master|master>

by pingsutw

<https://github.com/flyteorg/flytekit/commit/207d01986d744f8212a3e2e28826ce947efa96d9|207d0198>

- Support configuring pip index url with the image spec (#1692) flyteorg/flytekit

1 new commit pushed to

<https://github.com/flyteorg/flytesnacks/tree/master|master>

by samhita-alla

<https://github.com/flyteorg/flytesnacks/commit/374d29e89c12142b24ba0fd5aab3f0f4b4a9a3e6|374d29e8>

- Image spec example (#988) flyteorg/flytesnacks

Release - flytesnacks v0.3.199 New release published by flyte-bot Changelog • 374d29e Image spec example (#988) • 4b031a5 Update tasks_and_workflows.md (#997) flyteorg/flytesnacks

#1695 Feat: Allow disabling rich tracebacks via env var Pull request opened by fg91 TL;DR In

flytekit==1.6.0

rich tracebacks were introduced (#1582). Before, exceptions were shown like this: After, they are shown like this: This happens as long as

import flytekit

is called somewhere in the imported code base, also when not directly working with flyte. * * * Style ultimately is a question of taste but several of our engineers have complained about this change:

When/Why did the stacktrace get so ugly colorful. It takes up way too much space and pycharm cannot create crosslinks to the files anymore for me 🙈 And long lines are truncated

I feel the same, does not help me

Can we configure this

I wanted to complain about this too 😄

The tracebacks contain way more (white space) characters, meaning I can paste a lot less of them into chat gpt

Because of this, in this PR I propose to introduce an environment variable which disables rich tracebacks. I chose a naming inspired by

"FLYTE_SDK_LOGGING_LEVEL"

or

"FLYTE_SDK_LOGGING_FORMAT"

in

flytekit/loggers.py

. Would this change be be acceptable to you @cosmicBboy @Ketan (kumare3) @eapolinario ? Type ☐ Bug Fix ☑︎ Feature ☐ Plugin Are all requirements met? ☑︎ Code completed ☑︎ Smoke tested ☐ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description NA Tracking Issue NA Follow-up issue NA flyteorg/flytekit ✅ All checks have passed 30/30 successful checks

1 new commit pushed to

<https://github.com/flyteorg/flyteplugins/tree/master|master>

by hamersaw

<https://github.com/flyteorg/flyteplugins/commit/dfdf6f95aef7bebff160d6660f5c62f5832c39e4|dfdf6f95>

- Allow using pod start time in kubeflow plugin log links (#362) flyteorg/flyteplugins

#3782 [BUG] Cannot pass enums for execution to remote workflows fetched using `remote.fetch_workflow` Issue created by redartera Describe the bug Flytekit throws an error when attempting to execute a remote workflow that takes an enum as an input Error for a workflow that takes

input_enum

as an enum:

╭───────────────────────────────────────── Traceback (most recent call last) ─────────────────────────────────────────╮
│ /Users/reda/Projects/unionai_bugs/enum_remote/replicate.py:52 in <module>                                           │
│                                                                                                                     │
│ ❱ 52 workflow_execution = remote.execute(                                                                           │
│                                                                                                                     │
│ /Users/reda/miniconda3/envs/union_ai/lib/python3.10/site-packages/flytekit/remote/remote.py:1138 in execute         │
│                                                                                                                     │
│ ❱ 1138 │   │   │   return self.execute_remote_task_lp(                                                              │
│                                                                                                                     │
│ /Users/reda/miniconda3/envs/union_ai/lib/python3.10/site-packages/flytekit/remote/remote.py:1227 in                 │
│ execute_remote_task_lp                                                                                              │
│                                                                                                                     │
│ ❱ 1227 │   │   return self._execute(                                                                                │
│                                                                                                                     │
│ /Users/reda/miniconda3/envs/union_ai/lib/python3.10/site-packages/flytekit/remote/remote.py:1002 in _execute        │
│                                                                                                                     │
│ ❱ 1002 │   │   │   │   │   hint = type_hints[k]                                                                     │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
KeyError: 'input_enum'

This is similar to #2278 though the error message is different Expected behavior Workflow should execute Additional context to reproduce Here is a script to reproduce the bug against the sandbox

import typing
from enum import Enum

import flytekit
from flytekit.remote import FlyteRemote
from flytekit.configuration import (
    Config,
    ImageConfig,
    SerializationSettings,
    FastSerializationSettings
)


class MyEnum(Enum):
    FOO = "FOO"
    BAR = "BAR"

@flytekit.task
def get_enum_value(input_enum: MyEnum) -> str:
    return input_enum.value


@flytekit.workflow
def wf(input_enum: MyEnum) -> MyEnum:
    return get_enum_value(input_enum=input_enum)

_PROJECT = "flytesnacks"
_DOMAIN = "development"
_VERSION = "abc"

remote = FlyteRemote(config=Config.for_sandbox())

img = ImageConfig.auto_default_image()
fast_serialization_settings = FastSerializationSettings(enabled=True, destination_dir="/root")
serialization_settings = SerializationSettings(
    image_config=img,
    project=_PROJECT,
    domain=_DOMAIN,
    version=_VERSION,
    fast_serialization_settings=fast_serialization_settings
)

remote.register_workflow(wf, serialization_settings=serialization_settings)

remote_workflow = remote.fetch_workflow(
    project=_PROJECT,
    domain=_DOMAIN,
    version=_VERSION,
    name="wf"
)

workflow_execution = remote.execute(
    remote_workflow,
    inputs={"input_enum": MyEnum.FOO},
    project=_PROJECT,
    domain=_DOMAIN,
    wait=True
)

Screenshots No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

Release - v1.1.2 New release published by flyte-bot Changelog • dfdf6f9 Allow using pod start time in kubeflow plugin log links (#362) flyteorg/flyteplugins

#3783 [BUG] Error with `torch.nn.Module` and `flytekit` version &gt;1.5.0. Issue created by constanceferragu Describe the bug Flyte throws a warning and error with

torch.nn.Module

and

flytekit

version >1.5.0. Flyte uses

PickleFile

with type

nn.Module

as default when importing tasks/workflows. When registering, it does not recognize the type as

nn.Module

but as

PickleFile

- throws an error because the types aren't matching. Example of the error:

# a.py
from flytekit import task

@task
def foo() -> int:
    return 2

------------------------------------------------------------------------------

# b.py
from flytekit import task
from torch import nn

@task
def bla() -> nn.Module:
    return nn.Module()

Warning message:

WARNING  {"asctime": "2023-06-15 17:40:51,991", "name": "flytekit", "levelname": "WARNING", "message": "Unsupported Type <class 'torch.nn.modules.module.Module'> found,      
                                    Flyte will default to use PickleFile as the transport. Pickle can only be used to send objects between the exact same version of Python, and we strongly recommend                   
                                    to use python type that flyte support."}

Error message:

Error 0: Code: MismatchingTypes, Node Id: end-node, Description: Variable [model] (type [blob:<format:"PyTorchModule" > ]) doesn't match expected type [blob:<format:"PythonPickle" > metadata:<fields:<key:"python_class_name" value:<string_value:"Module" > > > ].
Error 1: Code: MismatchingTypes, Node Id: n1, Description: Variable [model] (type [blob:<format:"PythonPickle" > metadata:<fields:<key:"python_class_name" value:<string_value:"Module" > > > ]) doesn't match expected type [blob:<format:"PyTorchModule" > ].

Expected behavior Should be able to register. The culprit is most likely `if is_imported("torch")`: in

flytekit.core.type_engine

Additional context to reproduce No response Screenshots No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#3784 [Core feature] Support tasks with a`NamedTuple` output to be wrapped in `map_task` Issue created by redartera Motivation: Why do you think this is important? Map tasks are a convenient way to branch a task out in parallel in a node-efficient way compared to dynamic tasks. However,

map_task

is restricted to single-input single-output tasks. There is interest in multi-input map tasks, and using a @DataClass object is a way to get around it (as long as inputs are JSON-serializable). For the outputs, a task producing a

NamedTuple

cannot be used with map_task, and we have to use

@dynamic

instead. This issue suggests to support tasks outputting a`NamedTuple` to be wrapped within

map_task

. Goal: What should the final outcome look like, ideally? Allow tasks outputting a

NamedTuple

to be used within

map_task

. Describe alternatives you've considered Using

@dynamic

Propose: Link/Inline OR Additional context No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte