#502 Add opentelemetry instrumentation Pull request opened by hamersaw on <!date^1669243554^{date_short}|2022-11-23T22:45:54Z> TL;DR Adding opentelemetry instrumentation for k8s client operations, admin eventing, datacatalog operations, etc Type ☐ Bug Fix ☑︎ Feature ☐ Plugin Are all requirements met? ☐ Code completed ☐ Smoke tested ☐ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description ^^^ Tracking Issue flyteorg/flyte#3304 Follow-up issue NA flyteorg/flytepropeller ✅ All checks have passed 2/2 successful checks

#3099 Add opentelemetry instrumentation Pull request opened by hamersaw on <!date^1669243335^{date_short}|2022-11-23T22:42:15Z> Tracking issue #3304 Describe your changes Adding open-telemetry support for single binary to track orchestration level time-series metrics. Check all the applicable boxes ☐ I updated the documentation accordingly. ☐ All new and existing tests passed. ☐ All commits are signed-off. Screenshots NA Note to reviewers NA flyteorg/flyte ✅ All checks have passed 2/2 successful checks

#1542 R v1.2 backport to 1.4 Pull request opened by wild-endeavor Pull in changes in master from 7a77508 to d4bf234 dropping 45c0110 Also cherry-picking in the first commit from https://github.com/flyteorg/flytekit/pull/1537/commits and resolving merge conflict. The earlier PR was closed due to branch protection rules on branches named

release-*

. flyteorg/flytekit GitHub Actions: build (windows-latest, 3.9) GitHub Actions: build (windows-latest, 3.8) GitHub Actions: build (windows-latest, 3.7) GitHub Actions: build (ubuntu-latest, 3.10) GitHub Actions: build (ubuntu-latest, 3.9) GitHub Actions: build (ubuntu-latest, 3.8) GitHub Actions: build (ubuntu-latest, 3.7) DCO: DCO ✅ 1 other check has passed 1/9 successful checks

Release - v1.3.10 New release published by flyte-bot Changelog • 6ad4072 Use TokenCache in ClientCredentialsTokenSourceProvider (#377) flyteorg/flyteidl

1 new commit pushed to

<https://github.com/flyteorg/flytesnacks/tree/master|master>

by cosmicBboy

<https://github.com/flyteorg/flytesnacks/commit/f8d538931f0baf3da453d3fa9c7ec22e33a7c937|f8d53893>

- docs: add empty file to shell_task.py flytedirectory (#964) flyteorg/flytesnacks

1 new commit pushed to

<https://github.com/flyteorg/flytectl/tree/master|master>

by pmahindrakar-oss

<https://github.com/flyteorg/flytectl/commit/6f376a3cf7745468a24bb1f3397fa5d759f3df9a|6f376a3c>

- Doc updates (#398) flyteorg/flytectl

1 new commit pushed to

<https://github.com/flyteorg/flytekit/tree/master|master>

by eapolinario

<https://github.com/flyteorg/flytekit/commit/be24c52f3fb8f7949db172f490cca95fa0f0e413|be24c52f>

- Pin fsspec to <=2023.1 (#1537) flyteorg/flytekit

Release - v1.3.3 New release published by eapolinario What's Changed • Pin fsspec by @wild-endeavor in #1540 Full Changelog: v1.3.2...v1.3.3 flyteorg/flytekit

Release - v1.4.6 New release published by flyte-bot 1.4.6 (2023-03-07) Bug Fixes • update_npmversion (#713) (0c4e559) flyteorg/flyteconsole

Release - flytesnacks v0.3.177 New release published by flyte-bot Changelog • f8d5389 docs: add empty file to shell_task.py flytedirectory (#964) flyteorg/flytesnacks

Release - v1.4.1 New release published by eapolinario What's Changed • Pin fsspec to <=2023.1 by @wild-endeavor in #1537 Full Changelog: v1.4.0...v1.4.1 flyteorg/flytekit

#3421 Closing invalid bug report Issue created by timle2 Received help on Flyte Slack, and this was my error, not a bug in the code. Closing this. flyteorg/flyte

Release - v1.2.10 New release published by eapolinario What's Changed • Backport 1.4 onto v1.2 release branch by @wild-endeavor in #1542 Full Changelog: v1.2.9...v1.2.10 flyteorg/flytekit

#3422 Enable workflow dispatch for single-binary image builds Pull request opened by jeevb flyteorg/flyte GitHub Actions: generate_helm GitHub Actions: docs GitHub Actions: test-bootstrap GitHub Actions: compile GitHub Actions: generate_kustomize ✅ 2 other checks have passed 2/7 successful checks

#3423 [BUG] ARM64 Docker Images contains x86_64 binaries (helm) Issue created by qbx2 Describe the bug I am not able to run the arm64 docker images on aws graviton instances because arm64 docker images are built in incorrect way. To check:

$ docker run -it --platform linux/arm64 -u root <http://cr.flyte.org/flyteorg/flyteadmin-release:latest|cr.flyte.org/flyteorg/flyteadmin-release:latest> sh

/ # apk add file

/ # file bin/flyteadmin
bin/flyteadmin: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, Go BuildID=pwwxVfCtJjTsZm9XA0ED/BJdCEuBm0QjhMujOZ7E7/a_vHl8mBZ4xrHPFYqAdf/on6kzeFZfsYdRjhPjzhK, stripped

Other images also have this issue, for example:

$ docker run -it --platform linux/arm64 -u root <http://cr.flyte.org/flyteorg/datacatalog:latest|cr.flyte.org/flyteorg/datacatalog:latest> sh

/ # apk add file

/ # file bin/datacatalog
bin/datacatalog: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, Go BuildID=YBzDJD8nguzh9dRW4GPp/iKNJqrMQzbqKL7lVR_L0/_wLIwVTI2EO3sXJO2Sup/5m-2xGfqxh4rTZlXehX4, not stripped

I did not check every images, but I'm pretty sure that most images have the issue. Expected behavior ARM64 Docker Images should not contain x86_64 binaries. They should contain arm64 binaries instead. Graviton instances should be able to run flyte using docker. Additional context to reproduce No response Screenshots No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#3424 [BUG] FlyteFile cannot be used with `Annotated[FlyteFile, HashMethod(...)]` Issue created by AdrianoKF Describe the bug Original discussion on Slack: https://flyte-org.slack.com/archives/CP2HDHKE1/p1678183814829309 Using

Annotated[FlyteFile, HashMethod(...)]

for calculating a custom hash for a

FlyteFile

doesn't work out-of-the-box, failing an `issubclass` check in flytekit/types/file/file.py. This makes it impossible to use custom hash functions with

FlyteFile

, e.g., to implement content-aware caching. Note, that the same approach with

Annotated[FlyteDirectory, HashMethod(...)]

works as expected out of the box. Example workflow code

import hashlib
from typing import Annotated

import pandas as pd
from flytekit import task, workflow
from flytekit.core.hash import HashMethod
from flytekit.types.file import FlyteFile


def calc_hash(f: FlyteFile) -> str:
    """Calculate SHA1 hash of a file"""

    h = hashlib.sha1(usedforsecurity=False)
    with open(f.path, "rb") as f:
        while chunk := f.read(4096):
            h.update(chunk)
    return str(h.hexdigest())


CachedFlyteFile = Annotated[FlyteFile, HashMethod(calc_hash)]


@task
def write_file() -> CachedFlyteFile:
    print("write_file")
    local_path = "data.parquet"

    df = pd.DataFrame(data={"a": [1, 2, 3], "b": [3, 4, 5]})
    df.to_parquet(local_path)

    return FlyteFile(local_path, remote_path=f"<s3://test-repo/main/{local_path}>")


@task(cache=True, cache_version="1")
def print_file(file: FlyteFile) -> None:
    file.download()
    print(pd.read_parquet(file))


@workflow
def wf() -> None:
    f = write_file()
    print_file(file=f)


if __name__ == "__main__":
    wf()
    wf()  # don't expect output from `print_file`, since it should be cached

Expected behavior The first execution of

wf

should run both the

write_file

and

print_file

tasks, the second execution should only run

write_file

and hit the cache for

print_file

. In reality, an exception is raised in `flytekit/types/file/file.py, in to_literal()`:

TypeError: issubclass() arg 1 must be a class

Additional context to reproduce See example workflow above. I was able to solve the issue by inserting the following before the

issubclass

check in

flytekit/types/file/file.py

(I haven't run the flytekit test suite, so this might not be suitable for a mergeable PR just yet - should I submit a draft PR regardless?):

@@ -284,6 +288,10 @@
                 "None value cannot be converted to a file."
             )
 
+        # Handle Annotated[FlyteFile, ...] correctly by extracting the wrapped type
+        if issubclass(typing.get_origin(python_type), typing.Annotated):
+            python_type = typing.get_args(python_type)[0]
+
         if not (python_type is os.PathLike or issubclass(python_type, FlyteFile)):
             raise ValueError(
                 f"Incorrect type {python_type}, must be either a FlyteFile or os.PathLike"

Screenshots No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

1 new commit pushed to

<https://github.com/flyteorg/flytectl/tree/master|master>

by pingsutw

<https://github.com/flyteorg/flytectl/commit/e55c64995f4ebce251130a730945abc3f3430d07|e55c6499>

- Fix docs for updating description using workflow-meta subcommand (#394) flyteorg/flytectl

#345 Update documentation Pull request opened by flyte-bot Update documentation • Auto-generated by [flyte-bot] flyteorg/flytectl ✅ All checks have passed 2/2 successful checks

Release - v0.6.33 New release published by flyte-bot Changelog • 6f376a3 Doc updates (#398) • e55c649 Fix docs for updating description using workflow-meta subcommand (#394) flyteorg/flytectl

1 new commit pushed to

<https://github.com/flyteorg/homebrew-tap/tree/main|main>

by flyte-bot

<https://github.com/flyteorg/homebrew-tap/commit/bcf44fbdf1f23b1db4f79711e8a6232710a107a3|bcf44fbd>

- Brew formula update for flytectl version v0.6.33 flyteorg/homebrew-tap

#3426 Enable FlyteDeck button for recovered tasks Issue created by hamersaw This PR enables task FlyteDecks to be recovered from previous workflow executions. However, the FlyteDeck button on the UI is disabled even though the information is successfully recovered. I suspect this is because the FlyteDeck button is disabled when the task is not in the

SUCCEEDED

phase, but on recovery this will be in the

RECOVERED

phase. flyteorg/flyte

#609 [UI] Refactor NodeExecutionRow to fix discrepancies between table and timeline tabs Issue created by olga-union flyteorg/flyteconsole

1 new commit pushed to

<https://github.com/flyteorg/flytesnacks/tree/master|master>

by cosmicBboy

<https://github.com/flyteorg/flytesnacks/commit/641195c097bfb6ee6a7778d4f5dc9d4643d903e5|641195c0>

- DuckDB integration docs (#966) flyteorg/flytesnacks

Release - flytesnacks v0.3.178 New release published by flyte-bot Changelog • 641195c DuckDB integration docs (#966) flyteorg/flytesnacks

1 new commit pushed to

<https://github.com/flyteorg/flyte/tree/master|master>

by jeevb

<https://github.com/flyteorg/flyte/commit/6b04701b3fe8f1b39d63952a99863c11d865c83b|6b04701b>

- Enable workflow dispatch for single-binary image builds (#3422) flyteorg/flyte

#3427 [BUG] Flyteadmin should validate entity versions at registration time to ensure they're url safe Issue created by katrogan Describe the bug Say if a version contains a "/" in it, then it's impossible later fetch the workflow because grpc gateway interprets the "/" as part of the http path and doesn't know where to route the request, resulting in a 404. In general, flyteadmin should validate that entity versions are url safe Expected behavior Permitted entity versions should be validated at registration time so that fetching the entities later on doesn't break the console & remote experience. Additional context to reproduce Register a workflow with a "/" in it. Screenshots No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#3428 [BUG] validate version id at registration time to be url-safe Issue created by cosmicBboy Describe the bug The purpose of this issue is to add validation of the version id (e.g. the

--version

flag) that makes sure that it's URL-safe. We want to validate that the version id consists of only: • alphanumeric characters •

.

,

-

,

_

,

%

Otherwise, version ids with, for example

/

, won't be rendered correctly on the Flyte UI. Expected behavior If a user tries to do something like

pyflyte register ... --version my/version/1

• flytekit should raise an error. •

flytectl register

should also raise an error _Edit_: the validation should occur in admin, so that we don't do repeat validation across clients. Additional context to reproduce No response Screenshots No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#714 fix: fix sed makefile error Pull request opened by ursucarina TL;DR Fixes sed error Type ☑︎ Bug Fix ☐ Feature ☐ Plugin Are all requirements met? ☑︎ Code completed ☐ Smoke tested ☐ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description Fixes sed error flyteorg/flyteconsole GitHub Actions: Build & Push FlyteConsole Image GitHub Actions: Get Release Tag GitHub Actions: Generate Release GitHub Actions: Build Flyteconsole Image / Push to Github Registry GitHub Actions: extract_branch GitHub Actions: unit_tests_with_coverage GitHub Actions: lint_project ✅ 2 other checks have passed 2/9 successful checks

1 new commit pushed to

<https://github.com/flyteorg/flyteconsole/tree/master|master>

by ursucarina

<https://github.com/flyteorg/flyteconsole/commit/3bc3714ce653c634a0dddcde646f3a84f17d2d2e|3bc3714c>

- fix: fix sed makefile error (#714) flyteorg/flyteconsole

Release - v1.4.7 New release published by flyte-bot 1.4.7 (2023-03-08) Bug Fixes • fix sed makefile error (#714) (3bc3714) flyteorg/flyteconsole