1 new commit pushed to

<https://github.com/flyteorg/flytepropeller/tree/master|master>

by jeevb

<https://github.com/flyteorg/flytepropeller/commit/b9d1f621f2de0ba20d672561251f4900c26a1847|b9d1f621>

- fix makefile to read variables from environment and overrides (#554) flyteorg/flytepropeller

1 new commit pushed to

<https://github.com/flyteorg/datacatalog/tree/master|master>

by jeevb

<https://github.com/flyteorg/datacatalog/commit/0638a9e764be7ba11792985d7d09bed4ef5da375|0638a9e7>

- fix makefile to read variables from environment and overrides (#104) flyteorg/datacatalog

1 new commit pushed to

<https://github.com/flyteorg/flyteadmin/tree/master|master>

by jeevb

<https://github.com/flyteorg/flyteadmin/commit/97a6aa4e8c91a570ae480a2060402d54508e1ae1|97a6aa4e>

- Infer GOOS and GOARCH from environment (#550) flyteorg/flyteadmin

1 new commit pushed to

<https://github.com/flyteorg/flyte/tree/master|master>

by wild-endeavor

<https://github.com/flyteorg/flyte/commit/0bd4ab919b2ac49dbad888d0fe4195d02af5bf24|0bd4ab91>

- Fix: Stow scope for signing URLs in GKE >= 1.25.0 (#3383) flyteorg/flyte

#3383 Fix stow token scope for GKE >= 1.25.0 Pull request opened by fellhorn Describe your changes The previous default scope for stow in GCP

<https://www.googleapis.com/auth/devstorage.read_write>

seems to be no longer enough in GKE versions >=1.25.0. We could not find any release notes from Google pointing in this direction but noticed that Flyte would no longer work after we updated the GKE cluster version to 1.25.5. Fast registration failed with this error response from `flyteadmin`:

..
  File "/opt/pyenv-root/versions/3.9.12/lib/python3.9/site-packages/flytekit/remote/remote.py", line 580, in fast_package
    return self._upload_file(pathlib.Path(zip_file))
  File "/opt/pyenv-root/versions/3.9.12/lib/python3.9/site-packages/flytekit/remote/remote.py", line 598, in _upload_file
    upload_location = self.client.get_upload_signed_url(
  File "/opt/pyenv-root/versions/3.9.12/lib/python3.9/site-packages/flytekit/clients/friendly.py", line 998, in get_upload_signed_url
    return super(SynchronousFlyteClient, self).create_upload_location(
  File "/opt/pyenv-root/versions/3.9.12/lib/python3.9/site-packages/flytekit/clients/raw.py", line 41, in handler
    return fn(*args, **kwargs)
  File "/opt/pyenv-root/versions/3.9.12/lib/python3.9/site-packages/flytekit/clients/raw.py", line 856, in create_upload_location
    return self._dataproxy_stub.CreateUploadLocation(create_upload_location_request, metadata=self._metadata)
  File "/opt/pyenv-root/versions/3.9.12/lib/python3.9/site-packages/grpc/_channel.py", line 946, in __call__
    return _end_unary_response_blocking(state, call, False, None)
  File "/opt/pyenv-root/versions/3.9.12/lib/python3.9/site-packages/grpc/_channel.py", line 849, in _end_unary_response_blocking
    raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
        status = StatusCode.INTERNAL
        details = "failed to create a signed url. Error: unable to sign bytes: googleapi: Error 403: Request had insufficient authentication scopes.
Details:
[
  {
    "@type": "<http://type.googleapis.com/google.rpc.ErrorInfo|type.googleapis.com/google.rpc.ErrorInfo>",
    "domain": "<http://googleapis.com|googleapis.com>",
    "metadata": {
      "method": "google.iam.credentials.v1.IAMCredentials.SignBlob",
      "service": "<http://iamcredentials.googleapis.com|iamcredentials.googleapis.com>"
    },
    "reason": "ACCESS_TOKEN_SCOPE_INSUFFICIENT"
  }
]"
        debug_error_string = "UNKNOWN:Error received from peer ipv4:{removed} {created_time:"2023-02-20T11:01:31.243469439+00:00", grpc_status:13, grpc_message:"failed to create a signed url. Error: unable to sign bytes: googleapi: Error 403: Request had insufficient authentication scopes.\nDetails:\n[\n  {\n    \"@type\": \"<http://type.googleapis.com/google.rpc.ErrorInfo\|type.googleapis.com/google.rpc.ErrorInfo\>",\n    \"domain\": \"<http://googleapis.com|googleapis.com>\",\n    \"metadata\": {\n      \"method\": \"google.iam.credentials.v1.IAMCredentials.SignBlob\",\n      \"service\": \"<http://iamcredentials.googleapis.com|iamcredentials.googleapis.com>\"\n    },\n    \"reason\": \"ACCESS_TOKEN_SCOPE_INSUFFICIENT\"\n  }\n]"}"

Check all the applicable boxes ☐ I updated the documentation accordingly. (no documentation needs to be updated?) ☑︎ All new and existing tests passed. (I ran the helm and kustomize test targets) ☑︎ All commits are signed-off. Note to reviewers We started a discussion in Slack on this issue. @Yee . As far as we know there is no more restrictive scope available. We checked this Google page and it seems like

google.iam.credentials.v1.IAMCredentials.SignBlob

is only available in the

cloud-platform

scope. We also tried

<https://www.googleapis.com/auth/devstorage.full_control>

and it was not enough. flyteorg/flyte GitHub Actions: trigger-sandbox-bundled-build GitHub Actions: trigger-single-binary-build ✅ 10 other checks have passed 10/12 successful checks

Release - v1.0.46 New release published by flyte-bot Changelog • 0638a9e fix makefile to read variables from environment and overrides (#104) flyteorg/datacatalog

1 new commit pushed to

<https://github.com/flyteorg/flytecopilot/tree/master|master>

by jeevb

<https://github.com/flyteorg/flytecopilot/commit/a913fde9516d649085ba7b977fbc46e3790b0ea8|a913fde9>

- Enable multiarch build and infer GOOS and GOARCH from environment (#57) flyteorg/flytecopilot

Release - v0.0.30 New release published by flyte-bot Changelog • a913fde Enable multiarch build and infer GOOS and GOARCH from environment (#57) flyteorg/flytecopilot

Release - v1.1.89 New release published by flyte-bot Changelog • 97a6aa4 Infer GOOS and GOARCH from environment (#550) flyteorg/flyteadmin

Release - v1.1.80 New release published by flyte-bot Changelog • b9d1f62 fix makefile to read variables from environment and overrides (#554) flyteorg/flytepropeller

1 new commit pushed to

<https://github.com/flyteorg/flyteconsole/tree/master|master>

by ursucarina

<https://github.com/flyteorg/flyteconsole/commit/530c10dacce9b1e38c01487da48f03488c02a618|530c10da>

- fix: passthrough runtime env vars (#741) flyteorg/flyteconsole

#3588 [BUG] No Flyte Entities Detected with example Jupyter Notebook workflow registration Issue created by peterklingelhofer Describe the bug Using the example Jupyter Notebook file found here: https://docs.flyte.org/projects/cookbook/en/latest/auto/integrations/kubernetes/ray_example/ray_example.html When I try to register the ipynb file's workflow, I get this output:

pyflyte register workflows --project repo --image localhost:30000/$FLYTE_DOCKER_IMAGE_TAG
Running pyflyte register from ~/Dev/repo with images ImageConfig(default_image=Image(name='default', fqn='localhost:30000/repo', tag='imageTag'), images=[Image(name='default', fqn='localhost:30000/repo', tag='imageTag')]) and image destination folder /root on 1 package(s) ('~/Dev/repo/workflows',)
Registering against localhost:30080
Detected Root ~/Dev/repo, using this to create deployable package...
No output path provided, using a temporary directory at /var/folders/t4/7xnpxvjj3w3g7cf752g1qvjh0000gn/T/tmp7dmep_68 instead
Computed version is kChHd1ibpkx7jpAR1MIWIg==
Loading packages ['workflows'] under source root ~/Dev/repo
Successfully serialized 0 flyte objects
No Flyte entities were detected. Aborting!

Expected behavior I would expect this to work the way the python example (downloadable from that same page in the docs) works:

pyflyte register workflows --project repo --image localhost:30000/$FLYTE_DOCKER_IMAGE_TAG
Running pyflyte register from ~/Dev/repo with images ImageConfig(default_image=Image(name='default', fqn='localhost:30000/repo', tag='imageTag'), images=[Image(name='default', fqn='localhost:30000/repo', tag='imageTag')]) and image destination folder /root on 1 package(s) ('~/Dev/repo/workflows',)
Registering against localhost:30080
Detected Root ~/Dev/repo, using this to create deployable package...
No output path provided, using a temporary directory at /var/folders/t4/7xnpxvjj3w3g7cf752g1qvjh0000gn/T/tmpq6mm2c8o instead
Computed version is 69UqeNHQfi0GyndHMSA6uw==
Loading packages ['workflows'] under source root /Users/peterklingelhofer/Dev/am-etl-poc
Successfully serialized 3 flyte objects
[✔] Registration workflows.ray_example.ray_task type TASK successful with version 69UqeNHQfi0GyndHMSA6uw==
[✔] Registration workflows.ray_example.ray_workflow type WORKFLOW successful with version 69UqeNHQfi0GyndHMSA6uw==
[✔] Registration workflows.ray_example.ray_workflow type LAUNCH_PLAN successful with version 69UqeNHQfi0GyndHMSA6uw==
Successfully registered 3 entities

Additional context to reproduce No response Screenshots No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

1 new commit pushed to

<https://github.com/flyteorg/flyte/tree/master|master>

by cosmicBboy

<https://github.com/flyteorg/flyte/commit/c66d37ca9fce42ffb4a0c1550583665873862b0a|c66d37ca>

- fixes copy button issue (#3584) flyteorg/flyte

Release - v1.6.1 New release published by flyte-bot 1.6.1 (2023-04-12) Bug Fixes • passthrough runtime env vars (#741) (530c10d) flyteorg/flyteconsole

1 new commit pushed to

<https://github.com/flyteorg/flytekit-python-template/tree/main|main>

by zeryx

<https://github.com/flyteorg/flytekit-python-template/commit/49612b39468aad48f084f8e25fd5549ec483ce1f|49612b39>

- added github hash as secondary docker push for each image (#28) flyteorg/flytekit-python-template

#3589 Test Monorepo Pull request opened by pingsutw Test CI/CD monorepo: https://github.com/pingsutw/flyte-monorepo codecov: https://app.codecov.io/gh/pingsutw/flyte-monorepo all the files I changed: pingsutw/flyte-monorepo@744dc11 Stage 1: • push: • CI will build the images for all the components (Flyteadmin, flytepropeller, etc) • Build and push the images (flyteorg/flyteadmin:<git_sha>) • Run lint, unit, integration, end to end tests for all the components • Update Owner file. Move the owner files to root directory. https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/about-code-owners#codeowners-file-location • Remove all the boilerplate directory in the sub directory • release: • Push tag (v0.0.1, flyteadmin/v.0.0.1, flytepropeller/v.0.0.1, etc) • Push new image (flyteorg/flyteadmin:0.0.1) Stage 2: • Merge

go.mod

, so we will only have one

go.mod

in root directory • Single image for all the component. • Single binary to start different component. For example,

flyte start admin

,

flyte start propeller

. • Move single binary to

/flyte-single-binary

folder. • Update generate_flyte_manifest.yml flyteorg/flyte GitHub Actions: build-and-push-single-binary-image GitHub Actions: unit-tests (flytestdlib) GitHub Actions: unit-tests (flyteplugins) GitHub Actions: unit-tests (flytecopilot) GitHub Actions: unit-tests (datacatalog) GitHub Actions: unit-tests (flyteadmin) GitHub Actions: unit-tests (flytepropeller) GitHub Actions: Push to Github Registry GitHub Actions: Push to Github Registry GitHub Actions: lint (flytestdlib) GitHub Actions: lint (flyteplugins) GitHub Actions: unit-tests (flytestdlib) GitHub Actions: lint (flytecopilot) GitHub Actions: unit-tests (flyteplugins) GitHub Actions: lint (datacatalog) GitHub Actions: unit-tests (flytecopilot) GitHub Actions: lint (flyteadmin) GitHub Actions: lint (flytepropeller) GitHub Actions: unit-tests (flyteadmin) GitHub Actions: integration-tests GitHub Actions: unit-tests (flytepropeller) GitHub Actions: trigger-sandbox-lite-build GitHub Actions: go-generate (flytestdlib) GitHub Actions: go-generate (flyteplugins) GitHub Actions: go-generate (flytecopilot) GitHub Actions: go-generate (datacatalog) GitHub Actions: go-generate (flyteadmin) ✅ 3 other checks have passed 3/30 successful checks

1 new commit pushed to

<https://github.com/flyteorg/flytekit-python-template/tree/main|main>

by zeryx

<https://github.com/flyteorg/flytekit-python-template/commit/7129df62290bc13375ba29552391133d67d39ad4|7129df62>

- missed the -latest flag flyteorg/flytekit-python-template

#29 [WIP] Basic CI testing Pull request opened by zeryx This PR adds basic CI testing for branches to-be-merged into master. This will build images and attempt to run the flytekit projects on a flyte enabled cluster. If the workflows within each project completes successfully, it will be deemed that the template has passed CI. flyteorg/flytekit-python-template WIP: Title contains &quot;WIP&quot; ✅ 1 other check has passed 1/2 successful checks

#3590 [BUG] Wrong primary container name when using pod template breaks PytorchJob Issue created by fg91 Describe the bug When creating a task with such a pod template passed to

@task

...

PodTemplate(
    pod_spec = V1PodSpec(
        containers=[],
        tolerations=[
            V1Toleration(<some toleration>)
        ],
        node_selector=<some nodeselector>
    )
)

... such a Pod is created:

apiVersion: v1
kind: Pod
...
spec:
  containers:
  - image: ...
    name: primary
  nodeSelector:
    <some nodeselector>
  tolerations:
  - <some toleration>

The nodeselector and toleration is applied correctly but the primary container name is

primary

instead of the typical task id. For a normal task running in a single Pod, the container name doesn't really matter. However, this leads to problems e.g. with PytorchJobs for which the primary container name is expected to be

pytorch

. Here, a typical task id-based container name would be replaced with

pytorch

, the string

primary

is not. Expected behavior Here, the container name in

target.K8SPod.PodSpec

should not be

primary

. If this were the case, it would correctly be replaced with

pytorch

here. Additional context to reproduce No response Screenshots No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#3592 [UI Feature] Add full-list log output to execution detail panel Issue created by jsonporter Motivation: Why do you think this is important? Currently, while an execution is running we offer the ability to see the latest log message however the latest message often doesn't offer much useful information to debug a task. To fix this we will change this component to accept a list of all received messages that a user can scroll through to help them debug. Acceptance Criteria: • All messages are displayed in message window • User can scroll through • Component auto-scrolls to latest message when new data is received. • Remove white fade-out on bottom of text area See these for PR describing new request: flyteorg/flyteidl#382 #3501 Goal: What should the final outcome look like, ideally? Note: icon is only available during

RUNNING

Propose: Link/Inline OR Additional context https://www.figma.com/file/D96LNTCFgt5TDfEhePdTMU/Flyte-Console?node-id=5989-175110&amp;t=lXxa8kSU04htpRNc-0 Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

1 new commit pushed to

<https://github.com/flyteorg/flytectl/tree/master|master>

by pingsutw

<https://github.com/flyteorg/flytectl/commit/bd0070609ebd1f14f1498412c39a3d1f14125cac|bd007060>

- Update workflow_execution_config (#401) flyteorg/flytectl

Release - v0.6.35 New release published by flyte-bot Changelog • bd00706 Update workflow_execution_config (#401) flyteorg/flytectl

1 new commit pushed to

<https://github.com/flyteorg/homebrew-tap/tree/main|main>

by flyte-bot

<https://github.com/flyteorg/homebrew-tap/commit/267728127888aa16b1bee501533baca7ca720965|26772812>

- Brew formula update for flytectl version v0.6.35 flyteorg/homebrew-tap

#551 Enrich TerminateExecution error to tell propeller the execution already terminated Pull request opened by EngHabu TL;DR

TerminateExecution

error when a workflow is already in terminal state is

PermissionDenied

. This changes it to return

PreconditionFailed

w/ specific grpc status message. Type ☑︎ Bug Fix ☐ Feature ☐ Plugin Are all requirements met? ☑︎ Code completed ☑︎ Smoke tested ☑︎ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Tracking Issue fixes flyteorg/flyte#3582 flyteorg/flyteadmin GitHub Actions: Build & Push Flytescheduler Image GitHub Actions: Build & Push Flyteadmin Image GitHub Actions: Goreleaser GitHub Actions: Bump Version Codecov: 58.85% (-1.24%) compared to a89b359 ✅ 9 other checks have passed 9/14 successful checks

#553 Check for TerminateExecution error and eat Precondition status Pull request opened by EngHabu TL;DR Fix error check for TerminateExecution to eat errors with Precondition on AlreadyInTerminal. Type ☑︎ Bug Fix ☐ Feature ☐ Plugin Are all requirements met? ☑︎ Code completed ☐ Smoke tested ☐ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Tracking Issue fixes flyteorg/flyte#3582 flyteorg/flytepropeller GitHub Actions: Build & Push Flytepropeller Image GitHub Actions: Goreleaser GitHub Actions: Bump Version ✅ 11 other checks have passed 11/14 successful checks

#1581 Measuring the time taken for each component when executing a task Pull request opened by Yicheng-Lu-llll TL;DR The concept involves measuring the time taken by each component when executing a task, including both Flytekit and user-defined parts. Users have the option to view a timeline graph by setting the

disable_deck parameter=False

. For example:

import flytekit
from flytekit import task, Resources, workflow
from flytekit.core.utils import MeasureExecutionTime

@task(
disable_deck=False,
limits=Resources(mem="4Gi",cpu="1"),
)
def t1():
    import time

    # Test MeasureExecutionTime used as a decorator
    @MeasureExecutionTime("download data from s3")
    def download_data():
        time.sleep(1)

    download_data()

    # Test MeasureExecutionTime used as a context manager
    # Simulate a user using a very long name for a time measurement
    with MeasureExecutionTime("test long string"*10):
        time.sleep(1)

    # Simulate multiple time measurements
    for i in range(10):
        with MeasureExecutionTime(f"small tasks {i}"):
            # The 'print' function will execute quickly compared to 'time.sleep(1)'.
            # This is used to test if the timeline graph can accurately represent
            # measurements with significantly varying execution times.
            print("hello world")
         
   
    
@workflow
def wf():
    t1()

if __name__ == "__main__":
    wf()

Test in sandbox: Note: • Setting the disable_deck parameter to True will only affect whether the user can see the timeline graph. The data will still be collected and stored in the background. However, for input and output decks, if the disable_deck parameter is set to True, these decks will not be created at all. Type ☐ Bug Fix ☐ Feature ☐ Plugin Are all requirements met? ☐ Code completed ☐ Smoke tested ☐ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description How did you fix the bug, make the feature etc. Link to any design docs etc flyteorg/flytekit ✅ All checks have passed 2/2 successful checks

#395 DO NOT MERGE - testing verification tests Pull request opened by hamersaw on <!date^1681408407^{date_short}|2023-04-13T17:53:27Z> Read then delete this section - Make sure to use a concise title for the pull-request. _- Use #patch, #minor or #major in the pull-request title to bump the corresponding version. Otherwise, the patch version will be bumped. More details_ TL;DR Please replace this text with a description of what this PR accomplishes. Type ☐ Bug Fix ☐ Feature ☐ Plugin Are all requirements met? ☐ Code completed ☐ Smoke tested ☐ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description How did you fix the bug, make the feature etc. Link to any design docs etc Tracking Issue _Remove the '_fixes_' keyword if there will be multiple PRs to fix the linked issue_ fixes https://github.com/flyteorg/flyte/issues/ Follow-up issue NA OR https://github.com/flyteorg/flyte/issues/ flyteorg/flyteidl ✅ All checks have passed 13/13 successful checks

#1589 Add classifier for Python 3.11 Pull request opened by honnix TL;DR Add classifier for Python 3.11 Type ☑︎ Bug Fix ☐ Feature ☐ Plugin Are all requirements met? ☑︎ Code completed ☐ Smoke tested ☐ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description It looks that flytekit now supports Python 3.11, so let's add the classifier Tracking Issue Closes flyteorg/flyte#3594 Follow-up issue NA flyteorg/flytekit ✅ All checks have passed 30/30 successful checks

#3594 [BUG] flytekit is missing classifier for Python 3.11 Issue created by honnix Describe the bug It looks that flytekit now supports Python 3.11, but it is missing the classifier in setup.py Expected behavior The classifier is added Additional context to reproduce No response Screenshots No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

1 new commit pushed to

<https://github.com/flyteorg/flytepropeller/tree/master|master>

by hamersaw

<https://github.com/flyteorg/flytepropeller/commit/0c982eaa17301bddd7c27ef0e1c6165a0435431c|0c982eaa>

- Remove BarrierTick (#545) flyteorg/flytepropeller