#725 fix: backfill index on execution task logs Pull request opened by ursucarina TL;DR Please replace this text with a description of what this PR accomplishes. Type ☐ Bug Fix ☐ Feature ☐ Plugin Are all requirements met? ☐ Code completed ☐ Smoke tested ☐ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description How did you fix the bug, make the feature etc. Link to any design docs etc Tracking Issue _Remove the '_fixes_' keyword if there will be multiple PRs to fix the linked issue_ fixes https://github.com/flyteorg/flyte/issues/ Follow-up issue NA OR https://github.com/flyteorg/flyte/issues/ flyteorg/flyteconsole GitHub Actions: unit_tests_with_coverage ✅ 2 other checks have passed 2/3 successful checks

1 new commit pushed to

<https://github.com/flyteorg/flytesnacks/tree/master|master>

by cosmicBboy

<https://github.com/flyteorg/flytesnacks/commit/ed2c9f6524e908e5d7ebd89b96a206712f36ac17|ed2c9f65>

- Update use_secrets.py (#970) flyteorg/flytesnacks

#3484 [Core feature] [flytekit] Device Authflow implementation Issue created by kumare3 Motivation: Why do you think this is important? When using pyflyte in headless mode, I would like to use PKCE style token exchange, but not use the browser. Refer to more on this doc - https://auth0.com/docs/get-started/authentication-and-authorization-flow/device-authorization-flow Goal: What should the final outcome look like, ideally?

pyflyte run ...

Go to <https://myexample.com/activate>
Type Code: XYZ-AVC

And user can navigate to the path in their browser and things work Describe alternatives you've considered flytectl supports this Propose: Link/Inline OR Additional context No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#1552 Device auth flow Pull request opened by kumare3 TL;DR This PR implements DeviceAuth flow for Pyflyte - which supports headless authentication Type ☐ Bug Fix ☑︎ Feature ☐ Plugin Are all requirements met? ☑︎ Code completed ☑︎ Smoke tested ☐ Unit tests added ☑︎ Code documentation added ☑︎ Any pending items have an associated Issue Tracking Issue flyteorg/flyte#3484 flyteorg/flytekit ✅ All checks have passed 2/2 successful checks

#3092 [Core feature] Support overriding node name Issue created by honnix Motivation: Why do you think this is important? When composing a workflow out of tasks, it is not always good enough to use the task name as is (Python function name) for node name, and sometimes it is better give a more meaningful node name. This is currently not supported by flytekit. Goal: What should the final outcome look like, ideally? Users can override node name using

with_overrides(name="A better Task Name")

in flytekit. Describe alternatives you've considered There is no alternative can achieve this. Propose: Link/Inline OR Additional context No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

1 new commit pushed to

<https://github.com/flyteorg/flyte/tree/master|master>

by hamersaw

<https://github.com/flyteorg/flyte/commit/e923f1a9ab67e1030d18dc3066fa9843076706ef|e923f1a9>

- starting controller runtime manager (#3471) flyteorg/flyte

Release - flytesnacks v0.3.182 New release published by flyte-bot Changelog • ed2c9f6 Update use_secrets.py (#970) flyteorg/flytesnacks

1 new commit pushed to

<https://github.com/flyteorg/flyteconsole/tree/master|master>

by ursucarina

<https://github.com/flyteorg/flyteconsole/commit/426ce41b7d2fe07e42e45ac3449f8ff4c64c47e5|426ce41b>

- fix: backfill index on execution task logs (#725) flyteorg/flyteconsole

#3019 Updated default FlytePropeller subqueue base delay to 0 Pull request opened by hamersaw Describe your changes This PR updates the subqueue default configuration for FlytePropeller. This construct is used to queue a workflow when FlytePropeller detects a changein a node status. This can happen using a k8s watch over Pods to detect state changes. Currently, the configuration defaults to a 5s base-delay, this means that everytime an item is added to the queue, FlytePropeller waits 5s to process it, which results in unnecessary delays in workflow execution. The trade-off is more aggressive workflow evaluations. We have tested this change at some scale and do not believe this is too frequent, meaning FlytePropeller is overwhelmed by the number of workflows in the queue. However, if it does prove too small, this can be increased in the configuration. Check all the applicable boxes ☑︎ I updated the documentation accordingly. ☑︎ All new and existing tests passed. ☑︎ All commits are signed-off. flyteorg/flyte ✅ All checks have passed 9/9 successful checks

#72 Update end2end.yml Pull request opened by pingsutw flyteorg/flytetools ✅ All checks have passed 2/2 successful checks

#972 Temporarily disable gate node example workflow Pull request opened by katrogan End to end tests use sandbox, which won't work with the gate node example pending backend changes, which themselves are pending working e2e tests flyteorg/flytesnacks GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/kubernetes/ray_example) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/kubernetes/pod) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/kubernetes/kftensorflow) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/kubernetes/kfpytorch) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/kubernetes/kfmpi) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/kubernetes/k8s_spark) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/kubernetes/k8s_dask) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/flytekit_plugins/whylogs_examples) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/flytekit_plugins/sql) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/flytekit_plugins/papermilltasks) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/flytekit_plugins/pandera_examples) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/flytekit_plugins/onnx_examples) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/flytekit_plugins/modin_examples) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/flytekit_plugins/mlflow_example) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/flytekit_plugins/greatexpectations) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/flytekit_plugins/duckdb_examples) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/flytekit_plugins/dolt) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/flytekit_plugins/dbt_example) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/external_services/databricks) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/aws/sagemaker_training) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/aws/sagemaker_pytorch) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/aws/batch) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/aws/athena) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/core) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/case_studies/ml_training/spark_horovod) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/case_studies/ml_training/pima_diabetes) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/case_studies/ml_training/nlp_processing) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/case_studies/ml_training/mnist_classifier) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/case_studies/ml_training/house_price_prediction) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/case_studies/feature_engineering/feast_integration) ✅ No checks have passed 0/30 successful checks

1 new commit pushed to

<https://github.com/flyteorg/flytesnacks/tree/master|master>

by katrogan

<https://github.com/flyteorg/flytesnacks/commit/f00bb6a7b8be4c1d6887589619d592ba5ce12e90|f00bb6a7>

- Temporarily disable gate node example workflow (#972) flyteorg/flytesnacks

Release - flytesnacks v0.3.183 New release published by flyte-bot Changelog • f00bb6a Temporarily disable gate node example workflow (#972) flyteorg/flytesnacks

#212 Fix dynamic workflows serialization Pull request opened by narape TL;DR Fix dynamic workflow task serialization. We broke dynamic workflows when we added strong types to workflows. Type ☑︎ Bug Fix ☐ Feature ☐ Plugin Are all requirements met? ☑︎ Code completed ☑︎ Smoke tested ☑︎ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description Dynamic workflows output a DynamicJobSpec that gets written in the propeller bucket as

futures.pb

, then propeller read this file and executes the workflow. The problem was that we changes how we register outputs with the strong types and this spec proto didn't contain the binding for the outputs, then the written file was missing this information and propeller failed and retried the task until the retry counts reach its limit, We didn't notice the breakage because we didn't contain unit tests for that (fixed) and the integration test was disabled (now enabled as well) Tracking Issue NA Follow-up issue NA flyteorg/flytekit-java ✅ All checks have passed 3/3 successful checks

1 new commit pushed to

<https://github.com/flyteorg/flytekit-java/tree/master|master>

by narape

<https://github.com/flyteorg/flytekit-java/commit/f2f6a09dee93ec41a04627ee9fdd5a935d1708e9|f2f6a09d>

- Fix dynamic workflows serialization (#212) flyteorg/flytekit-java

1 new commit pushed to

<https://github.com/flyteorg/flytekit-java/tree/master|master>

by narape

<https://github.com/flyteorg/flytekit-java/commit/6e1518c811b01d222588290b5996f0d3c8efb46d|6e1518c8>

- fix: upgrade com.google.api.grpc:proto-google-common-protos (#207) flyteorg/flytekit-java

#3485 [BUG] Flyteconsole >= 1.4.8 front page and other pages broken Issue created by sbrunk Describe the bug The layout of flyteconsole is broken in multiple places. Especially the front page isn't working at all. This happens in the

1.4.1

release, i.e. if you install current helm chart or run

flytectl demo

. Downgrading to console

1.4.8

helps as a workaround. Console front page: Workflow page: Expected behavior Flyteconsole should show projects on the front page and other pages without broken layout. Additional context to reproduce Update flyteconsole to

1.4.8

or higher Screenshots No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

1 new commit pushed to

<https://github.com/flyteorg/flytekit-java/tree/master|master>

by narape

<https://github.com/flyteorg/flytekit-java/commit/a2229ed7d5d2d63242b875d9d03f0c11d1a760b8|a2229ed7>

- Add registerLaunchPlan with action to SimpleSdkLaunchPlanRegistry (#211) flyteorg/flytekit-java

1 new commit pushed to

<https://github.com/flyteorg/flytesnacks/tree/master|master>

by samhita-alla

<https://github.com/flyteorg/flytesnacks/commit/dac8042bf289eb766ef3183ab8e4e8136141a17a|dac8042b>

- Add DD integration details (#962) flyteorg/flytesnacks

#321 feat: add support for inter-cluster communication for Ray plugin Pull request opened by aybidi TL;DR This PR addresses the concern "Supporting customized k8s client in various k8s plugins" but narrows it down to the ray plugin. In short, users can configure ray plugin to do inter-cluster communication, essentially submitting a custom resource to a specified k8s cluster instead of the one flytepropeller runs on. Type ☐ Bug Fix ☑︎ Feature ☐ Plugin Are all requirements met? ☐ Code completed ☐ Smoke tested ☐ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description How did you fix the bug, make the feature etc. Link to any design docs etc Tracking Issue flyteorg/flyte#2883 Follow-up issue NA OR https://github.com/flyteorg/flyte/issues/ flyteorg/flyteplugins Codecov: 0.00% of diff hit (target 62.90%) Codecov: 62.77% (-0.14%) compared to 5981c35 ✅ 4 other checks have passed 4/6 successful checks

1 new commit pushed to

<https://github.com/flyteorg/flytekit-java/tree/master|master>

by github-actions[bot]

<https://github.com/flyteorg/flytekit-java/commit/d8d2cf9c8e28704a6441bfae2ab31e9d65d1cbf1|d8d2cf9c>

- [maven-release-plugin] prepare release 0.4.7 flyteorg/flytekit-java

1 new commit pushed to

<https://github.com/flyteorg/flytekit-java/tree/master|master>

by github-actions[bot]

<https://github.com/flyteorg/flytekit-java/commit/f482fda3e2ddb733bfcf872d70a35c863ca3d636|f482fda3>

- [maven-release-plugin] prepare for next development iteration flyteorg/flytekit-java

Release - flytesnacks v0.3.184 New release published by flyte-bot Changelog • dac8042 Add DD integration details (#962) flyteorg/flytesnacks

1 new commit pushed to

<https://github.com/flyteorg/flytekit-java/tree/master|master>

by github-actions[bot]

<https://github.com/flyteorg/flytekit-java/commit/c4618c7f0ffbdbc33c88d9dc19cff7973bc86b1e|c4618c7f>

- [maven-release-plugin] prepare release 0.4.8 flyteorg/flytekit-java

1 new commit pushed to

<https://github.com/flyteorg/flytekit-java/tree/master|master>

by github-actions[bot]

<https://github.com/flyteorg/flytekit-java/commit/a5935d353869cc44333befeed3839b74b424d39b|a5935d35>

- [maven-release-plugin] prepare for next development iteration flyteorg/flytekit-java

#3486 [BUG] - Flytekit - Auth for locally defined scopes Issue created by franco-bocci Describe the bug Locally defined scopes are ignored and others are used which depend on the

authType

. This is the remote client config being used which is called here (and maybe in some other places) When this function is called, it receives 2 configs. The one we define (the PlatformConfig), and that RemoteStoreConfig I just linked. scopes passed from the PlatformConfig are 100% ignored, and the ones defined in that RemoteClientConfigStore.get_client_config are used, which are [offline, all] Expected behavior The scopes used for auth should be the ones defined locally. Additional context to reproduce

admin:
  endpoint: dns:///console.flyte.dev.foo.bar.com
  insecure: false
  authType: ClientSecret
  clientId: github-client
  clientSecretLocation: /etc/secrets/client_secret
  scopes: ["all"]

from flytekit.configuration import Config
from flytekit.remote import FlyteRemote


def get_latest_workflow_version(project, domain, workflow):
    cfg = Config.auto(config_file=path)
    remote = FlyteRemote(cfg)
    wf_version = remote.fetch_workflow(project, domain, workflow).id.version
    return wf_version

That will use

['offline', 'all']

as the auth scopes, and not

all

as expected. Screenshots No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

1 new commit pushed to

<https://github.com/flyteorg/flytekit-java/tree/master|master>

by github-actions[bot]

<https://github.com/flyteorg/flytekit-java/commit/5419919e81af71db6daa1e93c9d37f73d125a843|5419919e>

- [maven-release-plugin] prepare release 0.4.9 flyteorg/flytekit-java

1 new commit pushed to

<https://github.com/flyteorg/flytekit-java/tree/master|master>

by github-actions[bot]

<https://github.com/flyteorg/flytekit-java/commit/5d3cfdc8b5c93603ed19d3d4738979f0cdd0f3d1|5d3cfdc8>

- [maven-release-plugin] prepare for next development iteration flyteorg/flytekit-java

1 new commit pushed to

<https://github.com/flyteorg/flytekit-java/tree/master|master>

by github-actions[bot]

<https://github.com/flyteorg/flytekit-java/commit/4a149813d21145b70d5538c89bc4d9105e59a8cd|4a149813>

- [maven-release-plugin] prepare release 0.4.10 flyteorg/flytekit-java

1 new commit pushed to

<https://github.com/flyteorg/flytekit-java/tree/master|master>

by github-actions[bot]

<https://github.com/flyteorg/flytekit-java/commit/722a8c0f8fec6a4f9154ed87816c894561ccde9d|722a8c0f>

- [maven-release-plugin] prepare for next development iteration flyteorg/flytekit-java