#196 Do not delegate class loading to parent for some classes Pull request opened by honnix TL;DR Only load certain classes from child. Type ☑︎ Bug Fix ☐ Feature ☐ Plugin Are all requirements met? ☑︎ Code completed ☑︎ Smoke tested ☑︎ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description Unfortunately #145 did not fully fix flyteorg/flyte#3147 because it only fixed resource, not class. This PR will skip delegating to parent for class load as well. This is what slf4j <2.0.x does:

if (!isAndroid()) {
                staticLoggerBinderPathSet = findPossibleStaticLoggerBinderPathSet();
                reportMultipleBindingAmbiguity(staticLoggerBinderPathSet);
            }
            // the next line does the binding
            StaticLoggerBinder.getSingleton();

findPossibleStaticLoggerBinderPathSet

does the resource load so even we prevent delegation to parent at that point,

StaticLoggerBinder.getSingleton()

will still delegate to parent, without this PR. Tracking Issue _Remove the '_fixes_' keyword if there will be multiple PRs to fix the linked issue_ fixes flyteorg/flyte#3147 Follow-up issue NA flyteorg/flytekit-java ✅ All checks have passed 3/3 successful checks

1 new commit pushed to

<https://github.com/flyteorg/flytekit-java/tree/master|master>

by narape

<https://github.com/flyteorg/flytekit-java/commit/32af350a1950469f82082e722837a3003cf0294c|32af350a>

- Only load certain classes from child (#196) flyteorg/flytekit-java

Pending review on flyteorg/flyteplugins - Manage reminder [#310] Transition to failed state if the task cannot be created (pingsutw) 5 days stale · a month old · Waiting on _ hamersaw_

Pending review on flyteorg/flyteidl - Manage reminder [#297] Proposal : Settings proto (pmahindrakar-oss) 2 months stale · 9 months old · Waiting on _ @Haytham Abuelfutuh, katrogan_

#3356 [Docs] Find a place to document applying changes via helm chart across all namespaces and projects Issue created by ppiegaze Description The discussion begins here Need to sort through this thread and figure out where to document the essential points. Among other issues: The rbac given in values.yaml is wrong and had to be replaced with the rbac given in eks-production.yaml Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#197 [Snyk] Upgrade info.picocli:picocli from 4.7.0 to 4.7.1 Pull request opened by snyk-bot Snyk has created this PR to upgrade info.picocli:picocli from 4.7.0 to 4.7.1. ℹ️ Keep your dependencies up-to-date. This makes it easier to fix existing vulnerabilities and to more quickly identify and fix newly disclosed vulnerabilities when they affect your project. * * * • The recommended version is 1 version ahead of your current version. • The recommended version was released 23 days ago, on 2023-01-29. * * * Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open upgrade PRs. For more information: https://camo.githubusercontent.com/94ab81de2fd36c5f146e7ea1736971ab306c82a285ef01907f7322cddb7fd2f7/68747470733a2f2f6170692e7365676d656e742e696f2f76312f706978656c2f747261636b3f646174613d65794a33636d6c305a55746c65534936496e4a79576d785a634564485932527954485a7362306c596430645563566734576b4652546e4e434f5545774969776959573576626e6c746233567a535751694f694a6d4d6a4a6d4e445933596930304f5445334c5451354e6d51744f545a6b596930794d44566c596a51355a5751354e7a49694c434a6c646d567564434936496c425349485a705a58646c5a434973496e42796233426c636e52705a584d694f6e736963484a4a5a434936496d59794d6d59304e6a64694c5451354d5463744e446b325a4330354e6d52694c5449774e5756694e446c6c5a446b334d694a3966513d3d 🧐 View latest project report 🛠️ Adjust upgrade PR settings 🔕 Ignore this dependency or unsubscribe from future upgrade PRs flyteorg/flytekit-java ✅ All checks have passed 3/3 successful checks

#959 Update raw container docs Pull request opened by samhita-alla This PR updates raw container docs to clarity rawContainer tasks and relationship. Fixes flyteorg/flyte#3269. flyteorg/flytesnacks GitHub Actions: Mark github pre-release as Release GitHub Actions: Publish artifacts to github release GitHub Actions: Create Prerelease GitHub Actions: Bump Version ✅ 26 other checks have passed 26/30 successful checks

#3357 [UI Feature] UX improvement when running out of project quota Issue created by pradithya Motivation: Why do you think this is important? Flyte provides the capability to enforce CPU and Memory quota per project. We use this feature to avoid one project from consuming shared K8S cluster resources. As the size of the workflow is getting larger, the quota limit per project is getting hit quite often which delays the workflow execution. Currently, there are several UX areas that could be improved to handle the project quota. 1. When a task is scheduled but the host project is running out of quota, the timeline page still shows the task state as running. Ideally, the running state is only entered when there is a pod executing the task. 2. There should be a way to inform users that their workflow execution is affected by project quota. As of now, we can only see this information in the flyte propeller's log

Failed to launch job, resource quota exceeded. err: [BackOffError] The operation was attempted but failed, caused by: pods "fec52e914a8c0b0b7000-n3-0" is forbidden: exceeded quota: project-quota, requested: limits.memory=64Gi, used: limits.memory=192Gi, limited: limits.memory=200Gi

Goal: What should the final outcome look like, ideally? • Users should be able to identify quickly if their workflow execution is affected by project quota. • Transition to "RUNNING" state should be accurate Propose: Link/Inline OR Additional context No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#960 Add mlflow example Pull request opened by pingsutw Fixes flyteorg/flyte#3286 Add a flytesnack mlflow example • https://flyte--960.org.readthedocs.build/projects/cookbook/en/960/auto/integrations/flytekit_plugins/mlflow_example/index.html • https://flyte--960.org.readthedocs.build/projects/cookbook/en/960/auto/integrations/flytekit_plugins/mlflow_example/mlflow_example.html#sphx-glr-auto-integrations-flytekit-plugins-mlflow-example-mlflow-example-py flyteorg/flytesnacks GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/kubernetes/ray_example) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/kubernetes/pod) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/kubernetes/kftensorflow) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/kubernetes/kfpytorch) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/kubernetes/kfmpi) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/kubernetes/k8s_spark) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/kubernetes/k8s_dask) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/flytekit_plugins/whylogs_examples) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/flytekit_plugins/sql) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/flytekit_plugins/papermilltasks) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/flytekit_plugins/pandera_examples) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/flytekit_plugins/onnx_examples) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/flytekit_plugins/modin_examples) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/flytekit_plugins/mlflow_example) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/flytekit_plugins/greatexpectations) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/flytekit_plugins/duckdb_examples) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/flytekit_plugins/dolt) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/flytekit_plugins/dbt_example) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/external_services/databricks) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/aws/sagemaker_training) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/aws/sagemaker_pytorch) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/aws/batch) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/integrations/aws/athena) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/core) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/case_studies/ml_training/spark_horovod) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/case_studies/ml_training/pima_diabetes) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/case_studies/ml_training/nlp_processing) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/case_studies/ml_training/mnist_classifier) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/case_studies/ml_training/house_price_prediction) GitHub Actions: Serialize & Register Flytesnacks workflow / Serialize Example (cookbook/case_studies/feature_engineering/feast_integration) ✅ No checks have passed 0/30 successful checks

1 new commit pushed to

<https://github.com/flyteorg/flyteconsole/tree/master|master>

by jsonporter

<https://github.com/flyteorg/flyteconsole/commit/860e356718e3a6a9cfb07460baf2eccf152ec75f|860e3567>

- Add support for ApprovedCondition for GateNodes (#688) flyteorg/flyteconsole

#369 Adding support for structured dataset Pull request opened by pmahindrakar-oss TL;DR Adds support for structured data set for flytectl Followup PR in flytectl to dumps the output which were missing in details flag. Eg o/p for execution with structured dataset. It dumps the URI currently for the stored o/p.

flytectl get executions -p flytesnacks -d development f12ce5ce3aae045378c1  --details 
TABLE format is not supported on detailed view and defaults to tree view. Choose either json/yaml

└── start-node - SUCCEEDED - 2023-02-21 19:20:26.545951047 +0000 UTC - 2023-02-21 19:20:26.545951047 +0000 UTC
│   ├── Outputs :
│       └── a: 2
└── n0 - SUCCEEDED - 2023-02-21 19:20:26.777756742 +0000 UTC - 2023-02-21 19:20:41.033027942 +0000 UTC
│   ├── Attempt :0
│   │   ├── Task - SUCCEEDED - 2023-02-21 19:20:27.069960835 +0000 UTC - 2023-02-21 19:20:40.827311935 +0000 UTC
│   │   ├── Task Type - python-task
│   │   ├── Reason - [ContainersNotReady|ContainerCreating]: containers with unready status: [f12ce5ce3aae045378c1-n0-0]|
│   │   ├── Metadata
│   │   │   ├── Generated Name : f12ce5ce3aae045378c1-n0-0
│   │   │   ├── Plugin Identifier : container
│   │   │   ├── External Resources
│   │   │   ├── Resource Pool Info
│   │   ├── Logs :
│   │       └── Name :Cloudwatch Logs (User)
│   │       └── URI :<https://us-east-2.console.aws.amazon.com/cloudwatch/home?region=us-east-2#logsV2:log-groups/log-group/$252Faws$252Fcontainerinsights$252Fopta-oc-canary$252Fapplication$3FlogStreamNameFilter$3Dvar.log.containers.f12ce5ce3aae045378c1-n0-0_flytesnacks-development_f12ce5ce3aae045378c1-n0-0>
│   ├── Outputs :
│       └── o0: <s3://union-oc-canary-playground/7y/f12ce5ce3aae045378c1-n0-0/2a1d85ef4a87410f5a04ce889e876776>
└── n1 - SUCCEEDED - 2023-02-21 19:20:41.420976535 +0000 UTC - 2023-02-21 19:20:57.090747824 +0000 UTC
│   ├── Attempt :0
│   │   ├── Task - SUCCEEDED - 2023-02-21 19:20:41.720309792 +0000 UTC - 2023-02-21 19:20:56.914747315 +0000 UTC
│   │   ├── Task Type - python-task
│   │   ├── Reason - [ContainersNotReady|ContainerCreating]: containers with unready status: [f12ce5ce3aae045378c1-n1-0]|
│   │   ├── Metadata
│   │   │   ├── Generated Name : f12ce5ce3aae045378c1-n1-0
│   │   │   ├── Plugin Identifier : container
│   │   │   ├── External Resources
│   │   │   ├── Resource Pool Info
│   │   ├── Logs :
│   │       └── Name :Cloudwatch Logs (User)
│   │       └── URI :<https://us-east-2.console.aws.amazon.com/cloudwatch/home?region=us-east-2#logsV2:log-groups/log-group/$252Faws$252Fcontainerinsights$252Fopta-oc-canary$252Fapplication$3FlogStreamNameFilter$3Dvar.log.containers.f12ce5ce3aae045378c1-n1-0_flytesnacks-development_f12ce5ce3aae045378c1-n1-0>
│   ├── Outputs :
│       └── o0: <s3://union-oc-canary-playground/g9/f12ce5ce3aae045378c1-n1-0/e340ecf4db7d64e3a539665cb7648047>
└── n2 - SUCCEEDED - 2023-02-21 19:20:57.318835153 +0000 UTC - 2023-02-21 19:21:11.021544907 +0000 UTC
│   ├── Attempt :0
│   │   ├── Task - SUCCEEDED - 2023-02-21 19:20:57.56183156 +0000 UTC - 2023-02-21 19:21:10.828786539 +0000 UTC
│   │   ├── Task Type - python-task
│   │   ├── Reason - [ContainersNotReady|ContainerCreating]: containers with unready status: [f12ce5ce3aae045378c1-n2-0]|
│   │   ├── Metadata
│   │   │   ├── Generated Name : f12ce5ce3aae045378c1-n2-0
│   │   │   ├── Plugin Identifier : container
│   │   │   ├── External Resources
│   │   │   ├── Resource Pool Info
│   │   ├── Logs :
│   │       └── Name :Cloudwatch Logs (User)
│   │       └── URI :<https://us-east-2.console.aws.amazon.com/cloudwatch/home?region=us-east-2#logsV2:log-groups/log-group/$252Faws$252Fcontainerinsights$252Fopta-oc-canary$252Fapplication$3FlogStreamNameFilter$3Dvar.log.containers.f12ce5ce3aae045378c1-n2-0_flytesnacks-development_f12ce5ce3aae045378c1-n2-0>
│   ├── Outputs :
│       └── o0: <s3://union-oc-canary-playground/o4/f12ce5ce3aae045378c1-n2-0/0d1195b0dbded5310076263af1f96315>
└── end-node - SUCCEEDED - 2023-02-21 19:21:11.401510279 +0000 UTC - 2023-02-21 19:21:11.541818271 +0000 UTC

Type ☐ Bug Fix ☐ Feature ☐ Plugin Are all requirements met? ☐ Code completed ☐ Smoke tested ☐ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description How did you fix the bug, make the feature etc. Link to any design docs etc Tracking Issue fixes flyteorg/flyte#2757 Follow-up issue flyteorg/flyteidl ✅ All checks have passed 13/13 successful checks

#390 Adding support for structured dataset and also for node outputs in details flag Pull request opened by pmahindrakar-oss TL;DR Type ☐ Bug Fix ☑︎ Feature ☐ Plugin Are all requirements met? ☐ Code completed ☐ Smoke tested ☐ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description How did you fix the bug, make the feature etc. Link to any design docs etc Tracking Issue flyteorg/flyte#2757 Follow-up issue NA flyteorg/flytectl GitHub Actions: Test Getting started GitHub Actions: Dry Run Goreleaser ✅ 7 other checks have passed 7/9 successful checks

#2864 [BUG] Failed to run BQ task when cache is enabled Issue created by pingsutw Describe the bug Slack Thread Failed to run BQ task when the cache is enabled because type validation is failing.

{"json":{"exec_id":"atfkcwwv5cfr7wzhqq94","node":"n1","ns":"flytesnacks-
development","res_ver":"5501760","routine":"worker-3","tasktype":"python-
task","wf":"flytesnacks:development:<http://example_test.wf|example_test.wf>"},"level":"error","msg":"DataCatalog failed to get outputs from 
artifact 45bd1d68-a013-43b1-a56b-b7597b559125, err: unexpected artifactData: [o0] type: 
[structured_dataset_type:\u003c\u003e ] does not match any task output type: 
[structured_dataset_type:\u003cformat:\"parquet\" \u003e ]","ts":"2022-09-12T06:56:41Z"}

When the cache is enabled, we'll retrieve artifacts from datacatalog and check if the structured dataset's schema and format match the expected type. However, the default format of the structured dataset in the expected type is always

Parquet

, but the format of the output structured dataset is

""

.

@task(cache=True, cache_version="1.0")
def t2() -> StructuredDataset: # The default format of structured dataset is Parquet here
    df = pd.DataFrame({"len": [len(sd.open(pd.DataFrame).all())]})
    return StructuredDataset(df, uri=bq_uri) # The format of structured dataset is ""

Two ways to fix it. 1. Change these lines to

if len(structuredDatasetType.Format) != 0 && !strings.EqualFold(structuredDatasetType.Format, t.literalType.GetStructuredDatasetType().Format) {
		return false
	}

2. Change the default format of the expected type to "" in flytekit, and change these lines to the below. However, it will break existing users. If users upgrade flytekit, they have to upgrade the propeller as well.

if len(t.literalType.GetStructuredDatasetType().Format) != 0 && !strings.EqualFold(structuredDatasetType.Format, t.literalType.GetStructuredDatasetType().Format) {
		return false
	}

structuredDatasetType

is input type

t.literalType.GetStructuredDatasetType()

is expected type Expected behavior BQ task should run successfully even if the cache is enabled Additional context to reproduce

import uuid
import pandas as pd
from typing_extensions import Annotated
from flytekit import task, workflow, StructuredDataset, kwtypes


@task(cache=True, cache_version="2.0")
def t1() -> StructuredDataset:
    df = pd.DataFrame({
        "name": ["dylan", "steve"],
        "age": [33, 32]
    })
    return StructuredDataset(df)


@task(cache=True, cache_version="2.0")
def t2(sd: StructuredDataset) -> StructuredDataset:
    df = pd.DataFrame({"len": [len(sd.open(pd.DataFrame).all())]})
    table_id = str(uuid.uuid4())
    bq_uri = f"<bq://flyte-test-340607.dataset>.{table_id}"
    return StructuredDataset(df, uri=bq_uri)


@workflow
def wf() -> StructuredDataset:
    return t2(sd=t1())


if __name__ == "__main__":
    wf()

Screenshots No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#3358 [Core feature] Ability to Capture Pipe stderr and stdout Issue created by rkrishnasanka Motivation: Why do you think this is important? Currently, I'm running algorithms that take over one week to run on the flight cluster; one of the best ways to keep track of the current status of the solver is to check the

stdout

. So the ability to pipe the

stdout

or the

stderr

to a

host:port

would be the best way to go about it. Goal: What should the final outcome look like, ideally? Being able to use a syntax like:

def stdout_pipe_callback(data: str):
   # send data to "<https://myfavoritehost:1234>"
   ...

def stderr_pipe_callback(data: str):
   # send data to "<https://myotherfavoritehost:1234>"
   ...

@stream_task(stdout_pipe=stdout_pipe_callback, stderr_pipe=stderr_pipe_callback)
def do_some_work(input:InputType) -> OutputType:
 .....
 return results

This would allow the user, to define custom callback methods to stream back the stdout. Describe alternatives you've considered I've considered using the shell task, but that won't allow me to "stream" the outputs until the end of the process. But I need help defining python code to use. Another alternative I'm considering is to figure out a way to modify the docker container to monitor all the python stdout. I haven't had any luck with that. Propose: Link/Inline OR Additional context No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

1 new commit pushed to

<https://github.com/flyteorg/flyteidl/tree/master|master>

by EngHabu

<https://github.com/flyteorg/flyteidl/commit/e69fd54aa9a68403361926a674e272aa85060943|e69fd54a>

- Adding support for structured dataset (#369) flyteorg/flyteidl

#2757 [BUG] `--details` flag in flytectl command is erroring out Issue created by samhita-alla Describe the bug When appending the

--details

flag to flytectl command, it throws a StructuredDataset error as follows:

flytectl get execution  -p flytesnacks -d development a47w6dwnr454vzzgqhxj -o yaml  --details

Error:

Error: unsupported literal scalar type *core.Scalar_StructuredDataset

Expected behavior The command should work. Additional context to reproduce Run

flytectl get execution -p flytesnacks -d development <execution-id> -o yaml --details

Screenshots No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

Release - v1.3.8 New release published by flyte-bot Changelog • e69fd54 Adding support for structured dataset (#369) flyteorg/flyteidl

#3359 [Core feature] Override/Extension support for dataclass type Issue created by elibixby Motivation: Why do you think this is important? Dramatically simplifies allowing user defined serialization/deserialization logic using a pattern that already is supported by

dataclass_json

Goal: What should the final outcome look like, ideally? Add support for custom encoders and decoders to

dataclass

types, as described here. https://lidatong.github.io/dataclasses-json/#extending Describe alternatives you've considered Currently the ugly workaround is decoding in

__post_init__

and encoding via a

@classmethod

Propose: Link/Inline OR Additional context Should be easy to add in https://github.com/flyteorg/flytekit/blob/245c4143b4ed20487d837850df4a140c7b230986/flytekit/core/type_engine.py Simply test for encoder and decoder when serializing/deserializing. Would be good to add support for

mm_field

as well, to allow for custom serializers/deserializers that use non-string representations. Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#527 Update flyteplugins to 1.0.37 Pull request opened by bstadlbauer TL;DR Updates

flyteplugins

from

1.0.34

to

1.0.37

to pull in the change from flyteorg/flyteplugins#315. Type ☑︎ Bug Fix ☐ Feature ☐ Plugin Are all requirements met? ☐ Code completed ☐ Smoke tested ☐ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue flyteorg/flytepropeller GitHub Actions: Build & Push Flytepropeller Image GitHub Actions: Goreleaser GitHub Actions: Bump Version ✅ 11 other checks have passed 11/14 successful checks

#961 Correct docs for interruptible retry behaviour Pull request opened by fg91 The documentation suggests that an interruptible task is retried on a spot instance after one preemption and that this doesn't count against the user-defined retry budget. In contrast to this, in the OSS community sync (21st February 23) @Ketan (kumare3) explained that when

n

retries are allowed,

n-1

of them are on spot instances and for the last attempt, Flyte resorts to a regular instance. I tested the behaviour using this workflow ...

from flytekit import task, workflow, Resources

@task(interruptible=True, retries=5, requests=Resources(gpu="1"))
def train():
    import time
    time.sleep(600)

@workflow
def wf():
    train()

... and by simulating a preemption with

gcloud compute instances simulate-maintenance-event <node name> --zone <zone>

. Result: • Pods with

FLYTE_ATTEMPT_NUMBER

from 0-4 had the spot/preemptible node selector. • The pod with

FLYTE_ATTEMPT_NUMBER=5

did not have the spot/preemptible node selector anymore. flyteorg/flytesnacks GitHub Actions: Build & Push to GHCR (cookbook/integrations/flytekit_plugins/dolt) GitHub Actions: Build & Push to GHCR (cookbook/integrations/flytekit_plugins/dbt_example) GitHub Actions: Build & Push to GHCR (cookbook/integrations/external_services/databricks) GitHub Actions: Build & Push to GHCR (cookbook/integrations/aws/sagemaker_training) GitHub Actions: Build & Push to GHCR (cookbook/integrations/aws/sagemaker_pytorch) GitHub Actions: Build & Push to GHCR (cookbook/integrations/aws/batch) GitHub Actions: Build & Push to GHCR (cookbook/integrations/aws/athena) GitHub Actions: Build & Push to GHCR (cookbook/core) GitHub Actions: Build & Push to GHCR (cookbook/case_studies/ml_training/spark_horovod) GitHub Actions: Build & Push to GHCR (cookbook/case_studies/ml_training/pima_diabetes) GitHub Actions: Build & Push to GHCR (cookbook/case_studies/ml_training/nlp_processing) GitHub Actions: Build & Push to GHCR (cookbook/case_studies/ml_training/mnist_classifier) GitHub Actions: Build & Push to GHCR (cookbook/case_studies/ml_training/house_price_prediction) GitHub Actions: Build & Push to GHCR (cookbook/case_studies/feature_engineering/feast_integration) GitHub Actions: Build & Push to GHCR (cookbook/case_studies/feature_engineering/eda) GitHub Actions: Build & Push to GHCR (cookbook/case_studies/bioinformatics/blast) GitHub Actions: Docs Warnings DCO: DCO ✅ 2 other checks have passed 2/20 successful checks

1 new commit pushed to

<https://github.com/flyteorg/flytepropeller/tree/master|master>

by hamersaw

<https://github.com/flyteorg/flytepropeller/commit/752f55e9c7f7e357de707aa3eaae6e3af03b186f|752f55e9>

- Update flyteplugins to 1.0.37 (#527) flyteorg/flytepropeller

#3360 [BUG] Flyte Console Error 501 Request URL: https://flyte.example.com/me Issue created by joaoloboguerra Describe the bug Hi everyone. I need help, I deploy Flyte on cluster but in my URL I received an error

Request URL: <https://flyte.example.com/me>
Request Method: GET
Status Code: 501

Status Code: 501 Expected behavior I look for all issue about this error and I can't find it. Additional context to reproduce No response Screenshots Status Error 501 Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#3361 [Docs] Enhance Datadog integration documentation Issue created by franco-bocci Description A week ago I had to integrate Flyte Workflows with Datadog logs system and as discussed in this thread in the Slack channel, we decided to extend the documentation for this. Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

Release - v1.1.66 New release published by flyte-bot Changelog • 752f55e Update flyteplugins to 1.0.37 (#527) flyteorg/flytepropeller

#962 Add DD integration details Pull request opened by privatedumbo Relates to Issue 3361. Extending Datadog logs integration. flyteorg/flytesnacks ✅ All checks have passed 2/2 successful checks

#691 Add version bump for ApproveCondition PR Pull request opened by james-union Read then delete this section - Make sure to use a concise title for the pull-request. _- Use #patch, #minor or #major in the pull-request title to bump the corresponding version. Otherwise, the patch version will be bumped. More details_ TL;DR Please replace this text with a description of what this PR accomplishes. Type ☐ Bug Fix ☐ Feature ☐ Plugin Are all requirements met? ☐ Code completed ☐ Smoke tested ☐ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description How did you fix the bug, make the feature etc. Link to any design docs etc Tracking Issue _Remove the '_fixes_' keyword if there will be multiple PRs to fix the linked issue_ fixes https://github.com/flyteorg/flyte/issues/ Follow-up issue NA OR https://github.com/flyteorg/flyte/issues/ flyteorg/flyteconsole GitHub Actions: Build Flyteconsole Image / Push to Github Registry GitHub Actions: extract_branch GitHub Actions: lint_project GitHub Actions: unit_tests_with_coverage ✅ 2 other checks have passed 2/6 successful checks

#690 feat: add draggable file upload component Pull request opened by 4nalog TL;DR This PR implements a generic file upload component with support for dragging/dropping to upload. Demo Type ☐ Bug Fix ☑︎ Feature ☐ Plugin Are all requirements met? ☑︎ Code completed ☐ Smoke tested ☐ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description This PR: • adds react-dropzone as dependency • adds a

DraggableFileUpload

component to support uploading files with drag and drop functionality Tracking Issue #3317 Follow-up issue flyteorg/flyte#3317 flyteorg/flyteconsole GitHub Actions: Build & Push FlyteConsole Image GitHub Actions: Get Release Tag GitHub Actions: Generate Release GitHub Actions: Build Flyteconsole Image / Push to Github Registry GitHub Actions: unit_tests_with_coverage ✅ 4 other checks have passed 4/9 successful checks

1 new commit pushed to

<https://github.com/flyteorg/flytekit-java/tree/master|master>

by github-actions[bot]

<https://github.com/flyteorg/flytekit-java/commit/4183cf27159a974dfecdd5f73086fa9751022169|4183cf27>

- [maven-release-plugin] prepare release 0.4.4 flyteorg/flytekit-java

1 new commit pushed to

<https://github.com/flyteorg/flytekit-java/tree/master|master>

by github-actions[bot]

<https://github.com/flyteorg/flytekit-java/commit/b88e987f8973a11c5719dcb9b8a27502b8712b2d|b88e987f>

- [maven-release-plugin] prepare for next development iteration flyteorg/flytekit-java

#3362 [Core feature] Task time estimation / progress bar capacity Issue created by zeryx Motivation: Why do you think this is important? Workflow visibility is fantastic for seeing the status of a workflow, and when a task launched. However for managed solutions it can be hard to view the direct kubernetes logs. Specifically the problem is most pronounced with long running tasks that have multiple steps that are doing an inner map or parallel map operation. Goal: What should the final outcome look like, ideally? The user should be able to go to the flyte dashboard, and for their long running mapping task - they could see an estimate for how long it will take for that task to complete, that the task is moving forward and progressing; and how many steps have been completed already. Describe alternatives you've considered printing task level logging to the user can provide this same level of observability, if for whatever reason there were security issues with providing this to a practitioner within a larger managed environment we might not be able to use a simple stdout / stderr log tail solution. Propose: Link/Inline OR Additional context TQDM: https://github.com/tqdm/tqdm This allows for wrapping for loops and iterators and print to some logging service / solution. For tasks with a large scale mapping flow, or parallel mapping flow this is normal and extremely practical and easy to use. Compatible with every ML framework and is an easy interaction on the users side. My proposal is that we wrap tqdm within the flytekit service, something like

import Task, Workflow, Tqdm from flytekit
import torch as tf
@Task
def some_func_with_mapping(...) -> ...:
  model = tf.load(...)
  output = []
  tqdm(for row in ...):
     output.append(model.forward(row))
  return output

Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte