#937 Improve pod example Pull request opened by samhita-alla Signed-off-by: Samhita Alla aallasamhita@gmail.com This PR improves the existing pod documentation and adds a node selector example. flyteorg/flytesnacks ✅ All checks have passed 2/2 successful checks

#3218 [BUG] Misleading/erroneous `Caching was disabled for this execution.` for tasks with cache enabled Issue created by fg91 Describe the bug When a Python task - which has the cache enabled - fails, the flyte console displays an erroneous and misleading warning that

Caching was disabled for this execution.

I have also seen this warning for *_successful_ tasks (see screenshot) even though I unfortunately cannot come up with a minimal example to consistently reproduce this. Expected behavior This warning should not be shown. Additional context to reproduce

from flytekit import task, workflow


@task(
    cache=True,
    cache_version="1.0",
)
def fail() -> None:
    raise Exception()

@workflow
def workflow():
    fail()

Screenshots Erroneous warning for a successful task with _cache enabled_: I downloaded the tarball containing the code in order to make sure that for the respective version the cache is in fact enabled: Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

1 new commit pushed to

<https://github.com/flyteorg/flyteconsole/tree/master|master>

by jsonporter

<https://github.com/flyteorg/flyteconsole/commit/cdafa9fcce52fd2a2ee3aa6b11b7e761ae62aa7b|cdafa9fc>

- chore(deps): bump minimatch from 3.0.4 to 3.1.2 (#659) flyteorg/flyteconsole

1 new commit pushed to

<https://github.com/flyteorg/flyteconsole/tree/master|master>

by jsonporter

<https://github.com/flyteorg/flyteconsole/commit/a9e1716f9906fd8530ce9b57dd02a0a42d9c93c1|a9e1716f>

- chore(deps): bump json5 from 1.0.1 to 1.0.2 (#658) flyteorg/flyteconsole

Pending review on flyteorg/flyte - Manage reminder [#1329] Update contributors' section (samhita-alla) a year stale · a year old · Waiting on _ @Ketan (kumare3)_ [#1408] Remove placeholders in secrets (jeevb) a year stale · a year old · Waiting on _ @Haytham Abuelfutuh_ [#2986] Automatic unassign for stale issues (Azanul) 2 months stale · 3 months old · Waiting on _ samhita-alla_ [#3213] Add conditional around the admin secrets (wild-endeavor) 3 days stale · 3 days old · Waiting on _ MorpheusXAUT, andrewwdye, hamersaw_

Pending review on flyteorg/flytekit-java - Manage reminder [#6] Update .env.local in README (kanterov) 9 months stale · 3 years old · Waiting on _ @Ketan (kumare3)_

#938 Document IgnoreOutputs exception Pull request opened by fg91 When trying to control which of the distributed workers of a PyTorch task returns its values to the subsequent tasks in the workflow (in the case that they are not the same), I couldn't find the information in the documentation and asked on Slack. In this PR I document the solution I was guided to on Slack in the mnist examples of distributed pytorch, tf, and mpi as this is where I first looked for an answer. It gives me a bit of sadness to propose to reproduce the same sentence three times, however, I cannot find one overarching documentation page for distributed training and as a user of the respective frameworks, this is where I would first look. Do you have a better proposition for where to document this? flyteorg/flytesnacks

#663 Publish multi-architecture image Pull request opened by jeevb on <!date^1673070063^{date_short}|2023-01-07T05:41:03Z> Depends on: flyteorg/flytetools#70 flyteorg/flyteconsole ✅ All checks have passed 2/2 successful checks

#939 Add databricks to index.rst Pull request opened by pingsutw Signed-off-by: Kevin Su pingsutw@gmail.com Add databricks to

index.rst

,

conf.py

, and

integration.rst

flyteorg/flytesnacks ✅ All checks have passed 3/3 successful checks

#514 Publish multi-architecture image Pull request opened by jeevb Depends on: flyteorg/flytetools#70 flyteorg/flytepropeller

#1313 Fix mypy errors Pull request opened by pingsutw TL;DR Fixed all the mypy errors Type ☑︎ Bug Fix ☐ Feature ☐ Plugin Are all requirements met? ☑︎ Code completed ☑︎ Smoke tested ☑︎ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description ^^^ Tracking Issue Fixes flyteorg/flyte#1030 Follow-up issue NA flyteorg/flytekit Codecov: 54.54% of diff hit (target 69.22%) ✅ 29 other checks have passed 29/30 successful checks

#3219 [Core feature] Default parquet-to-pandas encoder/decoder should support iterable read. Issue created by cosmicBboy Motivation: Why do you think this is important? The purpose of this issue is to support the use case where I can load `StructuredDataset`s iteratively as in:

structured_dataset.open(pd.Dataframe).iter()

Goal: What should the final outcome look like, ideally? The end user should be able to specify the partition column when they output a structured dataset:

@task
def make_df() -> StructuredDataset:
    df = pd.DataFrame.from_records([
        {
            "id": i,
            "partition": (i % 10) + 1,
            "name": "".join(
                random.choices(string.ascii_uppercase + string.digits, k=10)
            )
        }
        for i in range(1000)
    ])
    return StructuredDataset(dataframe=df, partition_cols=["partition"])  # or ["partition1", "partition2"]

And then consume it like so:

@task
def use_df(dataset: StructuredDataset) -> pd.DataFrame:
    output = []
    for dd in dataset.open(pd.DataFrame).iter():
        print(f"This is a partial dataframe")
        print(dd.head(3))
        output.append(dd)
    return pd.concat(output)

Describe alternatives you've considered The user needs to implement their own encoder/decoder for this use case. Propose: Link/Inline OR Additional context There is a working implementation of this here: https://github.com/flyteorg/flyte-demos/blob/main/flyte_demo/workflows/data_iter.py#L101 Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#940 Fix walkthrough command for pima diabetes classification example Pull request opened by kanyesthaker The current "Walkthrough" command for the Diabetes Classification example has its arguments in an incorrect order, which causes the command to fail with error

Error: No such option: --image

. This PR puts the arguments in the correct order. Signed-off-by: Kanyes Thaker kanyes.thaker@gmail.com flyteorg/flytesnacks

#3220 [Core feature] Kafka Authentication Issue created by pingsutw Motivation: Why do you think this is important? In flyteadmin, we send a cloud event to Kafka. However, we don't have any authentication for Kafka right now. we should expose auth config, use

Sarama

api to read the config, and send the request to Kafka. Goal: What should the final outcome look like, ideally? CloudEvent Config

cloudevents:
    enable: true
    kafka:
      brokers: <kafka_ip>
      version:  1.2.1
      security_ protocol: SASL_SSL
      sasl_config: <config for SASL>

Describe alternatives you've considered NA Propose: Link/Inline OR Additional context Slack Thread: • https://flyte-org.slack.com/archives/CP2HDHKE1/p1667307468058299 • https://flyte-org.slack.com/archives/CP2HDHKE1/p1667412056307629 • https://flyte-org.slack.com/archives/CP2HDHKE1/p1670577684352189 • https://flyte-org.slack.com/archives/CP2HDHKE1/p1667412056307629 Authentication Methods Overview Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#70 Publish multi-architecture images Pull request opened by jeevb on <!date^1673051136^{date_short}|2023-01-07T00:25:36Z> Addresses: flyteorg/flyte#2935 flyteorg/flytetools ✅ All checks have passed 2/2 successful checks

1 new commit pushed to

<https://github.com/flyteorg/flytetools/tree/master|master>

by wild-endeavor

<https://github.com/flyteorg/flytetools/commit/27211db21d30b8526e4dc06948bc5550efca4ec5|27211db2>

- Publish multi-architecture images (#70) flyteorg/flytetools

1 new commit pushed to

<https://github.com/flyteorg/flyteconsole/tree/master|master>

by jeevb

<https://github.com/flyteorg/flyteconsole/commit/567afbe19783949eab9a46a90668c47ede654893|567afbe1>

- Publish multi-architecture image (#663) flyteorg/flyteconsole

#664 Fix dependency of release pipeline Pull request opened by jeevb Read then delete this section - Make sure to use a concise title for the pull-request. _- Use #patch, #minor or #major in the pull-request title to bump the corresponding version. Otherwise, the patch version will be bumped. More details_ TL;DR Please replace this text with a description of what this PR accomplishes. Type ☐ Bug Fix ☐ Feature ☐ Plugin Are all requirements met? ☐ Code completed ☐ Smoke tested ☐ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description How did you fix the bug, make the feature etc. Link to any design docs etc Tracking Issue _Remove the '_fixes_' keyword if there will be multiple PRs to fix the linked issue_ fixes https://github.com/flyteorg/flyte/issues/ Follow-up issue NA OR https://github.com/flyteorg/flyte/issues/ flyteorg/flyteconsole

1 new commit pushed to

<https://github.com/flyteorg/flyteconsole/tree/master|master>

by jeevb

<https://github.com/flyteorg/flyteconsole/commit/516bbffd1e808d5b93f097ed962a26330f96b71f|516bbffd>

- fix: Dependency of release pipeline (#664) flyteorg/flyteconsole

#88 Publish multi-architecture image Pull request opened by jeevb Depends on: flyteorg/flytetools#70 flyteorg/datacatalog Codecov: 68.41% (-1.14%) compared to 8a0429f GitHub Actions: Goreleaser GitHub Actions: Build & Push Datacatalog Image GitHub Actions: Bump Version ✅ 7 other checks have passed 7/11 successful checks

#510 Publish multi-architecture image Pull request opened by jeevb Depends on: flyteorg/flytetools#70 flyteorg/flyteadmin GitHub Actions: End2End Test / End to End tests GitHub Actions: Check Go Generate / Go Generate ✅ 8 other checks have passed 8/10 successful checks

1 new commit pushed to

<https://github.com/flyteorg/datacatalog/tree/master|master>

by wild-endeavor

<https://github.com/flyteorg/datacatalog/commit/faa86dbf56cce108f2c0b91f8fa2a99f67c1586f|faa86dbf>

- Publish multi-architecture image (#88) flyteorg/datacatalog

Release - v1.0.41 New release published by flyte-bot Changelog • faa86db Publish multi-architecture image (#88) flyteorg/datacatalog

Release - v1.4.1 New release published by flyte-bot 1.4.1 (2023-01-09) Bug Fixes • Dependency of release pipeline (#664) (516bbff) flyteorg/flyteconsole

1 new commit pushed to

<https://github.com/flyteorg/flytepropeller/tree/master|master>

by wild-endeavor

<https://github.com/flyteorg/flytepropeller/commit/0b5007b1c7157213deff1f9a4c65bec86e9f1b63|0b5007b1>

- Publish multi-architecture image (#514) flyteorg/flytepropeller

1 new commit pushed to

<https://github.com/flyteorg/flyteadmin/tree/master|master>

by wild-endeavor

<https://github.com/flyteorg/flyteadmin/commit/cb1224b0554347e62ea99e4495d0057f885f4fff|cb1224b0>

- Publish multi-architecture image (#510) flyteorg/flyteadmin

#509 Debug multi-architecture image build Pull request opened by jeevb on <!date^1673075110^{date_short}|2023-01-07T07:05:10Z> flyteorg/flyteadmin DCO: DCO ✅ 2 other checks have passed 2/3 successful checks

#3221 Update Flyte components Pull request opened by flyte-bot Updated flyte deployment • Updated GCP Flyte kustomize generated manifest file • Updated EKS Flyte kustomize generated manifest file • Updated Sandbox Flyte kustomize generated manifest file • Updated TEST Flyte kustomize generated manifest file • Updated GCP Flyte helm generated manifest file • Updated EKS Flyte helm generated manifest file • Updated Sandbox Flyte helm generated manifest file • Updated TEST Flyte helm generated manifest file • Auto-generated by [flyte-bot] flyteorg/flyte

Release - v1.1.60 New release published by flyte-bot Changelog • 0b5007b Publish multi-architecture image (#514) flyteorg/flytepropeller

1 new commit pushed to

<https://github.com/flyteorg/flyte/tree/master|master>

by wild-endeavor

<https://github.com/flyteorg/flyte/commit/7b401bc20aad4b665daf466b862868c1ef27a0e9|7b401bc2>

- Update Flyte components (#3221) flyteorg/flyte