1 new commit pushed to

<https://github.com/flyteorg/flytesnacks/tree/master|master>

by cosmicBboy

<https://github.com/flyteorg/flytesnacks/commit/a4930ce829faae64b02c36b0a21c233bced8ac41|a4930ce8>

- Feat: Add pod start and finish time in RFC3339 time format to logging link templating variables (#998) flyteorg/flytesnacks

#3785 [BUG] Cannot kickoff executions for workflows containing parameter of type `List[Optional[X]]` Issue created by eapolinario Describe the bug The launchform does not allow for parameters of type

List[Optional[X]]

to be specified, regardless of what type

X

is. For example:

import typing
from flytekit import task, workflow


@task
def t():
    pass


@workflow
def wf(ints: typing.List[typing.Optional[int]]):
    t()

Kicking off an execution via flyteconsole's launch form results in: Expected behavior We should be able to kickoff an execution for these workflows/tasks. Additional context to reproduce No response Screenshots No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

1 new commit pushed to

<https://github.com/flyteorg/flytekit/tree/master|master>

by pingsutw

<https://github.com/flyteorg/flytekit/commit/20427ee9cdbe068f17a3e1f67910fcd231a48c3d|20427ee9>

- Improve error message for pyflyte-fast-execute (#1686) flyteorg/flytekit

#3786 [Docs] Simplify flytesnacks examples build and structure Issue created by cosmicBboy Description The purpose of this issue is to make it easier to contribute and maintain flytesnacks examples. This issue involves a few sets of changes: Restructure and simplify flytesnacks Flatten the directory structure so that sets of examples that share dependencies and can logically be grouped together are in a project directory. Basically any directory that contains a Dockerfile and requirements.txt would be a project (Note: we would migrate these projects to use ImageSpec). An example layout would be:

flytesnacks/
    core
    aws_athena
    aws_batch
    etc...

Where each directory is essentially an isolated flyte project that defines its own dependencies Decouple documentation structure from example directory structure Currently, we use sphinx gallery for examples. This introduces an opinionated structure of how to organize the docs, such that the directory structure of

cookbook

needs to follow a certain structure for the docs to correctly render. In this issue we'll decouple these two things: the docs should define the structure via the sphinx conventions, and point to various example directories to get content. Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#999 remove katacoda tutorials Pull request opened by cosmicBboy this PR deletes a now-defunct set of tutorials that are hosted on flytesnacks. flyteorg/flytesnacks GitHub Actions: Mark github pre-release as Release GitHub Actions: Publish artifacts to github release GitHub Actions: Create Prerelease GitHub Actions: Bump Version ✅ 26 other checks have passed 26/30 successful checks

Release - flytesnacks v0.3.200 New release published by flyte-bot Changelog • a4930ce Feat: Add pod start and finish time in RFC3339 time format to logging link templating variables (#998) flyteorg/flytesnacks

#773 chore: restore subnav Pull request opened by ursucarina TL;DR Add ability for subnav on all paths Type ☐ Bug Fix ☐ Feature ☐ Plugin Are all requirements met? ☐ Code completed ☐ Smoke tested ☐ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description How did you fix the bug, make the feature etc. Link to any design docs etc Tracking Issue _Remove the '_fixes_' keyword if there will be multiple PRs to fix the linked issue_ fixes https://github.com/flyteorg/flyte/issues/ Follow-up issue NA OR https://github.com/flyteorg/flyte/issues/ flyteorg/flyteconsole ✅ All checks have passed 2/2 successful checks

#417 wip: Add client middleware to forward custom headers Pull request opened by katrogan on <!date^1686855078^{date_short}|2023-06-15T18:51:18Z> Read then delete this section - Make sure to use a concise title for the pull-request. _- Use #patch, #minor or #major in the pull-request title to bump the corresponding version. Otherwise, the patch version will be bumped. More details_ TL;DR Please replace this text with a description of what this PR accomplishes. Type ☐ Bug Fix ☐ Feature ☐ Plugin Are all requirements met? ☐ Code completed ☐ Smoke tested ☐ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description How did you fix the bug, make the feature etc. Link to any design docs etc Tracking Issue _Remove the '_fixes_' keyword if there will be multiple PRs to fix the linked issue_ fixes https://github.com/flyteorg/flyte/issues/ Follow-up issue NA OR https://github.com/flyteorg/flyte/issues/ flyteorg/flyteidl Codecov: 33.33% of diff hit (target 75.92%) Codecov: 75.47% (-0.46%) compared to e506206 WIP: Title contains &quot;wip&quot; GitHub Actions: Lint / Run Lint ✅ 9 other checks have passed 9/13 successful checks

1 new commit pushed to

<https://github.com/flyteorg/flytekit/tree/master|master>

by eapolinario

<https://github.com/flyteorg/flytekit/commit/572a33f6f426459d5d52e1b3cd3e1c412cc0a969|572a33f6>

- Allow disabling rich tracebacks via env var (#1695) flyteorg/flytekit

1 new commit pushed to

<https://github.com/flyteorg/flytekit/tree/master|master>

by pingsutw

<https://github.com/flyteorg/flytekit/commit/0f0416f77d8c3b9bafea7914c8cd04f50e35fd1a|0f0416f7>

- Fix task type version in pytorch and mpi task plugin (#1690) flyteorg/flytekit

1 new commit pushed to

<https://github.com/flyteorg/flytesnacks/tree/master|master>

by samhita-alla

<https://github.com/flyteorg/flytesnacks/commit/48c6d8d0f8058df879d6bb681e0e3e9cffa4d922|48c6d8d0>

- remove katacoda tutorials (#999) flyteorg/flytesnacks

#1000 Update BigQuery template Pull request opened by pingsutw Use

version

from input instead of hard-coding it to

1

flyteorg/flytesnacks GitHub Actions: Mark github pre-release as Release GitHub Actions: Publish artifacts to github release GitHub Actions: Create Prerelease GitHub Actions: Bump Version ✅ 26 other checks have passed 26/30 successful checks

#1001 Doc for pushing image to local registry in the sandbox Pull request opened by pingsutw blocked by tensorchord/envd#1656 This documentation explains the process of pushing the image built from the imageSpec to the local registry. flyteorg/flytesnacks GitHub Actions: Build & Push to GHCR (cookbook/integrations/flytekit_plugins/sql) GitHub Actions: Build & Push to GHCR (cookbook/integrations/flytekit_plugins/papermilltasks) GitHub Actions: Build & Push to GHCR (cookbook/integrations/flytekit_plugins/pandera_examples) GitHub Actions: Build & Push to GHCR (cookbook/integrations/flytekit_plugins/onnx_examples) GitHub Actions: Build & Push to GHCR (cookbook/integrations/flytekit_plugins/modin_examples) GitHub Actions: Build & Push to GHCR (cookbook/integrations/flytekit_plugins/mlflow_example) GitHub Actions: Build & Push to GHCR (cookbook/integrations/flytekit_plugins/greatexpectations) GitHub Actions: Build & Push to GHCR (cookbook/integrations/flytekit_plugins/duckdb_examples) GitHub Actions: Build & Push to GHCR (cookbook/integrations/flytekit_plugins/dolt) GitHub Actions: Build & Push to GHCR (cookbook/integrations/flytekit_plugins/dbt_example) GitHub Actions: Build & Push to GHCR (cookbook/integrations/external_services/databricks) GitHub Actions: Build & Push to GHCR (cookbook/integrations/aws/sagemaker_training) GitHub Actions: Build & Push to GHCR (cookbook/integrations/aws/sagemaker_pytorch) GitHub Actions: Build & Push to GHCR (cookbook/integrations/aws/batch) GitHub Actions: Build & Push to GHCR (cookbook/integrations/aws/athena) GitHub Actions: Build & Push to GHCR (cookbook/core) GitHub Actions: Build & Push to GHCR (cookbook/case_studies/ml_training/spark_horovod) GitHub Actions: Build & Push to GHCR (cookbook/case_studies/ml_training/pima_diabetes) GitHub Actions: Build & Push to GHCR (cookbook/case_studies/ml_training/nlp_processing) GitHub Actions: Build & Push to GHCR (cookbook/case_studies/ml_training/mnist_classifier) GitHub Actions: Build & Push to GHCR (cookbook/case_studies/ml_training/house_price_prediction) GitHub Actions: Build & Push to GHCR (cookbook/case_studies/feature_engineering/feast_integration) GitHub Actions: Build & Push to GHCR (cookbook/case_studies/feature_engineering/eda) GitHub Actions: Build & Push to GHCR (cookbook/case_studies/bioinformatics/blast) GitHub Actions: Docs Warnings ✅ 3 other checks have passed 3/28 successful checks

#3787 [BUG] Map (array) task cache miss Issue created by bstadlbauer Describe the bug When re-running the same map task with different inputs, the subsequent executions won't be written to cache even though Flyteconsole indicates so. Expected behavior Tasks should also be cached on subsequent runs of the same task Additional context to reproduce Workflow:

from flytekit import task, workflow, map_task


@task(
    cache=True,
    cache_version="1",
)
def my_task(a: int) -> bool:
    return True


@workflow
def my_workflow(inputs: list[int]):
    return map_task(my_task)(a=inputs)

Steps to reproduce: 1. Make sure you've restarted all flyte services (and don't have any previous cached data) 2. Run

my_workflow

with input

inputs=[0]

-> Artifact is written to cache as expected 3. Re-run

my_workflow

with input

inputs=[0]

-> Artifact is read from cache as expected 4. Run

my_workflow

with input

inputs=[1]

-> Task runs as expected. Artifact is NOT written to cache 5. Run

my_workflow

with input

inputs=[1]

-> Task is executed again as there has been no cache-hit Debugging showed that this is happening because the

workItemID

(here) of the of the

Writer

queue (which connects propeller to catalog) does not include any information about the input values (as

Key.String()

does not include the input here) This leads to subsequent runs creating the same

workItemID

, which will prevent them to being added to the writer queue (here) Screenshots No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#363 Fix map task cache misses Pull request opened by bstadlbauer TL;DR Fixes an issue with cache misses on subsequent runs of map tasks Checkout flyteorg/flyte#3787 for the full issue description. Type ☑︎ Bug Fix ☐ Feature ☐ Plugin Are all requirements met? ☐ Code completed ☐ Smoke tested ☑︎ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description At the moment, this adds the hashed inputs as suffix to the key. The hashed inputs implementation has been copied over from

generateArtifactTagName

(here) so in case we stick with this solution, we should probably refactor out into

flyteplugins

This PR also refactors the LiteralMap hashing function from propeller into flyteplugins Tracking Issue fixes flyteorg/flyte#3787 flyteorg/flyteplugins ✅ All checks have passed 7/7 successful checks

#3 add a folder for devweek latin america Pull request opened by samhita-alla flyteorg/flyte-conference-talks ✅ All checks have passed 2/2 successful checks

2 new commits pushed to

<https://github.com/flyteorg/flyte-conference-talks/tree/main|main>

by samhita-alla

<https://github.com/flyteorg/flyte-conference-talks/commit/7fcca2deeffdb0057505a6b59b7fc7d00c0f2f66|7fcca2de>

- add a folder for devweek latin america

<https://github.com/flyteorg/flyte-conference-talks/commit/dd2c31af21529fabe3e477cc2c14a4961493e20b|dd2c31af>

- Merge pull request #3 from flyteorg/add-devweek-2023 flyteorg/flyte-conference-talks

#1663 test s3 creds Pull request opened by samhita-alla on <!date^1685441055^{date_short}|2023-05-30T10:04:15Z> TL;DR Please replace this text with a description of what this PR accomplishes. Type ☐ Bug Fix ☐ Feature ☐ Plugin Are all requirements met? ☐ Code completed ☐ Smoke tested ☐ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description How did you fix the bug, make the feature etc. Link to any design docs etc Tracking Issue https://github.com/flyteorg/flyte/issues/ Follow-up issue NA OR https://github.com/flyteorg/flyte/issues/ flyteorg/flytekit ✅ All checks have passed 30/30 successful checks

1 new commit pushed to

<https://github.com/flyteorg/flytekit/tree/master|master>

by pingsutw

<https://github.com/flyteorg/flytekit/commit/db996429093d859d82eedb81ff44148738a4c7cf|db996429>

- get_transformer returns pickle transformer if type is unsupported (#1694) flyteorg/flytekit

1 new commit pushed to

<https://github.com/flyteorg/flytekit/tree/master|master>

by pingsutw

<https://github.com/flyteorg/flytekit/commit/afe5a4a72e5ec753c7ee84747ad76bab2a9ea635|afe5a4a7>

- Add Cuda to ImageSpec (#1688) flyteorg/flytekit

Release - v1.7.1b0 New release published by pingsutw What's Changed • Support configuring pip index url with the image spec by @yini7777 in #1692 • Improve error message for pyflyte-fast-execute by @pingsutw in #1686 • Feat: Allow disabling rich tracebacks via env var by @fg91 in #1695 • Fix task type version in pytorch and mpi task plugin by @yubofredwang in #1690 • get_transformer returns pickle transformer if type is unsupported by @pingsutw in #1694 • Add Cuda to ImageSpec by @pingsutw in #1688 New Contributors • @yini7777 made their first contribution in #1692 Full Changelog: v1.7.0...v1.7.1b0 flyteorg/flytekit

1 new commit pushed to

<https://github.com/flyteorg/flyte-conference-talks/tree/main|main>

by cosmicBboy

<https://github.com/flyteorg/flyte-conference-talks/commit/17599ae5f7b70e175ac7c8046928cbcc7060e464|17599ae5>

- add scipy 2023 directory flyteorg/flyte-conference-talks

1 new commit pushed to

<https://github.com/flyteorg/flytekit-python-template/tree/main|main>

by zeryx

<https://github.com/flyteorg/flytekit-python-template/commit/e70e8843ded1248de0d701184552069910ebc5b3|e70e8843>

- removed docker_build.sh from the default flytekit template flyteorg/flytekit-python-template

#1696 Airflow agent Pull request opened by kumare3 TL;DR Please replace this text with a description of what this PR accomplishes. Type ☐ Bug Fix ☐ Feature ☐ Plugin Are all requirements met? ☐ Code completed ☐ Smoke tested ☐ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description How did you fix the bug, make the feature etc. Link to any design docs etc Tracking Issue https://github.com/flyteorg/flyte/issues/ Follow-up issue NA OR https://github.com/flyteorg/flyte/issues/ flyteorg/flytekit GitHub Actions: lint GitHub Actions: build-plugins (3.11, flytekit-sqlalchemy) GitHub Actions: build-plugins (3.11, flytekit-snowflake) GitHub Actions: build-plugins (3.11, flytekit-polars) GitHub Actions: build-plugins (3.11, flytekit-papermill) GitHub Actions: build-plugins (3.11, flytekit-pandera) GitHub Actions: build-plugins (3.11, flytekit-kf-tensorflow) GitHub Actions: build-plugins (3.11, flytekit-kf-pytorch) GitHub Actions: build-plugins (3.11, flytekit-kf-mpi) GitHub Actions: build-plugins (3.11, flytekit-k8s-pod) GitHub Actions: build-plugins (3.11, flytekit-huggingface) GitHub Actions: build-plugins (3.11, flytekit-hive) GitHub Actions: build-plugins (3.11, flytekit-envd) GitHub Actions: build-plugins (3.11, flytekit-duckdb) GitHub Actions: build-plugins (3.11, flytekit-dolt) GitHub Actions: build-plugins (3.11, flytekit-deck-standard) GitHub Actions: build-plugins (3.11, flytekit-dbt) GitHub Actions: build-plugins (3.11, flytekit-data-fsspec) GitHub Actions: build-plugins (3.11, flytekit-dask) GitHub Actions: build-plugins (3.11, flytekit-bigquery) GitHub Actions: build-plugins (3.11, flytekit-aws-sagemaker) GitHub Actions: build-plugins (3.11, flytekit-aws-batch) GitHub Actions: build-plugins (3.11, flytekit-aws-athena) GitHub Actions: build-plugins (3.10, flytekit-whylogs) GitHub Actions: build-plugins (3.10, flytekit-vaex) GitHub Actions: build-plugins (3.10, flytekit-sqlalchemy) GitHub Actions: build-plugins (3.10, flytekit-spark) GitHub Actions: build-plugins (3.10, flytekit-snowflake) GitHub Actions: build-plugins (3.10, flytekit-ray) GitHub Actions: build-plugins (3.10, flytekit-polars) ✅ No checks have passed 0/30 successful checks

#3788 Add service account for agent Pull request opened by pingsutw As the title, add a service account without a role for the agent. people can add annotations, and image secrets for SA by updating the helm chart value.yaml flyteorg/flyte GitHub Actions: generate_kustomize GitHub Actions: compile GitHub Actions: generate_helm GitHub Actions: trigger-sandbox-lite-build GitHub Actions: test-bootstrap GitHub Actions: trigger-sandbox-build GitHub Actions: docs ✅ 2 other checks have passed 2/9 successful checks

1 new commit pushed to

<https://github.com/flyteorg/flytesnacks/tree/master|master>

by pingsutw

<https://github.com/flyteorg/flytesnacks/commit/c28621c2625735cbda6607a48f3bdbff305c5ad0|c28621c2>

- Update BigQuery template (#1000) flyteorg/flytesnacks

1 new commit pushed to

<https://github.com/flyteorg/flyte/tree/master|master>

by pingsutw

<https://github.com/flyteorg/flyte/commit/9d019382427ccc36e6502f13c61da42904236081|9d019382>

- Add service account for agent (#3788) flyteorg/flyte

Release - flytesnacks v0.3.201 New release published by flyte-bot Changelog • c28621c Update BigQuery template (#1000) • 48c6d8d remove katacoda tutorials (#999) flyteorg/flytesnacks

#774 feat: Add localStorage for selected Project/Domain Pull request opened by jsonporter This PR will provide a way to store certain default behaviors/values in

localStorage

. Specifically, this allows users to save a users last selected

project

and

domain

from the project select UI so that subsequent visits will default to those values and skip the need to select project/domain. • Saves users last selected project/domain in localStorage; will skip the project select UI if value exists in localStorage • Adds a new "View all projects" link in the project-select dropdown (in left-nav) • Changes project select URL to

/select-project

• User intents captured: 1. User selects from project select view 2. User select from project select drop-down (in side-nav) Type ☐ Bug Fix ☑︎ Feature ☐ Plugin Are all requirements met? ☑︎ Code completed ☑︎ Smoke tested ☐ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description Follow-up issue NA flyteorg/flyteconsole GitHub Actions: Build & Push FlyteConsole Image GitHub Actions: Get Release Tag GitHub Actions: Generate Release GitHub Actions: unit_tests_with_coverage ✅ 5 other checks have passed 5/9 successful checks

#416 Add prefix to data upload Pull request opened by wild-endeavor TL;DR This adds a field that will replace the content_md5-based string in Admin in the data upload link create request. See flyteorg/flyteadmin#577 for use Type ☐ Bug Fix ☑︎ Feature ☐ Plugin Are all requirements met? ☐ Code completed ☐ Smoke tested ☐ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description How did you fix the bug, make the feature etc. Link to any design docs etc Tracking Issue flyteorg/flyteidl ✅ All checks have passed 11/11 successful checks