#3307 Update envoy configuration to support minio console websockets Pull request opened by jeevb Fixes a bug in sandbox-bundled minio console where bucket contents fail to load. flyteorg/flyte ✅ All checks have passed 11/11 successful checks

1 new commit pushed to

<https://github.com/flyteorg/flyte/tree/master|master>

by jeevb

<https://github.com/flyteorg/flyte/commit/ed33357b2cd06c1bceae9fd9daddb3685e63d3ff|ed33357b>

- Update envoy configuration to support minio console websockets (#3307) flyteorg/flyte

#3308 [Core feature] Allow setting separate resource configs for different replica types in Kubeflow Jobs Issue created by yubofredwang Motivation: Why do you think this is important? It is a common user scenario for users to set different resources for different type of replicas in Kubeflow Jobs such as TFJob and MPIJob. For example, a user would want to run TFJob using the following resources:

PS:
    replicas: 1
    template:
        spec:
            containers:
                resources:
                    limits:
                    cpu: "1"
Worker:
     replicas: 2
     template:
         spec:
             containers:
                  resources:
                  limits:
                      <http://nvidia.com/gpu|nvidia.com/gpu>: 1
                      memory: "10G"

However, this is not currently supported in Flyte. Goal: What should the final outcome look like, ideally? Users should be able to override the resources specified in task definition by providing extra resources configs in task config in TfJob.

@task(
    task_config=TfJob(
         worker: {num=2, requests=Resources(cpu="1", mem="2Gi"), limits=Resources(cpu="1", mem="4Gi")},
         ps: {num=1, requests=Resources(cpu="1", mem="2Gi"), limits=Resources(cpu="1", mem="4Gi"),
    )
    cache=True,
    cache_version="1.0",
)
def mnist_tensorflow_job(hyperparameters: Hyperparameters) -> training_outputs:

Describe alternatives you've considered We can make the resources field in task function to accept a new type TFJobResources, and implement different handling for related backend plugins. However, this requires lots of code changes and undermine consistencies of task definitions. Propose: Link/Inline OR Additional context No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

1 new commit pushed to

<https://github.com/flyteorg/flytekit-java/tree/master|master>

by narape

<https://github.com/flyteorg/flytekit-java/commit/9a8f088d8a79b1f4cecd130232ec0e9ced67a5e4|9a8f088d>

- Java doc flytekit api (#190) flyteorg/flytekit-java

#159 Remove apply boiler plate Pull request opened by sonjaer Read then delete this section - Make sure to use a concise title for the pull-request. _- Use #patch, #minor or #major in the pull-request title to bump the corresponding version. Otherwise, the patch version will be bumped. More details_ TL;DR Please replace this text with a description of what this PR accomplishes. Type ☐ Bug Fix ☐ Feature ☐ Plugin Are all requirements met? ☐ Code completed ☐ Smoke tested ☐ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description How did you fix the bug, make the feature etc. Link to any design docs etc Tracking Issue _Remove the '_fixes_' keyword if there will be multiple PRs to fix the linked issue_ fixes https://github.com/flyteorg/flyte/issues/ Follow-up issue NA OR https://github.com/flyteorg/flyte/issues/ flyteorg/flytekit-java ✅ All checks have passed 3/3 successful checks

#192 Bump error prone to 2.18.0 Pull request opened by narape TL;DR Bump error prone to 2.18.0 Type ☐ Bug Fix ☐ Feature ☐ Plugin Are all requirements met? ☑︎ Code completed ☐ Smoke tested ☐ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description Apply the same changes of #182 minus the pom reformatting and the violation of DCO rules Tracking Issue NA Follow-up issue NA flyteorg/flytekit-java ✅ All checks have passed 3/3 successful checks

1 new commit pushed to

<https://github.com/flyteorg/flytekit-java/tree/master|master>

by andresgomezfrr

<https://github.com/flyteorg/flytekit-java/commit/afffd94c87adc8cc7a37cd517ccbac861a8fc567|afffd94c>

- Bump error prone to 2.18.0 (#192) flyteorg/flytekit-java

1 new commit pushed to

<https://github.com/flyteorg/flytekit-java/tree/master|master>

by andresgomezfrr

<https://github.com/flyteorg/flytekit-java/commit/daafea909c92a3f070d1739509ee88294a09ec3d|daafea90>

- Add docs to flytekit-java module #patch (#189) flyteorg/flytekit-java

1 new commit pushed to

<https://github.com/flyteorg/flytekit-java/tree/master|master>

by andresgomezfrr

<https://github.com/flyteorg/flytekit-java/commit/dc0a81c528cde7a83a325b51ec153c47212a61d0|dc0a81c5>

- Document flytekit-local-engine (#191) flyteorg/flytekit-java

1 new commit pushed to

<https://github.com/flyteorg/flytekit-java/tree/master|master>

by github-actions[bot]

<https://github.com/flyteorg/flytekit-java/commit/d80775b0e2a07999b0e7397ba47ef413fe1b52da|d80775b0>

- [maven-release-plugin] prepare release 0.4.1 flyteorg/flytekit-java

1 new commit pushed to

<https://github.com/flyteorg/flytekit-java/tree/master|master>

by github-actions[bot]

<https://github.com/flyteorg/flytekit-java/commit/8759c42ad4149e368dafe19f4eeba2b000bb378e|8759c42a>

- [maven-release-plugin] prepare for next development iteration flyteorg/flytekit-java

#951 Add retry to interruptible docs Pull request opened by bstadlbauer This PR adds a small note about requiring

retries

when using

interruptible

nodes. It also updates the

isort

pre-commit version from

5.9.3

to

5.12.0

to avoid the issue similar to PyCQA/isort#2077 flyteorg/flytesnacks GitHub Actions: Mark github pre-release as Release GitHub Actions: Publish artifacts to github release GitHub Actions: Create Prerelease GitHub Actions: Bump Version ✅ 26 other checks have passed 26/30 successful checks

Release - 0.4.1 New release published by andresgomezfrr What's Changed • Add remote example by @sonjaer in #178 • Add remote example for launch plan by @sonjaer in #179 • Remove duplication of workflow inputs and outputs #minor by @sonjaer in #180 • Add description annotation for scala case classes by @narape in #183 • Add JavaDoc comments to flytekit-api module by @pablocasares in #186 • Add scaladocs #patch by @andresgomezfrr in #185 • Add jflyte docs by @andresgomezfrr in #187 • Flytekit testing javadocs by @andresgomezfrr in #188 • Java doc flytekit api by @pablocasares in #190 • Bump error prone to 2.18.0 by @narape in #192 • Add docs to flytekit-java module #patch by @narape in #189 • Document flytekit-local-engine by @lupasarin in #191 New Contributors • @lupasarin made their first contribution in #191 Full Changelog: 0.4.0...0.4.1 flyteorg/flytekit-java

#952 Remove deploying workflows page Pull request opened by samhita-alla Signed-off-by: Samhita Alla aallasamhita@gmail.com Removing this page because the content has been consolidated and organized on the page https://docs.flyte.org/projects/cookbook/en/latest/getting_started/package_register.html. flyteorg/flytesnacks GitHub Actions: Mark github pre-release as Release GitHub Actions: Publish artifacts to github release GitHub Actions: Create Prerelease GitHub Actions: Bump Version ✅ 26 other checks have passed 26/30 successful checks

1 new commit pushed to

<https://github.com/flyteorg/flytekit/tree/master|master>

by eapolinario

<https://github.com/flyteorg/flytekit/commit/d006df676315743c794aca4cdbaec837a53174d7|d006df67>

- Auto Backfill workflow (#1420) flyteorg/flytekit

#953 Added a non-bourne shell callout Pull request opened by zeryx Motivation The onboarding Flyte document almost entirely provides all of the requirements needed to run the quickstart; however when using a non-bourne shell like fish or xzsh, the pyflyte binary is installed in a location not already in PATH. This is typically something a sysadmin can fix, however for a intermediate level linux/osx user this might not be immediately apparent. This happened to me during onboarding and took some time to realize that the binary was actually installed. Fix Added a callout to help understand what the problem is, and how to address it with Fish. Definitely not the prettiest callout but please feel free to leave suggestions on how to make it prettier and more ergonomic for users. flyteorg/flytesnacks ✅ All checks have passed 3/3 successful checks

#3310 Fix pip-compile errors when building (doc-)requirements.txt Pull request opened by ByronHsu Tracking issue I am unable to generate

doc-requirements.txt

due to the following errors. Using backtracking resolver can solve this issue. Also, it seems that legacy resolver will be deprecated and replaced by backtracking resolver after

7.0.0

, which is the recommended way.

$ python --version
Python 3.9.5

$ make doc-requirements.txt
...
WARNING: the legacy dependency resolver is deprecated and will be removed in future versions of pip-tools. The default resolver will be changed to 'backtracking' in pip-tools 7.0.0. Specify --resolver=backtracking to silence this warning.
...
Using legacy resolver. Consider using backtracking resolver with `--resolver=backtracking`.
Could not find a version that matches docutils<0.18,>=0.14,~=0.18.0 (from sphinx==4.5.0->-r <http://doc-requirements.in|doc-requirements.in> (line 2))
Tried: 0.3, 0.3.5, 0.3.7, 0.3.9, 0.4, 0.5, 0.6, 0.7, 0.8, 0.8.1, 0.9, 0.9.1, 0.10, 0.11, 0.12, 0.12, 0.13.1, 0.13.1, 0.14, 0.14, 0.15, 0.15, 0.15.1.post1, 0.15.2, 0.15.2, 0.16, 0.16, 0.17, 0.17, 0.17.1, 0.17.1, 0.18, 0.18, 0.18.1, 0.18.1, 0.19, 0.19
Skipped pre-versions: 0.14rc1, 0.14rc2, 0.16b0.dev0, 0.16b0.dev0, 0.16rc1, 0.16rc1, 0.17b1, 0.17b1, 0.17.1b1.dev0, 0.17.1b1.dev0, 0.18b1, 0.18b1, 0.18.1b0, 0.18.1b0, 0.19b1, 0.19b1
There are incompatible versions in the resolved dependencies:
  docutils (from sphinx-panels==0.6.0->-r <http://doc-requirements.in|doc-requirements.in> (line 10))
  docutils<0.18,>=0.14 (from sphinx==4.5.0->-r <http://doc-requirements.in|doc-requirements.in> (line 2))
  docutils~=0.18.0 (from sphinx-tabs==3.4.1->-r <http://doc-requirements.in|doc-requirements.in> (line 14))
make: *** [doc-requirements.txt] Error 2

Describe your changes Add

--resolver=backtracking

to

pip-compile

Check all the applicable boxes ☑︎ I updated the documentation accordingly. ☑︎ All new and existing tests passed. ☑︎ All commits are signed-off. Tests

$ make doc-requirements.txt
...
sphinxcontrib-yt==0.2.2
    # via -r <http://doc-requirements.in|doc-requirements.in>
sphinxext-remoteliteralinclude==0.4.0
    # via -r <http://doc-requirements.in|doc-requirements.in>
typing-extensions==4.4.0
    # via astroid
unidecode==1.3.6
    # via sphinx-autoapi
urllib3==1.26.14
    # via requests
virtualenv==20.17.1
    # via pre-commit
wrapt==1.14.1
    # via astroid
zipp==3.12.0
    # via importlib-metadata

# The following packages are considered to be unsafe in a requirements file:
# setuptools

Note to reviewers I am using this os. Not sure if other platforms get the same behavior. Please help test on others! flyteorg/flyte ✅ All checks have passed 6/6 successful checks

#678 chore(deps): bump http-cache-semantics from 4.1.0 to 4.1.1 Pull request opened by dependabot[bot] Bumps http-cache-semantics from 4.1.0 to 4.1.1. Commits • `2449650` Update mocha • `560b2d8` Don't use regex to trim whitespace • `b1bdb92` Remove linting package zoo • `c20dc7e` Cache 308 • See full diff in compare view Dependabot compatibility score Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting

@dependabot rebase

. * * * Dependabot commands and options You can trigger Dependabot actions by commenting on this PR: •

@dependabot rebase

will rebase this PR •

@dependabot recreate

will recreate this PR, overwriting any edits that have been made to it •

@dependabot merge

will merge this PR after your CI passes on it •

@dependabot squash and merge

will squash and merge this PR after your CI passes on it •

@dependabot cancel merge

will cancel a previously requested merge and block automerging •

@dependabot reopen

will reopen this PR if it is closed •

@dependabot close

will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually •

@dependabot ignore this major version

will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) •

@dependabot ignore this minor version

will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) •

@dependabot ignore this dependency

will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) •

@dependabot use these labels

will set the current labels as the default for future PRs for this repo and language •

@dependabot use these reviewers

will set the current reviewers as the default for future PRs for this repo and language •

@dependabot use these assignees

will set the current assignees as the default for future PRs for this repo and language •

@dependabot use this milestone

will set the current milestone as the default for future PRs for this repo and language You can disable automated security fix PRs for this repo from the Security Alerts page. flyteorg/flyteconsole GitHub Actions: Build & Push FlyteConsole Image GitHub Actions: Get Release Tag GitHub Actions: Generate Release GitHub Actions: Build Flyteconsole Image / Push to Github Registry ✅ 7 other checks have passed 7/11 successful checks

1 new commit pushed to

<https://github.com/flyteorg/flyte/tree/master|master>

by EngHabu

<https://github.com/flyteorg/flyte/commit/09f584d4a46f087c2168fe3d1963e78ae45a595a|09f584d4>

- fix pip-compile errors (#3310) flyteorg/flyte

#1432 Fix primitive decoder when evaluating Promise Pull request opened by samhita-alla Signed-off-by: Samhita Alla aallasamhita@gmail.com TL;DR This PR rewrites the

get_primitive_val

logic to fix condition validation when the Promise includes an integer

0

, a boolean

False

or a float

0.0

. Slack conversation:

Hey all, I believe I found a small bug. LMK if there is a better channel for this. During local execution, conditionals interpret a int variable with value 0 as None.

@workflow
def _impl(
    delta: int,
) -> Set[str]:
    return (
        conditional("terminal_case")
        .if_((delta != -1) & (delta <= 10))
        .then(end())
        .else_()
        .then(
            iterate()
        )
    )


def cluster_distributed() -> Set[str]:
    _impl(delta=0)

Outputs an error comparing NoneType <= 10

Type ☑︎ Bug Fix ☐ Feature ☐ Plugin Are all requirements met? ☐ Code completed ☐ Smoke tested ☑︎ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description When the value is validated against

None

, the function returns the correct value. Tracking Issue https://github.com/flyteorg/flyte/issues/ Follow-up issue NA OR https://github.com/flyteorg/flyte/issues/ flyteorg/flytekit ✅ All checks have passed 30/30 successful checks

#315 Rename dask service port from `comm` to `tcp-comm` #patch Pull request opened by bstadlbauer TL;DR This PR renames the port of the

dask

plugin's scheduler service from

comm

to

tcp-comm

as this helps with routing within istio (docs). In addition, the

dask-kubernetes

controller defaults to this in certain places (e.g. here) Type ☑︎ Bug Fix ☐ Feature ☐ Plugin Are all requirements met? ☑︎ Code completed ☐ Smoke tested ☑︎ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue flyteorg/flyteplugins ✅ All checks have passed 6/6 successful checks

1 new commit pushed to

<https://github.com/flyteorg/flytekit/tree/master|master>

by cosmicBboy

<https://github.com/flyteorg/flytekit/commit/a5c99702661c056651907065c776c1d0eb109995|a5c99702>

- Fix primitive decoder when evaluating Promise (#1432) flyteorg/flytekit

#3312 [Core feature] Support Python 3.11 Issue created by cosmicBboy Motivation: Why do you think this is important? Currently flytekit does not support python 3.11 for many reasons, the mains ones being that: 1. upstream dependencies in the flytekit dev environment do not support 3.11 currently, such as

tensorflow

and others don't support 3.11 2. certain parts of the the flytekit codebase (such as the parts that use dataclasses) need to be updated to support 3.11 Python 3.7 will be EOL'd on June 2023, which will ease the transition to 3.11 because we can unpin certain restrictions. Goal: What should the final outcome look like, ideally? A user should be able to install and use flytekit on python 3.11 Describe alternatives you've considered NA Propose: Link/Inline OR Additional context No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#3313 [BUG] For flytekit, set max version of python to 3.10 Issue created by cosmicBboy Describe the bug Users who install flytekit on python 3.11 will come up against this error #3181 when they try to use it. This is a stop-gap measure to avoid confusion, and will be addressed more systematically by supporting python 3.11 #3312 Expected behavior When a user tries to install flytekit on 3.11 pip should error out saying that flytekit isn't supported. Additional context to reproduce No response Screenshots No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#3314 [Docs] Define pre-requisites for local development Issue created by davidmirror-ops Description Currently, for local development, Flyte docs don't clearly display: 1. System requirements (CPU, memory, storage, etc) 2. Supported Python versions The absence of this information causes users to go through errors (some of them not very informative) during deployment or workflow execution even when just following the Getting Started guide. Feel free to comment or upvote this issue. Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

1 new commit pushed to

<https://github.com/flyteorg/flyteplugins/tree/master|master>

by hamersaw

<https://github.com/flyteorg/flyteplugins/commit/c4abd4f2921dd55bd119f08f0acbb8eea7a5ef4a|c4abd4f2>

- Adding support for per-task PodTemplate configuration (#308) flyteorg/flyteplugins

Release - v1.0.33 New release published by flyte-bot Changelog • c4abd4f Adding support for per-task PodTemplate configuration (#308) flyteorg/flyteplugins

Pending review on flyteorg/flytestdlib - Manage reminder [#117] Update boilerplate version (flyte-bot) 7 months stale · a year old · Waiting on _ @Haytham Abuelfutuh, katrogan, @Yee, @Ketan (kumare3)_

Pending review on flyteorg/flytekit-java - Manage reminder [#6] Update .env.local in README (kanterov) 10 months stale · 3 years old · Waiting on _ @Ketan (kumare3)_

Pending review on flyteorg/flyteplugins - Manage reminder [#300] sending correct external resources for k8s-array plugin (hamersaw) 2 months stale · 2 months old · Waiting on _ @Haytham Abuelfutuh, @Ketan (kumare3), pingsutw_ [#310] Transition to failed state if the task cannot be created (pingsutw) 7 days stale · 17 days old · Waiting on _ hamersaw_ [#314] Only create copilot sidecars if interface requires (hamersaw) an hour stale · 6 days old · Waiting on _ eapolinario, @Niels Bantilan_