Release - v0.6.29 New release published by flyte-bot Changelog • 1b2221f Recognize windows arm as a valid OS+arch combination (#386) flyteorg/flytectl

1 new commit pushed to

<https://github.com/flyteorg/homebrew-tap/tree/main|main>

by flyte-bot

<https://github.com/flyteorg/homebrew-tap/commit/25bc71ddf77dda5cf87eaea086c7f517caf205b1|25bc71dd>

- Brew formula update for flytectl version v0.6.29 flyteorg/homebrew-tap

Release - v1.0.16 New release published by github-actions[bot] Changelog • 0dbe3c2 Add git branch to version package (#151) flyteorg/flytestdlib

#1504 [Snyk] Security upgrade ipython from 5.10.0 to 8.10.0 Pull request opened by EngHabu This PR was automatically created by Snyk using the credentials of a real user. Snyk has created this PR to fix one or more vulnerable packages in the

pip

dependencies of this project. Changes included in this PR • Changes to the following files to upgrade the vulnerable dependencies to a fixed version: • doc-requirements.txt ⚠️ Warning

vaex 4.16.0 requires vaex-core, which is not installed.
vaex-viz 0.5.4 requires vaex-core, which is not installed.
vaex-server 0.8.1 requires vaex-core, which is not installed.
vaex-server 0.8.1 requires fastapi, which is not installed.
vaex-server 0.8.1 requires uvicorn, which is not installed.
vaex-ml 0.18.1 requires vaex-core, which is not installed.
vaex-jupyter 0.8.1 requires vaex-core, which is not installed.
vaex-jupyter 0.8.1 requires ipyvolume, which is not installed.
vaex-jupyter 0.8.1 requires xarray, which is not installed.
vaex-hdf5 0.14.1 requires vaex-core, which is not installed.
vaex-astro 0.9.3 requires vaex-core, which is not installed.
protoc-gen-swagger 0.1.0 requires protobuf, which is not installed.
notebook 5.7.16 requires terminado, which is not installed.
nbformat 4.4.0 requires jsonschema, which is not installed.
nbconvert 5.6.1 has requirement mistune<2,>=0.8.1, but you have mistune 2.0.4.
jupyter 1.0.0 requires qtconsole, which is not installed.
jupyter 1.0.0 requires jupyter-console, which is not installed.
ipython 5.10.0 requires simplegeneric, which is not installed.
ipympl 0.9.2 has requirement matplotlib<4,>=3.4.0, but you have matplotlib 2.2.5.
ipyleaflet 0.17.0 requires xyzservices, which is not installed.
ImageHash 4.3.1 requires scipy, which is not installed.
ImageHash 4.3.1 requires PyWavelets, which is not installed.
google-auth 2.16.0 requires rsa, which is not installed.
flyteidl 1.3.5 requires googleapis-common-protos, which is not installed.
flyteidl 1.3.5 requires protobuf, which is not installed.
docker-image-py 0.1.12 requires regex, which is not installed.
databricks-cli 0.17.4 requires pyjwt, which is not installed.
databricks-cli 0.17.4 requires tabulate, which is not installed.

Vulnerabilities that will be fixed By pinning: (*) Note that the real score may have changed since the PR was raised. Some vulnerabilities couldn't be fully fixed and so Snyk will still find them when the project is tested again. This may be because the vulnerability existed within more than one direct dependency, but not all of the affected dependencies could be upgraded. Check the changes in this PR to ensure they won't cause issues with your project. * * * Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs. For more information: https://camo.githubusercontent.com/181f688b16e2fb15bfa14c7b36bf0c89eec6fb84a3f607ab54beb3ec598e4231/68747470733a2f2f6170692e7365676d656e742e696f2f76312f706978656c2f747261636b3f646174613d65794a33636d6c305a55746c65534936496e4a79576d785a634564485932527954485a7362306c596430645563566734576b4652546e4e434f5545774969776959573576626e6c746233567a535751694f694a6b5a444e6d5957566a4e69316c4e475a684c54526c596a6b74596a52694e5330774f575931597a51325a6a5932596a67694c434a6c646d567564434936496c425349485a705a58646c5a434973496e42796233426c636e52705a584d694f6e736963484a4a5a434936496d526b4d325a685a574d324c5755305a6d45744e4756694f5331694e4749314c5441355a6a566a4e445a6d4e6a5a694f434a3966513d3d 🧐 View latest project report 🛠️ Adjust project settings 📚 Read more about Snyk's upgrade and patch logic * * * Learn how to fix vulnerabilities with free interactive lessons: 🦉 Learn about vulnerability in an interactive lesson of Snyk Learn. flyteorg/flytekit GitHub Actions: build (windows-latest, 3.9) GitHub Actions: build (windows-latest, 3.8) GitHub Actions: build (windows-latest, 3.7, -spark2) GitHub Actions: build (windows-latest, 3.7) GitHub Actions: build (ubuntu-latest, 3.10) GitHub Actions: build (ubuntu-latest, 3.9) GitHub Actions: build (ubuntu-latest, 3.8) GitHub Actions: build (ubuntu-latest, 3.7, -spark2) GitHub Actions: build (ubuntu-latest, 3.7) GitHub Actions: lint GitHub Actions: docs GitHub Actions: build-plugins (3.10, flytekit-vaex) GitHub Actions: build-plugins (3.10, flytekit-sqlalchemy) GitHub Actions: build-plugins (3.10, flytekit-spark) GitHub Actions: build-plugins (3.10, flytekit-ray) GitHub Actions: build-plugins (3.10, flytekit-papermill) GitHub Actions: build-plugins (3.10, flytekit-hive) GitHub Actions: build-plugins (3.10, flytekit-dolt) GitHub Actions: build-plugins (3.10, flytekit-deck-standard) GitHub Actions: build-plugins (3.10, flytekit-dbt) GitHub Actions: build-plugins (3.10, flytekit-dask) ✅ 9 other checks have passed 9/30 successful checks

#1505 [Snyk] Security upgrade ipython from 7.34.0 to 8.10.0 Pull request opened by EngHabu This PR was automatically created by Snyk using the credentials of a real user. Snyk has created this PR to fix one or more vulnerable packages in the

pip

dependencies of this project. Changes included in this PR • Changes to the following files to upgrade the vulnerable dependencies to a fixed version: • plugins/flytekit-greatexpectations/requirements.txt ⚠️ Warning

protoc-gen-swagger 0.1.0 requires protobuf, which is not installed.
flyteidl 1.3.0 requires protobuf, which is not installed.
flyteidl 1.3.0 requires googleapis-common-protos, which is not installed.
docker-image-py 0.1.12 requires regex, which is not installed.
binaryornot 0.4.4 requires chardet, which is not installed.

Vulnerabilities that will be fixed By pinning: (*) Note that the real score may have changed since the PR was raised. Some vulnerabilities couldn't be fully fixed and so Snyk will still find them when the project is tested again. This may be because the vulnerability existed within more than one direct dependency, but not all of the affected dependencies could be upgraded. Check the changes in this PR to ensure they won't cause issues with your project. * * * Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs. For more information: https://camo.githubusercontent.com/19e0170646ad9f2e7b4c1d6145b7cea84dc2ece846eb28e60316358af50d92a2/68747470733a2f2f6170692e7365676d656e742e696f2f76312f706978656c2f747261636b3f646174613d65794a33636d6c305a55746c65534936496e4a79576d785a634564485932527954485a7362306c596430645563566734576b4652546e4e434f5545774969776959573576626e6c746233567a535751694f6949794d6d45314e7a6c684e5330305a6a59774c5451314d7a4174596a41784d4330334f474d79595746694d7a55335a544d694c434a6c646d567564434936496c425349485a705a58646c5a434973496e42796233426c636e52705a584d694f6e736963484a4a5a434936496a4979595455334f5745314c54526d4e6a41744e44557a4d4331694d4445774c546334597a4a685957497a4e54646c4d794a3966513d3d 🧐 View latest project report 🛠️ Adjust project settings 📚 Read more about Snyk's upgrade and patch logic * * * Learn how to fix vulnerabilities with free interactive lessons: 🦉 Learn about vulnerability in an interactive lesson of Snyk Learn. flyteorg/flytekit ✅ All checks have passed 30/30 successful checks

#3330 [BUG] pyflyte run results in error: Handshake failed with fatal error OPENSSL_internal:WRONG_VERSION_NUMBER while connecting to flyte-binary in k8s Issue created by abuvaneswari Describe the bug I followed the instructions and hosted flyte-binary in a Kubernetes cluster, v 1.20 (tried on 1.21 as well): https://docs.flyte.org/en/latest/deployment/deployment/cloud_simple.html I attach the values.yaml file that was used in the helm install command. values.yaml.txt I verified that I am able to access the flyte console (as I have configured NodePort. when I port-forward also, the same error occurs). Verified that the following curl command works: $ curl http://135.104.238.138:8088/api/v1/projects {"projects":[{"id":"flytesnacks","name":"flytesnacks","domains":[{"id":"development","name":"development"},{"id":"staging","name":"staging"},{"id":"production","name":"production"}],"description":"flytesnacks description"}]} Attempted pyflyte run command with this remote and got the following error: (Note that the same command runs fine when run in the 'demo' environment) $ pyflyte --config ~/.flyte/config.yaml run --remote example1.py training_workflow --hyperparameters '{"C": 0.1}' E0212 021723.386151523 3823 ssl_transport_security.cc:1501] Handshake failed with fatal error SSL_ERROR_SSL: error100000f7SSL routinesOPENSSL internalWRONG_VERSION_NUMBER. {"asctime": "2023-02-12 021723,412", "name": "flytekit.cli", "levelname": "ERROR", "message": "Non-auth RPC error <_InactiveRpcError of RPC that terminated with:\n\tstatus = StatusCode.UNAVAILABLE\n\tdetails = "failed to connect to all addresses; last error: UNKNOWN: ipv4135.104.238.1388088: Ssl handshake failed: SSL_ERROR_SSL: error100000f7SSL routinesOPENSSL internalWRONG_VERSION_NUMBER"\n\tdebug_error_string = "UNKNOWN:failed to connect to all addresses; last error: UNKNOWN: ipv4135.104.238.1388088: Ssl handshake failed: SSL_ERROR_SSL: error100000f7SSL routinesOPENSSL internalWRONG_VERSION_NUMBER {grpc_status:14, created_time:"2023-02-12T021723.412065046-05:00"}"\n>, sleeping 200ms and retrying"} {"asctime": "2023-02-12 021723,613", "name": "flytekit.cli", "levelname": "ERROR", "message": "Non-auth RPC error <_InactiveRpcError of RPC that terminated with:\n\tstatus = StatusCode.UNAVAILABLE\n\tdetails = "failed to connect to all addresses; last error: UNKNOWN: ipv4135.104.238.1388088: Ssl handshake failed: SSL_ERROR_SSL: error100000f7SSL routinesOPENSSL internalWRONG_VERSION_NUMBER"\n\tdebug_error_string = "UNKNOWN:failed to connect to all addresses; last error: UNKNOWN: ipv4135.104.238.1388088: Ssl handshake failed: SSL_ERROR_SSL: error100000f7SSL routinesOPENSSL internalWRONG_VERSION_NUMBER {grpc_status:14, created_time:"2023-02-12T021723.613242602-05:00"}"\n>, sleeping 400ms and retrying"} Traceback (most recent call last): File "/home/buvana/anaconda3/envs/flyte_qs/bin/pyflyte", line 8, in sys.exit(main()) File "/home/buvana/anaconda3/envs/flyte_qs/lib/python3.8/site-packages/click/core.py", line 1130, in call return self.main(*args, **kwargs) File "/home/buvana/anaconda3/envs/flyte_qs/lib/python3.8/site-packages/click/core.py", line 1055, in main rv = self.invoke(ctx) File "/home/buvana/anaconda3/envs/flyte_qs/lib/python3.8/site-packages/click/core.py", line 1657, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/home/buvana/anaconda3/envs/flyte_qs/lib/python3.8/site-packages/click/core.py", line 1657, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/home/buvana/anaconda3/envs/flyte_qs/lib/python3.8/site-packages/click/core.py", line 1657, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/home/buvana/anaconda3/envs/flyte_qs/lib/python3.8/site-packages/click/core.py", line 1404, in invoke return ctx.invoke(self.callback, **ctx.params) File "/home/buvana/anaconda3/envs/flyte_qs/lib/python3.8/site-packages/click/core.py", line 760, in invoke return __callback(*args, **kwargs) File "/home/buvana/anaconda3/envs/flyte_qs/lib/python3.8/site-packages/flytekit/clis/sdk_in_container/run.py", line 552, in _run remote_entity = remote.register_script( File "/home/buvana/anaconda3/envs/flyte_qs/lib/python3.8/site-packages/flytekit/remote/remote.py", line 800, in register_script upload_location, md5_bytes = fast_register_single_script( File "/home/buvana/anaconda3/envs/flyte_qs/lib/python3.8/site-packages/flytekit/tools/script_mode.py", line 111, in fast_register_single_script upload_location = create_upload_location_fn(content_md5=md5) File "/home/buvana/anaconda3/envs/flyte_qs/lib/python3.8/site-packages/flytekit/clients/friendly.py", line 998, in get_upload_signed_url return super(SynchronousFlyteClient, self).create_upload_location( File "/home/buvana/anaconda3/envs/flyte_qs/lib/python3.8/site-packages/flytekit/clients/raw.py", line 43, in handler return fn(*args, **kwargs) File "/home/buvana/anaconda3/envs/flyte_qs/lib/python3.8/site-packages/flytekit/clients/raw.py", line 871, in create_upload_location return self._dataproxy_stub.CreateUploadLocation(create_upload_location_request, metadata=self._metadata) File "/home/buvana/anaconda3/envs/flyte_qs/lib/python3.8/site-packages/grpc/_channel.py", line 946, in call return _end_unary_response_blocking(state, call, False, None) File "/home/buvana/anaconda3/envs/flyte_qs/lib/python3.8/site-packages/grpc/_channel.py", line 849, in _end_unary_response_blocking raise _InactiveRpcError(state) grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with: status = StatusCode.UNAVAILABLE details = "failed to connect to all addresses; last error: UNKNOWN: ipv4135.104.238.1388088: Ssl handshake failed: SSL_ERROR_SSL: error100000f7SSL routinesOPENSSL internalWRONG_VERSION_NUMBER" debug_error_string = "UNKNOWN:failed to connect to all addresses; last error: UNKNOWN: ipv4135.104.238.1388088: Ssl handshake failed: SSL_ERROR_SSL: error100000f7SSL routinesOPENSSL internalWRONG_VERSION_NUMBER {grpc_status:14, created_time:"2023-02-12T021724.014732644-05:00"}" Expected behavior flyte workflow should be deployed in the remote and got executed Additional context to reproduce flyte-binary running in k8s cluster 1.20 (and 1.21) openssl version command when executed within the flyte-binary pod produced 1.1.1s as the version client: flytekit==1.3.2 python 3.8 (same error occurs in 3.7 and 3.9) OS: Linux and Windows (error occurs in both) openssl version: 1.1.1s and 1.1.1n ~/.flyte/config.yaml file's contents: admin: endpoint: dns///135.104.238.1388088 authType: Pkce insecure: false logger: show-source: true level: 0 Screenshots No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#3331 [Docs] Where from I locate aks values to deploy in my cluster Azure? Issue created by joaoloboguerra Description Hey everyone. I'm new here, and I received of the my boss to install Flyte on cluster for my team ML to test on it. I can't deploy flyte to my cluster because I can't find how to configure it I need help if anyone can support me for Azure (aks) thanks Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#1506 Fix pytorch transformer Pull request opened by samhita-alla TL;DR This PR fixes type fetch for torch types. Type ☑︎ Bug Fix ☐ Feature ☐ Plugin Are all requirements met? ☐ Code completed ☐ Smoke tested ☐ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description The

guess_python_type

method has been moved to the subclasses without retaining it in the parent class. This will return the correct type when fetching the execution output via FlyteRemote. Tracking Issue flyteorg/flyte#3148 Follow-up issue NA OR https://github.com/flyteorg/flyte/issues/ flyteorg/flytekit GitHub Actions: build-plugins (3.8, flytekit-duckdb) GitHub Actions: build-plugins (3.8, flytekit-dolt) GitHub Actions: build-plugins (3.8, flytekit-deck-standard) GitHub Actions: build-plugins (3.8, flytekit-dbt) GitHub Actions: build-plugins (3.8, flytekit-data-fsspec) GitHub Actions: build-plugins (3.8, flytekit-dask) GitHub Actions: build-plugins (3.8, flytekit-bigquery) GitHub Actions: build-plugins (3.8, flytekit-aws-sagemaker) GitHub Actions: build-plugins (3.8, flytekit-aws-batch) GitHub Actions: build-plugins (3.8, flytekit-aws-athena) GitHub Actions: lint GitHub Actions: docs ✅ 2 other checks have passed 2/14 successful checks

#1507 Fix pytorch transformer Pull request opened by samhita-alla Signed-off-by: Samhita Alla aallasamhita@gmail.com TL;DR This PR fixes type fetch for torch types. Type ☑︎ Bug Fix ☐ Feature ☐ Plugin Are all requirements met? ☐ Code completed ☐ Smoke tested ☐ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description The guess_python_type method has been moved to the subclasses without retaining it in the parent class. This will return the correct type when fetching the execution output via FlyteRemote. Tracking Issue Fixes flyteorg/flyte#3148 Follow-up issue NA OR https://github.com/flyteorg/flyte/issues/ flyteorg/flytekit Codecov: 46.15% of diff hit (target 69.45%) ✅ 29 other checks have passed 29/30 successful checks

#193 Refactor SdkBindindData factory methods Pull request opened by narape TL;DR Add

SdkLiteralType

and move factory methods out of

SdkBindingData

Type ☐ Bug Fix ☑︎ Feature ☐ Plugin Are all requirements met? ☑︎ Code completed ☐ Smoke tested ☑︎ Unit tests added ☑︎ Code documentation added ☑︎ Any pending items have an associated Issue Complete description In the Sdk there is a SdkType that represent an abstraction for the type of a task or workflow and its multiple variables. However we are missing an equivalent to represent the type of a single value.

SdkBindingData.ofBindingCollection(
            LiteralType.ofCollectionType(LiteralType.ofSimpleType(SimpleType.INTEGER)),
            List.of(sum1.getOutputs().o(), sum2.getOutputs().o()));

The api module was designed to be stable a follow closely the proto classes, that is the reason of why is so verbose Instead with the proper abstraction we could write this instead:

SdkBindingDatas.ofBindingCollection(
            SdkLiteralTypes.integers(), List.of(sum1.getOutputs().o(), sum2.getOutputs().o()));

In addition, we moved the factory methods in SdkBindingData to it's own class, so Java and Scala have a more similar API Tracking Issue fixes flyteorg/flyte#3332 Follow-up issue Not an issue yet, but

SdkBindinDatas

flyteorg/flytekit-java ✅ All checks have passed 3/3 successful checks

#3332 [flytekit-java] Introduce abstraction for a value type Issue created by narape Describe the issue In the Sdk there is a

SdkType

that represent an abstraction for the type of a task or workflow and its multiple variables. However we are missing an equivalent to represent the type of a single value. This causes that we were leaking internal classes of the api module into the users code. For example, to create a SdkBindingData for a list of bindings, the user needs to:

SdkBindingData.ofBindingCollection(
            LiteralType.ofCollectionType(LiteralType.ofSimpleType(SimpleType.INTEGER)),
            List.of(sum1.getOutputs().o(), sum2.getOutputs().o()));

The api module was designed to be stable a follow closely the proto classes, that is the reason of why is so verbosed Instead with the proper abstraction we could write this instead:

SdkBindingDatas.ofBindingCollection(
            SdkLiteralTypes.integers(), List.of(sum1.getOutputs().o(), sum2.getOutputs().o()));

What if we do not do this? We would had a worse user experience because of the verbosity of the api Related component(s) flytekit-java Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

1 new commit pushed to

<https://github.com/flyteorg/flytesnacks/tree/master|master>

by samhita-alla

<https://github.com/flyteorg/flytesnacks/commit/2383f4a069000380bf7ff9c54d401c38ae90ffa6|2383f4a0>

- Improve pod example (#937) flyteorg/flytesnacks

#937 Improve pod example Pull request opened by samhita-alla Signed-off-by: Samhita Alla aallasamhita@gmail.com This PR improves the existing pod documentation and adds a node selector example. Issue: flyteorg/flyte#3196 flyteorg/flytesnacks GitHub Actions: Mark github pre-release as Release GitHub Actions: Publish artifacts to github release GitHub Actions: Create Prerelease GitHub Actions: Bump Version ✅ 26 other checks have passed 26/30 successful checks

#785 Bump pillow from 9.1.0 to 9.1.1 in /cookbook/integrations/kubernetes/pod Pull request opened by dependabot[bot] Bumps pillow from 9.1.0 to 9.1.1. Release notes Sourced from pillow's releases.

9.1.1

This release addresses several security problems.

CVE-2022-30595: When reading a TGA file with RLE packets that cross scan lines, Pillow reads the information past the end of the first line without deducting that from the length of the remaining file data. This vulnerability was introduced in Pillow 9.1.0, and can cause a heap buffer overflow.

Opening an image with a zero or negative height has been found to bypass a decompression bomb check. This will now raise a `SyntaxError` instead, in turn raising a

PIL.UnidentifiedImageError

.

Changelog Sourced from pillow's changelog.

9.1.1 (2022-05-17)

• When reading past the end of a TGA scan line, reduce bytes left. CVE-2022-30595 [radarhere]

• Do not open images with zero or negative height #6269 [radarhere]

Commits • `0f44136` 9.1.1 version bump • `f66f5e1` pre-commit: update Black to fix Click • `0153b37` Skip test_realloc_overflow unless libtiff 4.0.4 or higher • `6fcd31b` Added release notes for 9.1.1 • `c846cc8` When reading past the end of a scan line, reduce bytes left • `184b73e` Do not open images with zero or negative height • See full diff in compare view Dependabot compatibility score Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting

@dependabot rebase

. * * * Dependabot commands and options You can trigger Dependabot actions by commenting on this PR: •

@dependabot rebase

will rebase this PR •

@dependabot recreate

will recreate this PR, overwriting any edits that have been made to it •

@dependabot merge

will merge this PR after your CI passes on it •

@dependabot squash and merge

will squash and merge this PR after your CI passes on it •

@dependabot cancel merge

will cancel a previously requested merge and block automerging •

@dependabot reopen

will reopen this PR if it is closed •

@dependabot close

will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually •

@dependabot ignore this major version

will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) •

@dependabot ignore this minor version

will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) •

@dependabot ignore this dependency

will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) •

@dependabot use these labels

will set the current labels as the default for future PRs for this repo and language •

@dependabot use these reviewers

will set the current reviewers as the default for future PRs for this repo and language •

@dependabot use these assignees

will set the current assignees as the default for future PRs for this repo and language •

@dependabot use this milestone

will set the current milestone as the default for future PRs for this repo and language You can disable automated security fix PRs for this repo from the Security Alerts page. flyteorg/flytesnacks GitHub Actions: Mark github pre-release as Release GitHub Actions: Publish artifacts to github release GitHub Actions: Create Prerelease GitHub Actions: Bump Version GitHub Actions: Serialize & Register Flytesnacks workflow / Register example to sandbox ✅ 25 other checks have passed 25/30 successful checks

Release - flytesnacks v0.3.166 New release published by flyte-bot Changelog • 2383f4a Improve pod example (#937) • 2b814b0 Added a non-bourne shell callout (#953) • 8c96962 Add databricks to index.rst (#939) flyteorg/flytesnacks

1 new commit pushed to

<https://github.com/flyteorg/flyteplugins/tree/master|master>

by hamersaw

<https://github.com/flyteorg/flyteplugins/commit/5981c35dbe2033e9905283c4ea4b667e79f0bf8b|5981c35d>

- Rename dask port from to (#315) flyteorg/flyteplugins

Release - v1.0.37 New release published by flyte-bot Changelog • 5981c35 Rename dask port from to (#315) flyteorg/flyteplugins

#387 refactor: remove redundant YAML packages Pull request opened by Juneezee TL;DR Remove redundant YAML packages:

<http://github.com/ghodss/yaml|github.com/ghodss/yaml>

and

<http://gopkg.in/yaml.v2|gopkg.in/yaml.v2>

Type ☐ Bug Fix ☐ Feature ☐ Plugin Are all requirements met? ☑︎ Code completed ☐ Smoke tested ☐ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description Currently four YAML packages are used: 1.

<http://github.com/ghodss/yaml|github.com/ghodss/yaml>

2.

<http://gopkg.in/yaml.v2|gopkg.in/yaml.v2>

3.

<http://gopkg.in/yaml.v3|gopkg.in/yaml.v3>

4.

<http://sigs.k8s.io/yaml|sigs.k8s.io/yaml>

We can keep only two of them: 1.

<http://gopkg.in/yaml.v3|gopkg.in/yaml.v3>

2.

<http://sigs.k8s.io/yaml|sigs.k8s.io/yaml>

The package

<http://github.com/ghodss/yaml|github.com/ghodss/yaml>

is no longer actively maintained. See discussion in ghodss/yaml#80 and ghodss/yaml#75.

<http://sigs.k8s.io/yaml|sigs.k8s.io/yaml>

is a permanent fork of

ghodss/yaml

and is actively maintained by Kubernetes SIG, also widely used in K8s projects. We are only using

<http://gopkg.in/yaml.v2|gopkg.in/yaml.v2>

in

cmd/config/subcommand/project/project_config.go

. Switching to

<http://gopkg.in/yaml.v3|gopkg.in/yaml.v3>

doesn't introduce breaking change here. Tracking Issue Follow-up issue NA flyteorg/flytectl GitHub Actions: Generate documentation GitHub Actions: Goreleaser GitHub Actions: Bump Version ✅ 9 other checks have passed 9/12 successful checks

#3334 [Core feature] Iterable FlyteDirectory to support downloading individual files on the fly Issue created by cosmicBboy Motivation: Why do you think this is important? With compute-heavy, acceleration-dependent workloads like deep learning model training, it's desirable to start training as soon as possible so as to avoid wasting time downloading the full dataset. This becomes more important the larger the dataset size is. Assuming that the full dataset can fit in disk, it would improve cost efficiency to be able to start training on batches of data as soon as the machine is available. In the ML training use case, datasets are often organized as files, where each file is a data point. For example, imagine a dataset of images, where a special

labels.txt

file contains the labels of each image

example_*.png

.

dataset /
    labels.txt
    example_abc.png
    example_xyz.png
    ...

Goal: What should the final outcome look like, ideally? As a Flyte user, I should be able to lazily iterate over a

FlyteDirectory

of such a dataset such that I don't have to download the entire directory and instead start training as soon as the first batch of data is available on the running Pod. Requirements • Should support iteration over files in the directory in a random order • Potentially support iteration of batches of files in a random order Describe alternatives you've considered Users would have to create their own workaround to: 1. store the filenames for all the examples in a custom Flyte type (probably a

dataclass

) 2. create their own iterable downloader by combining the root FlyteDirectory with the filenames from (1) and use the FileAccessProvider to fetch individual files. 3. iterate over the files in the user-defined dataloader Propose: Link/Inline OR Additional context No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#3335 [Docs] Add Deployment Docs for Azure Issue created by cosmicBboy Description The purpose of this issue is to add deployment instructions for Flyte on Azure: https://docs.flyte.org/en/latest/deployment/deployment/index.html#deployment-deployment Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

1 new commit pushed to

<https://github.com/flyteorg/flytekit/tree/master|master>

by cosmicBboy

<https://github.com/flyteorg/flytekit/commit/ecded3e65771f77b6f5b341887b5f7c0440eadb2|ecded3e6>

- add undocumented objects/functions to flytekit api ref (#1502) flyteorg/flytekit

Pending review on flyteorg/flyteplugins - Manage reminder [#310] Transition to failed state if the task cannot be created (pingsutw) 14 days stale · 24 days old · Waiting on _ hamersaw_

Pending review on flyteorg/flytekit-java - Manage reminder [#6] Update .env.local in README (kanterov) 10 months stale · 3 years old · Waiting on _ @Ketan (kumare3)_

Pending review on flyteorg/flytestdlib - Manage reminder [#117] Update boilerplate version (flyte-bot) 7 months stale · a year old · Waiting on _ @Haytham Abuelfutuh, katrogan, @Yee, @Ketan (kumare3)_

1 new commit pushed to

<https://github.com/flyteorg/community/tree/main|main>

by kumare3

<https://github.com/flyteorg/community/commit/3b050050b5ef354bdc8e3236d30eb9ee10a7242b|3b050050>

- Initial commit flyteorg/community

1 new commit pushed to

<https://github.com/flyteorg/flytesnacks/tree/master|master>

by cosmicBboy

<https://github.com/flyteorg/flytesnacks/commit/2903a9598aaf1b3049467558aa6e6baceb5a73f3|2903a959>

- Getting started cleanup (#958) flyteorg/flytesnacks

#1508 Get the origin type when serializing dataclass Pull request opened by pingsutw Signed-off-by: Kevin Su pingsutw@apache.org TL;DR Fixes flyteorg/flyte#3011 Get the origin type from annotated when serializing dataclass because some args in annotated are not hashable. Type ☑︎ Bug Fix ☐ Feature ☐ Plugin Are all requirements met? ☑︎ Code completed ☑︎ Smoke tested ☐ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description ^^^ Tracking Issue flyteorg/flyte#3011 Follow-up issue NA flyteorg/flytekit GitHub Actions: build-plugins (3.8, flytekit-deck-standard) GitHub Actions: build-plugins (3.8, flytekit-dbt) GitHub Actions: build-plugins (3.8, flytekit-data-fsspec) GitHub Actions: build-plugins (3.8, flytekit-dask) GitHub Actions: build-plugins (3.8, flytekit-bigquery) GitHub Actions: build-plugins (3.8, flytekit-aws-sagemaker) GitHub Actions: build-plugins (3.8, flytekit-aws-batch) GitHub Actions: build-plugins (3.8, flytekit-aws-athena) GitHub Actions: build (windows-latest, 3.9) GitHub Actions: build (windows-latest, 3.8) GitHub Actions: build (windows-latest, 3.7, -spark2) GitHub Actions: build (windows-latest, 3.7) GitHub Actions: build (ubuntu-latest, 3.10) GitHub Actions: build (ubuntu-latest, 3.9) GitHub Actions: build (ubuntu-latest, 3.8) GitHub Actions: build (ubuntu-latest, 3.7, -spark2) GitHub Actions: build (ubuntu-latest, 3.7) GitHub Actions: lint GitHub Actions: docs ✅ 2 other checks have passed 2/21 successful checks

#3232 [Core feature] Pass inputs inline with execution event data Issue created by katrogan Motivation: Why do you think this is important? Follow-up to #1327. For a distributed propeller deployment it's useful to pass executions inputs inline rather than as an offloaded URI. Goal: What should the final outcome look like, ideally? Raw inputs should be (configurably) sent inline in execution events Describe alternatives you've considered Current implementation Propose: Link/Inline OR Additional context No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#3292 testing inline event inputs Pull request opened by katrogan on <!date^1674868184^{date_short}|2023-01-28T01:09:44Z> Signed-off-by: Katrina Rogan katrina@Katrinas-MBP.UNION.AI Tracking issue Describe your changes Check all the applicable boxes ☐ I updated the documentation accordingly. ☐ All new and existing tests passed. ☐ All commits are signed-off. Screenshots Note to reviewers flyteorg/flyte GitHub Actions: trigger-sandbox-bundled-build GitHub Actions: compile GitHub Actions: trigger-single-binary-build ✅ 5 other checks have passed 5/8 successful checks

#3336 Update Flyte components Pull request opened by flyte-bot Updated flyte deployment • Updated GCP Flyte kustomize generated manifest file • Updated EKS Flyte kustomize generated manifest file • Updated Sandbox Flyte kustomize generated manifest file • Updated TEST Flyte kustomize generated manifest file • Updated GCP Flyte helm generated manifest file • Updated EKS Flyte helm generated manifest file • Updated Sandbox Flyte helm generated manifest file • Updated TEST Flyte helm generated manifest file • Auto-generated by [flyte-bot] flyteorg/flyte ✅ All checks have passed 12/12 successful checks