GitHub
02/11/2023, 7:22 PMGitHub
02/11/2023, 7:22 PM<https://github.com/flyteorg/homebrew-tap/tree/main|main>
by flyte-bot
<https://github.com/flyteorg/homebrew-tap/commit/25bc71ddf77dda5cf87eaea086c7f517caf205b1|25bc71dd>
- Brew formula update for flytectl version v0.6.29
flyteorg/homebrew-tapGitHub
02/11/2023, 7:25 PMGitHub
02/12/2023, 12:00 AMpip
dependencies of this project.
Changes included in this PR
• Changes to the following files to upgrade the vulnerable dependencies to a fixed version:
• doc-requirements.txt
⚠️ Warning
vaex 4.16.0 requires vaex-core, which is not installed.
vaex-viz 0.5.4 requires vaex-core, which is not installed.
vaex-server 0.8.1 requires vaex-core, which is not installed.
vaex-server 0.8.1 requires fastapi, which is not installed.
vaex-server 0.8.1 requires uvicorn, which is not installed.
vaex-ml 0.18.1 requires vaex-core, which is not installed.
vaex-jupyter 0.8.1 requires vaex-core, which is not installed.
vaex-jupyter 0.8.1 requires ipyvolume, which is not installed.
vaex-jupyter 0.8.1 requires xarray, which is not installed.
vaex-hdf5 0.14.1 requires vaex-core, which is not installed.
vaex-astro 0.9.3 requires vaex-core, which is not installed.
protoc-gen-swagger 0.1.0 requires protobuf, which is not installed.
notebook 5.7.16 requires terminado, which is not installed.
nbformat 4.4.0 requires jsonschema, which is not installed.
nbconvert 5.6.1 has requirement mistune<2,>=0.8.1, but you have mistune 2.0.4.
jupyter 1.0.0 requires qtconsole, which is not installed.
jupyter 1.0.0 requires jupyter-console, which is not installed.
ipython 5.10.0 requires simplegeneric, which is not installed.
ipympl 0.9.2 has requirement matplotlib<4,>=3.4.0, but you have matplotlib 2.2.5.
ipyleaflet 0.17.0 requires xyzservices, which is not installed.
ImageHash 4.3.1 requires scipy, which is not installed.
ImageHash 4.3.1 requires PyWavelets, which is not installed.
google-auth 2.16.0 requires rsa, which is not installed.
flyteidl 1.3.5 requires googleapis-common-protos, which is not installed.
flyteidl 1.3.5 requires protobuf, which is not installed.
docker-image-py 0.1.12 requires regex, which is not installed.
databricks-cli 0.17.4 requires pyjwt, which is not installed.
databricks-cli 0.17.4 requires tabulate, which is not installed.
Vulnerabilities that will be fixed
By pinning:
(*) Note that the real score may have changed since the PR was raised.
Some vulnerabilities couldn't be fully fixed and so Snyk will still find them when the project is tested again. This may be because the vulnerability existed within more than one direct dependency, but not all of the affected dependencies could be upgraded.
Check the changes in this PR to ensure they won't cause issues with your project.
* * *
Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.
For more information: https://camo.githubusercontent.com/181f688b16e2fb15bfa14c7b36bf0c89eec6fb84a3f607ab54beb3ec598e4231/68747470733a2f2f6170692e7365676d656e742e696f2f76312f706978656c2f747261636b3f646174613d65794a33636d6c305a55746c65534936496e4a79576d785a634564485932527954485a7362306c596430645563566734576b4652546e4e434f5545774969776959573576626e6c746233567a535751694f694a6b5a444e6d5957566a4e69316c4e475a684c54526c596a6b74596a52694e5330774f575931597a51325a6a5932596a67694c434a6c646d567564434936496c425349485a705a58646c5a434973496e42796233426c636e52705a584d694f6e736963484a4a5a434936496d526b4d325a685a574d324c5755305a6d45744e4756694f5331694e4749314c5441355a6a566a4e445a6d4e6a5a694f434a3966513d3d
🧐 View latest project report
🛠️ Adjust project settings
📚 Read more about Snyk's upgrade and patch logic
* * *
Learn how to fix vulnerabilities with free interactive lessons:
🦉 Learn about vulnerability in an interactive lesson of Snyk Learn.
flyteorg/flytekit
GitHub Actions: build (windows-latest, 3.9)
GitHub Actions: build (windows-latest, 3.8)
GitHub Actions: build (windows-latest, 3.7, -spark2)
GitHub Actions: build (windows-latest, 3.7)
GitHub Actions: build (ubuntu-latest, 3.10)
GitHub Actions: build (ubuntu-latest, 3.9)
GitHub Actions: build (ubuntu-latest, 3.8)
GitHub Actions: build (ubuntu-latest, 3.7, -spark2)
GitHub Actions: build (ubuntu-latest, 3.7)
GitHub Actions: lint
GitHub Actions: docs
GitHub Actions: build-plugins (3.10, flytekit-vaex)
GitHub Actions: build-plugins (3.10, flytekit-sqlalchemy)
GitHub Actions: build-plugins (3.10, flytekit-spark)
GitHub Actions: build-plugins (3.10, flytekit-ray)
GitHub Actions: build-plugins (3.10, flytekit-papermill)
GitHub Actions: build-plugins (3.10, flytekit-hive)
GitHub Actions: build-plugins (3.10, flytekit-dolt)
GitHub Actions: build-plugins (3.10, flytekit-deck-standard)
GitHub Actions: build-plugins (3.10, flytekit-dbt)
GitHub Actions: build-plugins (3.10, flytekit-dask)
✅ 9 other checks have passed
9/30 successful checksGitHub
02/12/2023, 3:51 AMpip
dependencies of this project.
Changes included in this PR
• Changes to the following files to upgrade the vulnerable dependencies to a fixed version:
• plugins/flytekit-greatexpectations/requirements.txt
⚠️ Warning
protoc-gen-swagger 0.1.0 requires protobuf, which is not installed.
flyteidl 1.3.0 requires protobuf, which is not installed.
flyteidl 1.3.0 requires googleapis-common-protos, which is not installed.
docker-image-py 0.1.12 requires regex, which is not installed.
binaryornot 0.4.4 requires chardet, which is not installed.
Vulnerabilities that will be fixed
By pinning:
(*) Note that the real score may have changed since the PR was raised.
Some vulnerabilities couldn't be fully fixed and so Snyk will still find them when the project is tested again. This may be because the vulnerability existed within more than one direct dependency, but not all of the affected dependencies could be upgraded.
Check the changes in this PR to ensure they won't cause issues with your project.
* * *
Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.
For more information: https://camo.githubusercontent.com/19e0170646ad9f2e7b4c1d6145b7cea84dc2ece846eb28e60316358af50d92a2/68747470733a2f2f6170692e7365676d656e742e696f2f76312f706978656c2f747261636b3f646174613d65794a33636d6c305a55746c65534936496e4a79576d785a634564485932527954485a7362306c596430645563566734576b4652546e4e434f5545774969776959573576626e6c746233567a535751694f6949794d6d45314e7a6c684e5330305a6a59774c5451314d7a4174596a41784d4330334f474d79595746694d7a55335a544d694c434a6c646d567564434936496c425349485a705a58646c5a434973496e42796233426c636e52705a584d694f6e736963484a4a5a434936496a4979595455334f5745314c54526d4e6a41744e44557a4d4331694d4445774c546334597a4a685957497a4e54646c4d794a3966513d3d
🧐 View latest project report
🛠️ Adjust project settings
📚 Read more about Snyk's upgrade and patch logic
* * *
Learn how to fix vulnerabilities with free interactive lessons:
🦉 Learn about vulnerability in an interactive lesson of Snyk Learn.
flyteorg/flytekit
✅ All checks have passed
30/30 successful checksGitHub
02/12/2023, 7:22 AMGitHub
02/12/2023, 1:30 PMGitHub
02/13/2023, 6:45 AMguess_python_type
method has been moved to the subclasses without retaining it in the parent class. This will return the correct type when fetching the execution output via FlyteRemote.
Tracking Issue
flyteorg/flyte#3148
Follow-up issue
NA
OR
https://github.com/flyteorg/flyte/issues/
flyteorg/flytekit
GitHub Actions: build-plugins (3.8, flytekit-duckdb)
GitHub Actions: build-plugins (3.8, flytekit-dolt)
GitHub Actions: build-plugins (3.8, flytekit-deck-standard)
GitHub Actions: build-plugins (3.8, flytekit-dbt)
GitHub Actions: build-plugins (3.8, flytekit-data-fsspec)
GitHub Actions: build-plugins (3.8, flytekit-dask)
GitHub Actions: build-plugins (3.8, flytekit-bigquery)
GitHub Actions: build-plugins (3.8, flytekit-aws-sagemaker)
GitHub Actions: build-plugins (3.8, flytekit-aws-batch)
GitHub Actions: build-plugins (3.8, flytekit-aws-athena)
GitHub Actions: lint
GitHub Actions: docs
✅ 2 other checks have passed
2/14 successful checksGitHub
02/13/2023, 6:53 AMGitHub
02/13/2023, 8:19 AMSdkLiteralType
and move factory methods out of SdkBindingData
Type
☐ Bug Fix
☑︎ Feature
☐ Plugin
Are all requirements met?
☑︎ Code completed
☐ Smoke tested
☑︎ Unit tests added
☑︎ Code documentation added
☑︎ Any pending items have an associated Issue
Complete description
In the Sdk there is a SdkType that represent an abstraction for the type of a task or workflow and its multiple variables. However we are missing an equivalent to represent the type of a single value.
SdkBindingData.ofBindingCollection(
LiteralType.ofCollectionType(LiteralType.ofSimpleType(SimpleType.INTEGER)),
List.of(sum1.getOutputs().o(), sum2.getOutputs().o()));
The api module was designed to be stable a follow closely the proto classes, that is the reason of why is so verbose
Instead with the proper abstraction we could write this instead:
SdkBindingDatas.ofBindingCollection(
SdkLiteralTypes.integers(), List.of(sum1.getOutputs().o(), sum2.getOutputs().o()));
In addition, we moved the factory methods in SdkBindingData to it's own class, so Java and Scala have a more similar API
Tracking Issue
fixes flyteorg/flyte#3332
Follow-up issue
Not an issue yet, but SdkBindinDatas
flyteorg/flytekit-java
✅ All checks have passed
3/3 successful checksGitHub
02/13/2023, 8:30 AMSdkType
that represent an abstraction for the type of a task or workflow and its multiple variables. However we are missing an equivalent to represent the type of a single value.
This causes that we were leaking internal classes of the api module into the users code. For example, to create a SdkBindingData for a list of bindings, the user needs to:
SdkBindingData.ofBindingCollection(
LiteralType.ofCollectionType(LiteralType.ofSimpleType(SimpleType.INTEGER)),
List.of(sum1.getOutputs().o(), sum2.getOutputs().o()));
The api module was designed to be stable a follow closely the proto classes, that is the reason of why is so verbosed
Instead with the proper abstraction we could write this instead:
SdkBindingDatas.ofBindingCollection(
SdkLiteralTypes.integers(), List.of(sum1.getOutputs().o(), sum2.getOutputs().o()));
What if we do not do this?
We would had a worse user experience because of the verbosity of the api
Related component(s)
flytekit-java
Are you sure this issue hasn't been raised already?
☑︎ Yes
Have you read the Code of Conduct?
☑︎ Yes
flyteorg/flyteGitHub
02/13/2023, 12:56 PM<https://github.com/flyteorg/flytesnacks/tree/master|master>
by samhita-alla
<https://github.com/flyteorg/flytesnacks/commit/2383f4a069000380bf7ff9c54d401c38ae90ffa6|2383f4a0>
- Improve pod example (#937)
flyteorg/flytesnacksGitHub
02/13/2023, 12:56 PMGitHub
02/13/2023, 12:57 PM9.1.1
This release addresses several security problems.
CVE-2022-30595: When reading a TGA file with RLE packets that cross scan lines, Pillow reads the information past the end of the first line without deducting that from the length of the remaining file data. This vulnerability was introduced in Pillow 9.1.0, and can cause a heap buffer overflow.
Opening an image with a zero or negative height has been found to bypass a decompression bomb check. This will now raise a `SyntaxError` instead, in turn raising aChangelog Sourced from pillow's changelog..PIL.UnidentifiedImageError
9.1.1 (2022-05-17)
• When reading past the end of a TGA scan line, reduce bytes left. CVE-2022-30595 [radarhere]
• Do not open images with zero or negative height #6269 [radarhere]Commits • `0f44136` 9.1.1 version bump • `f66f5e1` pre-commit: update Black to fix Click • `0153b37` Skip test_realloc_overflow unless libtiff 4.0.4 or higher • `6fcd31b` Added release notes for 9.1.1 • `c846cc8` When reading past the end of a scan line, reduce bytes left • `184b73e` Do not open images with zero or negative height • See full diff in compare view Dependabot compatibility score Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting
@dependabot rebase
.
* * *
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
• @dependabot rebase
will rebase this PR
• @dependabot recreate
will recreate this PR, overwriting any edits that have been made to it
• @dependabot merge
will merge this PR after your CI passes on it
• @dependabot squash and merge
will squash and merge this PR after your CI passes on it
• @dependabot cancel merge
will cancel a previously requested merge and block automerging
• @dependabot reopen
will reopen this PR if it is closed
• @dependabot close
will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
• @dependabot ignore this major version
will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this minor version
will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this dependency
will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
• @dependabot use these labels
will set the current labels as the default for future PRs for this repo and language
• @dependabot use these reviewers
will set the current reviewers as the default for future PRs for this repo and language
• @dependabot use these assignees
will set the current assignees as the default for future PRs for this repo and language
• @dependabot use this milestone
will set the current milestone as the default for future PRs for this repo and language
You can disable automated security fix PRs for this repo from the Security Alerts page.
flyteorg/flytesnacks
GitHub Actions: Mark github pre-release as Release
GitHub Actions: Publish artifacts to github release
GitHub Actions: Create Prerelease
GitHub Actions: Bump Version
GitHub Actions: Serialize & Register Flytesnacks workflow / Register example to sandbox
✅ 25 other checks have passed
25/30 successful checksGitHub
02/13/2023, 1:37 PMGitHub
02/13/2023, 3:46 PM<https://github.com/flyteorg/flyteplugins/tree/master|master>
by hamersaw
<https://github.com/flyteorg/flyteplugins/commit/5981c35dbe2033e9905283c4ea4b667e79f0bf8b|5981c35d>
- Rename dask port from to (#315)
flyteorg/flytepluginsGitHub
02/13/2023, 3:47 PMGitHub
02/13/2023, 3:52 PM<http://github.com/ghodss/yaml|github.com/ghodss/yaml>
and <http://gopkg.in/yaml.v2|gopkg.in/yaml.v2>
Type
☐ Bug Fix
☐ Feature
☐ Plugin
Are all requirements met?
☑︎ Code completed
☐ Smoke tested
☐ Unit tests added
☐ Code documentation added
☐ Any pending items have an associated Issue
Complete description
Currently four YAML packages are used:
1. <http://github.com/ghodss/yaml|github.com/ghodss/yaml>
2. <http://gopkg.in/yaml.v2|gopkg.in/yaml.v2>
3. <http://gopkg.in/yaml.v3|gopkg.in/yaml.v3>
4. <http://sigs.k8s.io/yaml|sigs.k8s.io/yaml>
We can keep only two of them:
1. <http://gopkg.in/yaml.v3|gopkg.in/yaml.v3>
2. <http://sigs.k8s.io/yaml|sigs.k8s.io/yaml>
The package <http://github.com/ghodss/yaml|github.com/ghodss/yaml>
is no longer actively maintained. See discussion in ghodss/yaml#80 and ghodss/yaml#75. <http://sigs.k8s.io/yaml|sigs.k8s.io/yaml>
is a permanent fork of ghodss/yaml
and is actively maintained by Kubernetes SIG, also widely used in K8s projects.
We are only using <http://gopkg.in/yaml.v2|gopkg.in/yaml.v2>
in cmd/config/subcommand/project/project_config.go
. Switching to <http://gopkg.in/yaml.v3|gopkg.in/yaml.v3>
doesn't introduce breaking change here.
Tracking Issue
Follow-up issue
NA
flyteorg/flytectl
GitHub Actions: Generate documentation
GitHub Actions: Goreleaser
GitHub Actions: Bump Version
✅ 9 other checks have passed
9/12 successful checksGitHub
02/13/2023, 3:59 PMlabels.txt
file contains the labels of each image example_*.png
.
dataset /
labels.txt
example_abc.png
example_xyz.png
...
Goal: What should the final outcome look like, ideally?
As a Flyte user, I should be able to lazily iterate over a FlyteDirectory
of such a dataset such that I don't have to download the entire directory and instead start training as soon as the first batch of data is available on the running Pod.
Requirements
• Should support iteration over files in the directory in a random order
• Potentially support iteration of batches of files in a random order
Describe alternatives you've considered
Users would have to create their own workaround to:
1. store the filenames for all the examples in a custom Flyte type (probably a dataclass
)
2. create their own iterable downloader by combining the root FlyteDirectory with the filenames from (1) and use the FileAccessProvider to fetch individual files.
3. iterate over the files in the user-defined dataloader
Propose: Link/Inline OR Additional context
No response
Are you sure this issue hasn't been raised already?
☑︎ Yes
Have you read the Code of Conduct?
☑︎ Yes
flyteorg/flyteGitHub
02/13/2023, 4:17 PMGitHub
02/13/2023, 4:59 PM<https://github.com/flyteorg/flytekit/tree/master|master>
by cosmicBboy
<https://github.com/flyteorg/flytekit/commit/ecded3e65771f77b6f5b341887b5f7c0440eadb2|ecded3e6>
- add undocumented objects/functions to flytekit api ref (#1502)
flyteorg/flytekitGitHub
02/13/2023, 5:00 PMGitHub
02/13/2023, 5:00 PMGitHub
02/13/2023, 5:00 PMGitHub
02/13/2023, 5:19 PM<https://github.com/flyteorg/community/tree/main|main>
by kumare3
<https://github.com/flyteorg/community/commit/3b050050b5ef354bdc8e3236d30eb9ee10a7242b|3b050050>
- Initial commit
flyteorg/communityGitHub
02/13/2023, 8:03 PM<https://github.com/flyteorg/flytesnacks/tree/master|master>
by cosmicBboy
<https://github.com/flyteorg/flytesnacks/commit/2903a9598aaf1b3049467558aa6e6baceb5a73f3|2903a959>
- Getting started cleanup (#958)
flyteorg/flytesnacksGitHub
02/13/2023, 8:13 PMGitHub
02/13/2023, 8:19 PMGitHub
02/13/2023, 8:45 PMGitHub
02/13/2023, 8:47 PM