flyte-org #flyte-github

GitHub

02/06/2023, 5:41 PM

#3315 Fix maptask caching in FlyteConsole Issue created by cosmicBboy Map tasks in the console are not showing the correct cached status: This issue requires both backend and frontend fixes: • Backend PR: flyteorg/flyteplugins#300 • Issue for Frontend fix: flyteorg/flyteconsole#644 flyteorg/flyte

GitHub

02/06/2023, 6:04 PM

#3316 Execution Details Performance Refactor Issue created by cosmicBboy This issue is a duplicate of unionai/cloud#1646, just creating this issue for tracking under Flyte OSS project flyteorg/flyte

GitHub

02/06/2023, 6:11 PM

#3317 Support uploading files to Flyte blob store in the Flyte Console launch form Issue created by cosmicBboy There isnt a straightforward way of uploading data/artifacts (e.g. models) to Flyte blob store without writing a task, registering it, executing it. Assumption: As a user, I've registered a workflow that has a

FlyteFile

as an input. When I go to the launch form on the Flyte UI to execute a workflow, currently I need to reference a specific URI string (e.g. S3 path) pointing to a file that I want to use as input. As a user, I want the option to upload a file from my local computer as part of the launchform UI so that I don't have to manually upload an s3 file and point the workflow to it. Comps: comps: Launch Plan File upload flyteorg/flyte

GitHub

02/06/2023, 6:16 PM

#3318 Fix flytekit integration tests: need to fix pytest-flyte, then the flytekit integration test suite Issue created by cosmicBboy Currently, the

pytest-flyte

package uses a custom setup to create a local Flyte cluster for the purpose of integration testing. Due to some of its dependencies (e.g.

docker-compose

) we've decided to disable integration testing in

flytekit

until we've update

pytest-flyte

to use

flytectl

to handle the setup/teardown of the local Flyte cluster. Once that's done, we can re-enable integration tests in

flytekit

. Steps: ☐ refactor

pytest-flyte

to use

flytectl

instead of

docker-compose

to setup/teardown a local Flyte cluster ☐ re-enable integration tests as part of flytekit CI/CD flyteorg/flyte

GitHub

02/06/2023, 6:50 PM

1 new commit pushed to

<https://github.com/flyteorg/flyte/tree/master|master>

by eapolinario

<https://github.com/flyteorg/flyte/commit/0a6e46f8dbe6befe8637915b07885519af80f060|0a6e46f8>

- Fixed notification region config (#3297) flyteorg/flyte

GitHub

02/06/2023, 8:22 PM

1 new commit pushed to

<https://github.com/flyteorg/flytesnacks/tree/master|master>

by cosmicBboy

<https://github.com/flyteorg/flytesnacks/commit/4470b61327658e4a9c0fe470ac5e058e4a04524c|4470b613>

- Add retry to interruptible docs (#951) flyteorg/flytesnacks

GitHub

02/06/2023, 8:45 PM

#1433 set maximum python version to 3.10 Pull request opened by cosmicBboy Signed-off-by: Niels Bantilan niels.bantilan@gmail.com TL;DR Fixes flyteorg/flyte#3313, which allows users to install flytekit on python 3.11 but then come up against errors on import. Type ☑︎ Bug Fix ☐ Feature ☐ Plugin Are all requirements met? ☑︎ Code completed ☑︎ Smoke tested • ~~[ ] Unit tests added~~ • ~~[ ] Code documentation added~~ • ~~[ ] Any pending items have an associated Issue~~ Complete description This restriction was expressed softly in the

classifiers

tags but not programmatically enforced. Test Plan • Create a python virtual environment on python 3.11 (either with

pyenv

miniconda

) • Go to the

flytekit

root repo •

pip install -e .

to do an editable install • Expected outcome: you should see an error:

Copy code

ERROR: Package 'flytekit' requires a different Python: 3.11.0 not in '<3.11,>=3.7'

Tracking Issue https://github.com/flyteorg/flyte/issues/ Follow-up issue NA OR https://github.com/flyteorg/flyte/issues/ flyteorg/flytekit ✅ All checks have passed 30/30 successful checks

GitHub

02/06/2023, 9:46 PM

#3319 [BUG] OneHotEncoder type not supported as task input when None is supplied as input value. Issue created by xshen8888 Describe the bug The task’s input is of type sklearn.preprocessing.OneHotEncoder. It is either supplied by the caller or None if user doesn’t have a encoder instance as input (in which case the task code will get it from an offline storage location). I tried both, with or without Optional.

Copy code

@task
def transform(encoder: Optional[OneHotEncoder] = None, ...)
def transform(encoder: OneHotEncoder = None, ...)
When caller do transform(encoder=None, ...), the task will work successfully with pyflyte run (locally).
However, if I bake the module containing the task estimate() into an image and refer to it in a pyflyte remote run, no mater what I do, I keep getting at the terminal (before pyflyte submit the code to remote flyte server):
flytekit/core/type_engine.py", line 1060, in to_literal
    raise TypeError("Ambiguous choice of variant for union type")
...
flytekit/core/promise.py", line 937, in create_and_link_node
    raise AssertionError(f"Failed to Bind variable {k} for function {entity.name}.") from e
AssertionError: Failed to Bind variable **_encoder_** for function dai_mle_models.onehotenc.simple_onehotenc. transform.

Expected behavior The task should run remotely just like when None is supplied to pandas.DataFrame as task input value (which I have verified working). e.g.

Copy code

@task
def sometask(test_data: pd.DataFrame = None, ...)

Additional context to reproduce No response Screenshots No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

GitHub

02/06/2023, 10:59 PM

#954 [quickfix] hard-code max_iter Pull request opened by cosmicBboy Signed-off-by: Niels Bantilan niels.bantilan@gmail.com This is related to flyteorg/flyte#3158, which we're working on, but as a quickfix, hard-code the

max_iter

variable so that it doesn't cause issues with end users. flyteorg/flytesnacks ✅ All checks have passed 2/2 successful checks

Jimmy Du

02/06/2023, 11:06 PM

@Jimmy Du has left the channel

GitHub

02/06/2023, 11:17 PM

1 new commit pushed to

<https://github.com/flyteorg/flytesnacks/tree/master|master>

by cosmicBboy

<https://github.com/flyteorg/flytesnacks/commit/bbaf42bca36f9a0192ce14bb742a5fd32375d2bb|bbaf42bc>

- added scikit-learn dependency callout (#950) flyteorg/flytesnacks

GitHub

02/06/2023, 11:23 PM

#1434 Use scarf images Pull request opened by eapolinario Signed-off-by: Eduardo Apolinario eapolinario@users.noreply.github.com TL;DR Use scarf images to collect usage for default flytekit images. Type ☐ Bug Fix ☐ Feature ☐ Plugin Are all requirements met? ☐ Code completed ☐ Smoke tested ☐ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description How did you fix the bug, make the feature etc. Link to any design docs etc Tracking Issue https://github.com/flyteorg/flyte/issues/ Follow-up issue NA OR https://github.com/flyteorg/flyte/issues/ flyteorg/flytekit ✅ All checks have passed 30/30 successful checks

GitHub

02/06/2023, 11:28 PM

#90 Bump go version to 1.19 Pull request opened by hamersaw TL;DR Bumping the go version to 1.19 to fix pull request checks. Type ☐ Bug Fix ☑︎ Feature ☐ Plugin Are all requirements met? ☑︎ Code completed ☑︎ Smoke tested ☐ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description ^^^ Tracking Issue NA Follow-up issue NA flyteorg/datacatalog GitHub Actions: Goreleaser GitHub Actions: Build & Push Datacatalog Image GitHub Actions: Bump Version Codecov: 68.50% (-1.05%) compared to 8a0429f ✅ 7 other checks have passed 7/11 successful checks

GitHub

02/06/2023, 11:32 PM

#316 Bumping go version to 1.19 Pull request opened by hamersaw TL;DR Bumping the go version to 1.19 to fix pull request checks. Type ☐ Bug Fix ☑︎ Feature ☐ Plugin Are all requirements met? ☑︎ Code completed ☑︎ Smoke tested ☐ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description ^^^ Tracking Issue NA Follow-up issue NA flyteorg/flyteplugins Codecov: 62.94% (-0.09%) compared to 471a262 ✅ 5 other checks have passed 5/6 successful checks

GitHub

02/06/2023, 11:35 PM

#523 Bumping go version to 1.19 Pull request opened by hamersaw TL;DR Bumping the go version to 1.19 to fix pull request checks. Type ☐ Bug Fix ☑︎ Feature ☐ Plugin Are all requirements met? ☑︎ Code completed ☑︎ Smoke tested ☐ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description ^^^ Tracking Issue NA Follow-up issue NA flyteorg/flytepropeller GitHub Actions: Goreleaser GitHub Actions: Build & Push Flytepropeller Image GitHub Actions: Bump Version ✅ 11 other checks have passed 11/14 successful checks

GitHub

02/07/2023, 1:14 AM

1 new commit pushed to

<https://github.com/flyteorg/static-resources/tree/main|main>

by kumare3

<https://github.com/flyteorg/static-resources/commit/3f36d8546df23df84d596cd9a0d96dea61a8d02b|3f36d854>

- RFC images for Flyte tags flyteorg/static-resources

GitHub

02/07/2023, 1:15 AM

#363 Bumping go version to 1.19 Pull request opened by hamersaw TL;DR Bumping the go version to 1.19 to fix pull request checks. Type ☐ Bug Fix ☑︎ Feature ☐ Plugin Are all requirements met? ☑︎ Code completed ☑︎ Smoke tested ☐ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description ^^^ Tracking Issue NA Follow-up issue NA flyteorg/flyteidl ✅ All checks have passed 13/13 successful checks

GitHub

02/07/2023, 1:16 AM

#3320 [Proposal]: Flyte System Tags and metadata Pull request opened by kumare3 This RFC proposes a way to add execution Tags and description. this would help the user in multple ways. Please read the rfc for more context. Signed-off-by: Ketan Umare ketan.umare@gmail.com flyteorg/flyte GitHub Actions: docs GitHub Actions: compile GitHub Actions: generate_helm GitHub Actions: generate_kustomize ✅ 2 other checks have passed 2/6 successful checks

GitHub

02/07/2023, 1:20 AM

#150 Bumping go version to 1.19 Pull request opened by hamersaw TL;DR Bumping the go version to 1.19 to fix pull request checks. Type ☐ Bug Fix ☑︎ Feature ☐ Plugin Are all requirements met? ☑︎ Code completed ☑︎ Smoke tested ☐ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description ^^^ Tracking Issue NA Follow-up issue NA flyteorg/flytestdlib ✅ All checks have passed 5/5 successful checks

GitHub

02/07/2023, 1:58 AM

#679 chore: launch form bug for fast-registered workflows Pull request opened by ursucarina Signed-off-by: Carina Ursu carina@union.ai TL;DR Fixes launch form bug for fast-registered workflows Type ☑︎ Bug Fix ☐ Feature ☐ Plugin Are all requirements met? ☑︎ Code completed ☑︎ Smoke tested ☐ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description Changed the import for the launch workflow import flyteorg/flyteconsole GitHub Actions: Build & Push FlyteConsole Image GitHub Actions: Get Release Tag GitHub Actions: Generate Release ✅ 8 other checks have passed 8/11 successful checks

GitHub

02/07/2023, 2:16 AM

#71 Bumping go version to 1.19 Pull request opened by hamersaw Signed-off-by: Daniel Rammer daniel@union.ai flyteorg/flytetools ✅ All checks have passed 2/2 successful checks

GitHub

02/07/2023, 12:52 PM

1 new commit pushed to

<https://github.com/flyteorg/flytestdlib/tree/master|master>

by hamersaw

<https://github.com/flyteorg/flytestdlib/commit/82d114b519111eff3d4ca7c72fa50d4e07800924|82d114b5>

- bumping go version to 1.19 (#150) flyteorg/flytestdlib

GitHub

02/07/2023, 12:59 PM

Release - v1.0.15 New release published by github-actions[bot] Changelog • 82d114b bumping go version to 1.19 (#150) flyteorg/flytestdlib

GitHub

02/07/2023, 3:50 PM

1 new commit pushed to

<https://github.com/flyteorg/flytesnacks/tree/master|master>

by cosmicBboy

<https://github.com/flyteorg/flytesnacks/commit/80848a18f42e27de81dacab48bb1b9e971525a55|80848a18>

- [quickfix] hard-code max_iter (#954) flyteorg/flytesnacks

GitHub

02/07/2023, 4:01 PM

#955 always check docs in CI/CD Pull request opened by cosmicBboy Signed-off-by: Niels Bantilan niels.bantilan@gmail.com This PR makes it so that the docs are always built to check for errors/warnings. flyteorg/flytesnacks GitHub Actions: Mark github pre-release as Release GitHub Actions: Publish artifacts to github release GitHub Actions: Create Prerelease GitHub Actions: Bump Version ✅ 26 other checks have passed 26/30 successful checks

GitHub

02/07/2023, 4:43 PM

1 new commit pushed to

<https://github.com/flyteorg/flytesnacks/tree/master|master>

by eapolinario

<https://github.com/flyteorg/flytesnacks/commit/43585feeccabc8a48452dc6838426f3acf4c6a9d|43585fee>

- always check docs in CI/CD (#955) flyteorg/flytesnacks

GitHub

02/07/2023, 5:19 PM

1 new commit pushed to

<https://github.com/flyteorg/flyteconsole/tree/master|master>

by jsonporter

<https://github.com/flyteorg/flyteconsole/commit/bed9c1b7d7bf81cfc46fcf9d9fddbe47eb6319c0|bed9c1b7>

- chore: launch form bug for fast-registered workflows (#679) flyteorg/flyteconsole

GitHub

02/07/2023, 5:45 PM

#1435 [Snyk] Security upgrade numpy from 1.21.3 to 1.22.2 Pull request opened by eapolinario This PR was automatically created by Snyk using the credentials of a real user. Snyk has created this PR to fix one or more vulnerable packages in the
pip
dependencies of this project. Changes included in this PR • Changes to the following files to upgrade the vulnerable dependencies to a fixed version: • requirements.txt ⚠️ Warning

Copy code

protoc-gen-swagger 0.1.0 requires protobuf, which is not installed.
jsonschema 3.2.0 requires pyrsistent, which is not installed.
google-auth 2.16.0 requires rsa, which is not installed.
flyteidl 1.3.5 requires protobuf, which is not installed.
flyteidl 1.3.5 requires googleapis-common-protos, which is not installed.
docker-image-py 0.1.12 requires regex, which is not installed.

Vulnerabilities that will be fixed By pinning: (*) Note that the real score may have changed since the PR was raised. Some vulnerabilities couldn't be fully fixed and so Snyk will still find them when the project is tested again. This may be because the vulnerability existed within more than one direct dependency, but not all of the affected dependencies could be upgraded. Check the changes in this PR to ensure they won't cause issues with your project. * * * Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs. For more information: https://camo.githubusercontent.com/7f82e62a517211ed476a8243e3490eea20648d146b789657405cbeb40f23cacd/68747470733a2f2f6170692e7365676d656e742e696f2f76312f706978656c2f747261636b3f646174613d65794a33636d6c305a55746c65534936496e4a79576d785a634564485932527954485a7362306c596430645563566734576b4652546e4e434f5545774969776959573576626e6c746233567a535751694f694a6c4d6a646c4d7a4d334d6930784d7a51774c5451334d546b744f44566c4d53316a4e5459344d7a4535595449324d6d55694c434a6c646d567564434936496c425349485a705a58646c5a434973496e42796233426c636e52705a584d694f6e736963484a4a5a434936496d55794e32557a4d7a63794c54457a4e4441744e4463784f5330344e5755784c574d314e6a677a4d546c684d6a59795a534a3966513d3d 🧐 View latest project report 🛠️ Adjust project settings 📚 Read more about Snyk's upgrade and patch logic * * * Learn how to fix vulnerabilities with free interactive lessons: 🦉 Denial of Service (DoS) flyteorg/flytekit ✅ All checks have passed 30/30 successful checks

GitHub

02/07/2023, 5:50 PM

1 new commit pushed to

<https://github.com/flyteorg/flytekit/tree/master|master>

by cosmicBboy

<https://github.com/flyteorg/flytekit/commit/dee48042264933da99917bc18e2653098155702e|dee48042>

- set maximum python version to 3.10 (#1433) flyteorg/flytekit

GitHub

02/07/2023, 5:52 PM

#1436 [Snyk] Security upgrade numpy from 1.21.3 to 1.22.2 Pull request opened by snyk-bot Snyk has created this PR to fix one or more vulnerable packages in the
pip
dependencies of this project. Changes included in this PR • Changes to the following files to upgrade the vulnerable dependencies to a fixed version: • plugins/flytekit-aws-athena/requirements.txt ⚠️ Warning

Copy code

protoc-gen-swagger 0.1.0 requires protobuf, which is not installed.
flyteidl 1.3.0 requires protobuf, which is not installed.
flyteidl 1.3.0 requires googleapis-common-protos, which is not installed.
docker-image-py 0.1.12 requires regex, which is not installed.
binaryornot 0.4.4 requires chardet, which is not installed.

Vulnerabilities that will be fixed By pinning: Some vulnerabilities couldn't be fully fixed and so Snyk will still find them when the project is tested again. This may be because the vulnerability existed within more than one direct dependency, but not all of the affected dependencies could be upgraded. Check the changes in this PR to ensure they won't cause issues with your project. * * * Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs. For more information: https://camo.githubusercontent.com/5e92c84d53854ed56fd27a431e8a523154d0dc9a3ab32e10bc2ad5d3ac68672e/68747470733a2f2f6170692e7365676d656e742e696f2f76312f706978656c2f747261636b3f646174613d65794a33636d6c305a55746c65534936496e4a79576d785a634564485932527954485a7362306c596430645563566734576b4652546e4e434f5545774969776959573576626e6c746233567a535751694f6949795a446b314d6d497a4e5330315a475a6d4c5451314d546774596a4d314e6930334e6d4e694e7a677759574e694e5755694c434a6c646d567564434936496c425349485a705a58646c5a434973496e42796233426c636e52705a584d694f6e736963484a4a5a434936496a4a6b4f545579596a4d314c54566b5a6d59744e4455784f4331694d7a55324c546332593249334f444268593249315a534a3966513d3d 🧐 View latest project report 🛠️ Adjust project settings 📚 Read more about Snyk's upgrade and patch logic * * * Learn how to fix vulnerabilities with free interactive lessons: 🦉 Denial of Service (DoS) flyteorg/flytekit ✅ All checks have passed 30/30 successful checks