GitHub
01/10/2023, 11:55 PMGitHub
01/11/2023, 4:47 AM%{HOST_GATEWAY_IP}%
template variable.
flyteorg/flyte
GitHub Actions: trigger-sandbox-lite-build
GitHub Actions: trigger-single-binary-build
GitHub Actions: compile
GitHub Actions: Functional test
✅ 6 other checks have passed
6/10 successful checksGitHub
01/11/2023, 12:51 PMSdkBindingData.get()
to get the inner value in the run task context (java example, scala example), but at the same time these changes allow to recover the attributes by name in the workflow site (java example, scala example).
Tracking Issue
• flyteorg/flyte#3250
• flyteorg/flyte#3251
Follow-up issue
• flyteorg/flyte#3252
flyteorg/flytekit-java
✅ All checks have passed
3/3 successful checksGitHub
01/11/2023, 7:15 PMimage▾
GitHub
01/11/2023, 9:21 PMGitHub
01/11/2023, 9:50 PM<https://github.com/flyteorg/flytepropeller/tree/master|master>
by hamersaw
<https://github.com/flyteorg/flytepropeller/commit/ce57dbf15274a77c7037f367bda738e6ce33b453|ce57dbf1>
- use different perm (#517)
flyteorg/flytepropellerGitHub
01/11/2023, 10:11 PM<https://github.com/flyteorg/flyte/tree/master|master>
by eapolinario
<https://github.com/flyteorg/flyte/commit/782fe745452a07294db47f6ff782e5c559ac3c92|782fe745>
- Add dask operator (#3145)
flyteorg/flyteGitHub
01/11/2023, 10:13 PMGitHub
01/11/2023, 10:46 PMGitHub
01/11/2023, 10:48 PM<https://github.com/flyteorg/flyte/tree/master|master>
by jeevb
<https://github.com/flyteorg/flyte/commit/c674bfa31ef0d262112b083950c4425480a18bc3|c674bfa3>
- Add Kubernetes objects for dev mode with single-binary sandbox (#3228)
flyteorg/flyteGitHub
01/11/2023, 10:50 PMGitHub
01/11/2023, 10:55 PM<https://github.com/flyteorg/flyte/tree/master|master>
by eapolinario
<https://github.com/flyteorg/flyte/commit/c19c282df236b4f7917a64f7928eeaa121723b56|c19c282d>
- Changelog for 1.3 (#3209)
flyteorg/flyteGitHub
01/11/2023, 11:12 PMGitHub
01/11/2023, 11:14 PM<https://github.com/flyteorg/flytectl/tree/master|master>
by wild-endeavor
<https://github.com/flyteorg/flytectl/commit/b0d98931bf8780c193f9fc101242309058138795|b0d98931>
- Change extra host to host-gateway (#380)
flyteorg/flytectlGitHub
01/11/2023, 11:33 PM<https://github.com/flyteorg/flyte/tree/master|master>
by eapolinario
<https://github.com/flyteorg/flyte/commit/f69fb09ca189e8bf57e1a6a12db168274f640d15|f69fb09c>
- Update Flyte components (#3230)
flyteorg/flyteGitHub
01/11/2023, 11:37 PMGitHub
01/11/2023, 11:37 PM<https://github.com/flyteorg/homebrew-tap/tree/main|main>
by flyte-bot
<https://github.com/flyteorg/homebrew-tap/commit/6d942c75f7f2a62d6180fb5d16c454d6e28c8d54|6d942c75>
- Brew formula update for flytectl version v0.6.26
flyteorg/homebrew-tapGitHub
01/11/2023, 11:57 PMGitHub
01/11/2023, 11:57 PMFlyteRemote
however, though only a limited set of types can be passed in.
Notes
There are a couple things to point out with this release.
Caching on Structured Dataset
Please take a look at the flytekit PR notes for more information but if you haven't bumped Propeller to version v1.1.36 (aka Flyte v1.2) or later, tasks that take as input a dataframe or a structured dataset type, that are cached, will trigger a cache miss. If you've upgraded Propeller, it will not.
Flytekit Remote Types
In the FlyteRemote
experience, fetched tasks and workflows will now be based on their respective "spec" classes in the IDL (task/wf) rather than the template. The spec messages are a superset of the template messages so no information is lost. If you have code that was accessing elements of the templates directly however, these will need to be updated.
Usage Overview
Databricks
Please refer to the documentation for setting up Databricks.
Databricks is a subclass of the Spark task configuration so you'll be able to use the new class in place of the more general Spark
configuration.
from flytekitplugins.spark import Databricks
@task(
task_config=Databricks(
spark_conf={
"spark.driver.memory": "1000M",
"spark.executor.memory": "1000M",
"spark.executor.cores": "1",
"spark.executor.instances": "2",
"spark.driver.cores": "1",
},
databricks_conf={
"run_name": "flytekit databricks plugin example",
"new_cluster": {
"spark_version": "11.0.x-scala2.12",
"node_type_id": "r3.xlarge",
"aws_attributes": {
"availability": "ON_DEMAND",
"instance_profile_arn": "arn:aws:iam::1237657460:instance-profile/databricks-s3-role",
},
"num_workers": 4,
},
"timeout_seconds": 3600,
"max_retries": 1,
}
))
New Deployment Type
A couple releases ago, we introduced a new Flyte executable that combined all the functionality of Flyte's backend into one command. This simplifies the deployment in that only one image needs to run now. This approach is now our recommended way for new comers to the project to install and administer Flyte and there is a new Helm chart also. Documentation has been updated to take this into account. For new installations of Flyte, clusters that do not already have the flyte-core
or flyte
charts installed, users can
helm install flyte-server flyteorg/flyte-binary --namespace flyte --values your_values.yaml
New local demo environment
Users may have noticed that the environment provided by flytectl demo start
has also been updated to use this new style of deployment, and internally now installs this new Helm chart. The demo cluster now also exposes an internal docker registry on port 30000
. That is, with the new demo cluster up, you can tag and push to localhost:30000/yourimage:tag123
and the image will be accessible to the internal Docker daemon. The web interface is still at localhost:30080
, Postgres has been moved to 30001
and the Minio API (not web server) has been moved to 30002
.
Human-in-the-loop Workflows
Users can now insert sleeps, approval, and input requests, in the form of gate nodes. Check out one of our earlier issues for background information.
from flytekit import wait_for_input, approve, sleep
@workflow
def mainwf(a: int):
x = t1(a=a)
s1 = wait_for_input("signal-name", timeout=timedelta(hours=1), expected_type=bool)
s2 = wait_for_input("signal name 2", timeout=timedelta(hours=2), expected_type=int)
z = t1(a=5)
zzz = sleep(timedelta(seconds=10))
y = t2(a=s2)
q = t2(a=approve(y, "approvalfory", timeout=timedelta(hours=2)))
x >> s1
s1 >> z
z >> zzz
...
These also work inside @dynamic
tasks. Interacting with signals from flytekit's remote experience looks like
from flytekit.remote.remote import FlyteRemote
from flytekit.configuration import Config
r = FlyteRemote(
Config.auto(config_file="/Users/ytong/.flyte/dev.yaml"),
default_project="flytesnacks",
default_domain="development",
)
r.list_signals("atc526g94gmlg4w65dth")
r.set_signal("signal-name", "execidabc123", True)
Overwritten Cached Values on Execution
Users can now configure workflow execution to overwrite the cache. Each task in the workflow execution, regardless of previous cache status, will execute and write cached values - overwritting previous values if necessary. This allows previously corrupted cache values to be corrected without the tedious process of incrementing the cache_version
and re-registering Flyte workflows / tasks.
Support for Dask
Users will be able to spawn Dask ephemeral clusters as part of their workflows, similar to the support for Ray and Spark.
Looking Ahead
In the coming release, we are focusing on...
1. Out of core plugin: Make backend plugin scalable and easy to author. No need of code generation, using tools that MLEs and Data Scientists are not accustomed to using.
2. Performance Observability: We have made great progress on exposing both finer-grained runtime metrics and Flytes orchestration metrics. This is important to better understand workflow evaluation performance and mitigate inefficiencies thereof.
flyteorg/flyteGitHub
01/11/2023, 11:58 PMGitHub
01/12/2023, 1:12 AMGitHub
01/12/2023, 1:23 AMWait()
call on the errgroup.WithContext
doesn't return on first error. It waits til all the go-routines finish, and then returns the first error. I only cancels the context.
This means that all the goroutines have to monitor the context and terminate itself on context cancel. This is missing for the clusterresourcemanager -
flyte/cmd/single/start.go
Line 49 in </flyteorg/flyte/commit/f69fb09ca189e8bf57e1a6a12db168274f640d15|f69fb09>
.
This means that if Flyte Admin fails, the binary itself will proceed as normal, just without admin, since the cluster resource manager runs in perpetuity, which is how we found this issue.
Expected behavior
The whole process should fail if Admin fails. Add context monitoring logic to cluster resource controller and go through the other errgroups to make sure everything launched is also doing the right thing.
Additional context to reproduce
No response
Screenshots
No response
Are you sure this issue hasn't been raised already?
☑︎ Yes
Have you read the Code of Conduct?
☑︎ Yes
flyteorg/flyteGitHub
01/12/2023, 2:16 AM<https://github.com/flyteorg/flytekit/tree/master|master>
by eapolinario
<https://github.com/flyteorg/flytekit/commit/531db3cbadda1e19a42b307a989106db35a836c0|531db3cb>
- Add dask plugin #patch (#1366)
flyteorg/flytekitGitHub
01/12/2023, 2:16 AMdask
backend plugin (flytekitplugins-dask
)
Type
☐ Bug Fix
☑︎ Feature
☐ Plugin
Are all requirements met?
☑︎ Code completed
☑︎ Smoke tested
☑︎ Unit tests added
☑︎ Code documentation added
☑︎ Any pending items have an associated Issue
Complete description
This adds the Python side of the dask
backed plugin (links to the related PRs can be found in the linked issue below).
TODO before this can be merged:
After flyteorg/flyteidl#339 is merged:
☑︎ Make sure flytekit
version is up to date in plugins/flytekit-dask/setup.py
(needs to have buf
protobufs)
☑︎ Make sure flyteidl
version is up to date in plugins/flytekit-dask/setup.py
(needs to support dask
IDL)
☑︎ Add plugins/flytekit-dask/requirements.in
and lock (on Linux)
☑︎ Add dask[distributed]
to <http://doc-requirements.in|doc-requirements.in>
and re-lock (on Linux)
☐ Check to make sure documentation looks ok
Tracking Issue
flyteorg/flyte#427
Follow-up issue
NA
flyteorg/flytekit
✅ All checks have passed
30/30 successful checksGitHub
01/12/2023, 8:51 AMretry
package dependency in requirements.txt
for flytekit. The original unmaintained package has a dependency on the py
package, which has a CVE with no fix version available:
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-42969
Type
☑︎ Bug Fix
☐ Feature
☐ Plugin
Are all requirements met?
☑︎ Code completed
☑︎ Smoke tested
☐ Unit tests added
☐ Code documentation added
☐ Any pending items have an associated Issue
Complete description
In this PR, it is replaced with the recent release of the retry2
package version 0.9.5
, which is a fork of the original retry
package:
https://pypi.org/project/retry2/
Tracking Issue
flyteorg/flyte#3052
flyteorg/flytekitGitHub
01/12/2023, 9:49 AMGitHub
01/12/2023, 5:46 PMGitHub
01/12/2023, 6:57 PMpyflyte
the -d
switch was overloaded, used for both domain
and the destination-dir
that flytekit will extract user code into when using the fast-register feature. The latter has been updated to -D
. This may require some users to change commands in CI/CD if the short form of these switches is used.
Protobuf Support
In May of last year Google announced a breaking change to the support for protobuf in python. Starting with this release we moved to this new version of the protobuf tooling, specifically, this affects some of the flytekit dependencies, including grpcio
and grpcio-status
.
Some flytekit dependencies, namely tensorflow
haven't upgraded yet to the new protobuf version yet (more details in tensorflow/tensorflow#53234), so for the time being, flytekit v1.2.x will be maintained for bug fixes (and possibly features as well).
* * *
Additional
Please see Flyte 1.3 Release Notes for additional changes (Databricks, Human-in-the-loop Tasks, Dask, and FlyteRemote updates) and usage.
* * *
What's Changed
• Sanitize query template input in sqlite task by @eapolinario in #1359
• TypeTransformer for reading and writing from TensorFlowRecord format by @ryankarlos in #1240
• Bump numpy from 1.21.6 to 1.22.0 in /tests/flytekit/integration/remote/mock_flyte_repo/workflows by @dependabot in #1170
• Update ray plugin dependency by @pingsutw in #1361
• Bring in buf protobufs by @eapolinario in #1363
• Set default format of structured dataset to empty by @pingsutw in #1159
• Adds CLI reference for pyflyte by @samhita-alla in #1362
• Update plugins requirements files versions by @eapolinario in #1365
• Signaling by @wild-endeavor in #1133
• Adding created and updated at to ExecutionClosure model by @wild-endeavor in #1371
• Restrict numpy due to deprecated aliases by @eapolinario in #1376
• Add Databricks config to Spark Job by @pingsutw in #1358
• Add overwrite_cache option the to calls of remote and local executions by @hfurkanvural in #1375
• Remove project/domain from being overridden with execution values in serialized context by @wild-endeavor in #1378
• Use TaskSpec instead of TaskTemplate for fetch_task and avoid network when loading module by @kumare3 in #1348
• Register Databricks config by @pingsutw in #1379
• PodSpec should not require primary_container name by @kumare3 in #1380
• fix(pyflyte register): change -d to -D for --destination-dir as -d is already used for --domain by @mcloney-ddm in #1381
• [Snyk] Security upgrade torch from 1.12.1 to 1.13.1 by @snyk-bot in #1374
• Handle Optional[FlyteFile] in Dataclass type transformer by @eapolinario in #1393
• Update flyte deck plugin's dependency by @pingsutw in #1395
• add FastSerializationSettings to docs by @cosmicBboy in #1386
• Added more pod tests and an example pod task by @kumare3 in #1382
• Convert default value of dict to json string in pyflyte run by @pingsutw in #1399
• docs: update register help, non-fast version is supported by @pbrogan12 in #1402
• Update log level for structured dataset by @pingsutw in #1394
• Add Niels to code owners by @pingsutw in #1404
• Signal use by @wild-endeavor in #1398
• User Documentation Proposal by @pingsutw in #1200
• Add support MLFlow plugin by @kumare3 in #1274
• fix remote API reference by @cosmicBboy in #1405
• Read structured dataset from a folder by @pingsutw in #1406
• Update default config to work out-of-the-box with flytectl demo by @cosmicBboy in #1384
• Add dask plugin #patch by @bstadlbauer in #1366
New Contributors
• @hfurkanvural made their first contribution in #1375
• @mcloney-ddm made their first contribution in #1381
• @pbrogan12 made their first contribution in #1402
Full Changelog: v1.2.5...v1.3.0
flyteorg/flytekitGitHub
01/12/2023, 10:40 PMGitHub
01/13/2023, 12:40 PM