GitHub
03/20/2023, 3:13 PMGitHub
03/20/2023, 3:21 PMGitHub
03/20/2023, 4:19 PM<https://github.com/flyteorg/flyteplugins/tree/master|master>
by hamersaw
<https://github.com/flyteorg/flyteplugins/commit/0a681cd1a9e2c6586e6408532767539be65737f4|0a681cd1>
- feat: add support for inter-cluster communication for Ray plugin (#321)
flyteorg/flytepluginsGitHub
03/20/2023, 4:20 PMGitHub
03/20/2023, 4:25 PMFailed
state, the sub-workflows can stay in a Queued
state in the UI (cf. screenshot)
Those are impossible to stop as they are technically not running anymore. Here is the list of pods in the namespace associated with this project.
❯ k get pods
NAME READY STATUS RESTARTS AGE
ajrs5vwphfsx2sj9ldmq-n0-0 0/1 Completed 0 175m
ajrs5vwphfsx2sj9ldmq-n1-0 0/1 Completed 0 175m
ajrs5vwphfsx2sj9ldmq-n2-0 0/1 Completed 0 175m
ajrs5vwphfsx2sj9ldmq-n3-0 0/1 Completed 0 173m
aq8pt9rjwpssjw7whvsq-n0-0 0/1 Completed 0 24h
aq8pt9rjwpssjw7whvsq-n1-0 0/1 Completed 0 24h
avp8s7qxhgtpw7ckz2gv-n0-0 0/1 Completed 0 10h
avp8s7qxhgtpw7ckz2gv-n1-0 0/1 Completed 0 10h
azc4p4n9g9k8q4bdg767-n0-0 0/1 Completed 0 10h
azc4p4n9g9k8q4bdg767-n1-0 0/1 Completed 0 10h
Discussion in Slack with @hamersaw https://flyte-org.slack.com/archives/CP2HDHKE1/p1679075564278029
Expected behavior
The UI should also be showing the sub-workflows in Failed
stage or Aborted
stage I guess depending on what the real states of the sub-worklows are.
Additional context to reproduce
No response
Screenshots
image▾
GitHub
03/20/2023, 4:33 PMGitHub
03/20/2023, 5:02 PMGitHub
03/20/2023, 5:21 PM<https://github.com/flyteorg/datacatalog/tree/master|master>
by hamersaw
<https://github.com/flyteorg/datacatalog/commit/25c0a10b18ba0cdd72867168a5e5eca8f22d9b96|25c0a10b>
- replaced github.com/satori/go.uuid with github.com/gofrs/uuid (#101)
flyteorg/datacatalogGitHub
03/20/2023, 5:35 PMGitHub
03/20/2023, 6:10 PM<https://github.com/flyteorg/flytekit/tree/master|master>
by wild-endeavor
<https://github.com/flyteorg/flytekit/commit/93995e29dcd37317e91ebcfb3019a4ffed0e82de|93995e29>
- Update the pypi wait (#1554)
flyteorg/flytekitGitHub
03/20/2023, 6:31 PMGitHub
03/20/2023, 10:22 PMScreen Shot 2021-04-26 at 4 38 11 PM▾
GitHub
03/20/2023, 11:52 PM<https://github.com/flyteorg/flytekit/tree/master|master>
by wild-endeavor
<https://github.com/flyteorg/flytekit/commit/98e74c273a61fd767ea331f5f0812d22a4f7aa57|98e74c27>
- Stream Directories and Files using Flyte (#1512)
flyteorg/flytekitGitHub
03/20/2023, 11:52 PM@task
def copy_file(ff: FlyteFile) -> FlyteFile:
new_file = FlyteFile.new_remote_file(ff.remote_path)
with ff.open("r", cache_type="simplecache", cache_options={}) as r:
with new_file.open("w") as w:
w.write(r.read())
return new_file
Example: Accept a flyteDirectory and walk it, while copying the directory file by file to another Flyte directory
@task
def process_folder(fd: FlyteDirectory) -> FlyteDirectory:
out_fd = FlyteDirectory.new_remote()
for base, x in fd.crawl():
src = os.path.join(base, x)
out_file = out_fd.new_file(x)
with FlyteFile(src).open("rb") as f:
with out_file.open("wb") as o:
o.write(f.read())
return out_fd
Alternatively, Files can always be handled by fsspec streaming
@task
def copy_file(ff: FlyteFile) -> FlyteFile:
new_file = FlyteFile.new_remote_file(ff.name)
fs = flytekit.get_filesystem()
with fs.open(f"readahead::{ff.remote_path}", "rb", readahead={}) as r:
with new_file.open("wb") as w:
w.write(r.read())
return new_file
Type
☐ Bug Fix
☑︎ Feature
☐ Plugin
Are all requirements met?
☑︎ Code completed
☑︎ Smoke tested
☑︎ Unit tests added
☑︎ Code documentation added
☑︎ Any pending items have an associated Issue
Complete description
Other minor changes:
• Get rid of sqlite data download dependency by adding it to this repo
• Make the base to_html function not abstract.
Tracking Issue
flyteorg/flyte#3334
flyteorg/flytekit
Codecov: 41.48% of diff hit (target 69.70%)
Codecov: 69.31% (-0.39%) compared to 93995e2
✅ 28 other checks have passed
28/30 successful checksGitHub
03/20/2023, 11:54 PMGitHub
03/21/2023, 12:16 AMGitHub
03/21/2023, 7:14 AM<https://github.com/flyteorg/flyteadmin/tree/master|master>
by hamersaw
<https://github.com/flyteorg/flyteadmin/commit/dea5b2a8b21d1f6edd7fc4b4c5288cbef102075d|dea5b2a8>
- bumping fosite version (#538)
flyteorg/flyteadminGitHub
03/21/2023, 7:15 AM<https://github.com/flyteorg/flyteidl/tree/master|master>
by hamersaw
<https://github.com/flyteorg/flyteidl/commit/0f18851e644b1ee51fcf6d4fac855a65adcc34f8|0f18851e>
- Add Reasons field to TaskExecutionClosure to track time-series of reasons (#382)
flyteorg/flyteidlGitHub
03/21/2023, 7:19 AMGitHub
03/21/2023, 7:25 AMReasons
field on the TaskExecutionClosure
to track a reason time-series. This will be very useful in improving visualizations by tracking the reason for task phase transitions and updates.
Type
☐ Bug Fix
☑︎ Feature
☐ Plugin
Are all requirements met?
☑︎ Code completed
☑︎ Smoke tested
☐ Unit tests added
☐ Code documentation added
☐ Any pending items have an associated Issue
Complete description
^^^
Tracking Issue
flyteorg/flyte#3501
Follow-up issue
NA
flyteorg/flyteadmin
GitHub Actions: Unit Tests / Run Unit Test
GitHub Actions: Check Go Generate / Go Generate
GitHub Actions: Lint / Run Lint
GitHub Actions: Docker Build Images / Build Docker Image
✅ 2 other checks have passed
2/6 successful checksGitHub
03/21/2023, 7:52 AMPods
marked as a retryable failure are never cleaned up. This PR only fast-fails when aborting tasks in a success phase, to ensure that all resources are correctly cleaned up at the risk of unnecessary API calls (ex. object does not exist, etc).
Type
☑︎ Bug Fix
☐ Feature
☐ Plugin
Are all requirements met?
☑︎ Code completed
☑︎ Smoke tested
☐ Unit tests added
☐ Code documentation added
☐ Any pending items have an associated Issue
Complete description
Currently, the TaskHandler
fast-fails when aborting resources in a terminal phase. This causes issues where resources fail to cleanup when the task is reported as failed (ie. retryable failure or permanent failure) but the resource remains running.
One such scenario is handling Pods
in the "ImagePullBackoff" state. We mark these as failed, but because of the fast-fail during abort these k8s resources are never deleted.
This may occur within other plugins as well. To be safe, we may want to attempt to abort all task phases other than Success
.
Tracking Issue
fixes flyteorg/flyte#3239
Follow-up issue
NA
flyteorg/flytepropeller
GitHub Actions: Goreleaser
GitHub Actions: Build & Push Flytepropeller Image
GitHub Actions: Bump Version
✅ 11 other checks have passed
11/14 successful checksGitHub
03/21/2023, 7:57 AMGitHub
03/21/2023, 9:52 AMscopes
are ignored and the ones defined in the admin are always used.
Type
☑︎ Bug Fix
☐ Feature
☐ Plugin
Are all requirements met?
☑︎ Code completed
☐ Smoke tested
☑︎ Unit tests added
☑︎ Code documentation added
☐ Any pending items have an associated Issue
Complete description
If scopes
are locally defined, use those for auth. This mirrors the reference implementation here.
Tracking Issue
flyteorg/flyte#3486
Follow-up issue
flyteorg/flytekit
Codecov: 69.60% (-0.10%) compared to 34f80ba
✅ 29 other checks have passed
29/30 successful checksGitHub
03/21/2023, 10:42 AMretry
library dependency declared by flytekit: https://github.com/flyteorg/flytekit/blob/3b5d15acb5389d99b86ba342a77e1cab5ce8546d/setup.py#L68
This library is not used anywhere within flytekit that I can find searching for import retry
or from retry
, except in the sagemaker plugin: https://github.com/flyteorg/flytekit/blob/3b5d15acb5389d99b86ba342a77e1cab5ce8546d/plugins/flytekit-aws-sagemaker/flytekitplugins/awssagemaker/distributed_training.py#L8
The dependency could be moved to the sagemaker plugin's requirements so only users of that plugin need it installed. Though, given the unmaintained state of the retry library, I'd advise against that and suggest to look for a maintained alternative (reretry? https://github.com/leshchenko1979/reretry)
What if we do not do this?
All users of flytekit end up installing an unmaintained library (see invl/retry#36, not updated since 2016), which also ends up bringing with it unnecessary dependency on py
which itself has been the source of lots of security alerting pytest-dev/py#287
Related component(s)
flytekit
Are you sure this issue hasn't been raised already?
☑︎ Yes
Have you read the Code of Conduct?
☑︎ Yes
flyteorg/flyteGitHub
03/21/2023, 4:58 PMpyflyte run --remote -p nonexistent_project ...
the error message that's produced contains a whole lot of information about the task while not reporting what is actually wrong, which is that nonexistent_project
doesn't exist in the remote Flyte cluster.
See here for full context on what the current error message is.
Goal: What should the final outcome look like, ideally?
Consider specifying an invalid project or domain as simple input errors and should fail quickly with explicit messaging i.e. "Project 'flytesnacks' not found". A full stack trace is hard to mentally parse for such a simple input error even if it did actually say why it failed in the message.
Describe alternatives you've considered
NA
Propose: Link/Inline OR Additional context
No response
Are you sure this issue hasn't been raised already?
☑︎ Yes
Have you read the Code of Conduct?
☑︎ Yes
flyteorg/flyteGitHub
03/21/2023, 5:01 PMfrom flytekit import task, workflow
@task
def t1(x: int) -> int:
return x + 1
@task
def t2(x: str) -> str:
return str(x)
@workflow
def wf(x: int) -> str:
out1 = t1(x=x)
return t2(x=out1)
if __name__ == "__main__":
print(wf(x=5))
The error:
File "/Users/nielsbantilan/git/flytekit/flytekit/core/type_engine.py", line 185, in to_python_value
raise TypeTransformerFailedError(f"Cannot convert literal {lv} to {self._type}")
flytekit.core.type_engine.TypeTransformerFailedError: Cannot convert literal scalar {
primitive {
integer: 6
}
}
to <class 'str'>
Goal: What should the final outcome look like, ideally?
When there's a type mismatch error, identify where exactly in the workflow it's happening:
• the task where the conversion failed
• the input that fails conversion
Describe alternatives you've considered
NA
Propose: Link/Inline OR Additional context
No response
Are you sure this issue hasn't been raised already?
☑︎ Yes
Have you read the Code of Conduct?
☑︎ Yes
flyteorg/flyteGitHub
03/21/2023, 5:44 PMactions/setup-node@v3
to cache dependencies.
Tracking Issue
N/A
Follow-up issue
NA
flyteorg/flyteconsole
GitHub Actions: Build & Push FlyteConsole Image
GitHub Actions: Get Release Tag
GitHub Actions: Generate Release
✅ 8 other checks have passed
8/11 successful checksGitHub
03/21/2023, 5:45 PMimage▾
GitHub
03/21/2023, 6:15 PM<https://github.com/flyteorg/flyteconsole/tree/master|master>
by eapolinario
<https://github.com/flyteorg/flyteconsole/commit/d0d383e6e0036a9596222e37dd6e0e014dd8e99b|d0d383e6>
- Install deps directly in checks.yml (#728)
flyteorg/flyteconsoleGitHub
03/21/2023, 7:10 PM<https://github.com/flyteorg/flyte/tree/master|master>
by hamersaw
<https://github.com/flyteorg/flyte/commit/d30e0b5943411db51bb62df4a1ee26f75e9fc12f|d30e0b59>
- [RFC] Performance Benchmarking (#2995)
flyteorg/flyte