GitHub
03/21/2023, 7:10 PMGitHub
03/21/2023, 7:18 PMGitHub
03/21/2023, 8:37 PMGitHub
03/21/2023, 8:57 PMGitHub
03/21/2023, 9:00 PMmap_task
within a task
function body
@task
def t1():
map_task(...)(...)
• When a user tries to define a map_task
at the module-level scope
@map_task
@task
def t1(): ...
or
@task
def t1(): ...
my_map_task = map_task()(t1)
Goal: What should the final outcome look like, ideally?
The error should make it clear that `map_task`s can only be defined in workflow function bodies.
Describe alternatives you've considered
NA
Propose: Link/Inline OR Additional context
No response
Are you sure this issue hasn't been raised already?
☑︎ Yes
Have you read the Code of Conduct?
☑︎ Yes
flyteorg/flyteGitHub
03/21/2023, 9:11 PMGitHub
03/21/2023, 9:12 PM<https://github.com/flyteorg/flyte/tree/master|master>
by wild-endeavor
<https://github.com/flyteorg/flyte/commit/cfbc88900f0c32097ba41de64b574c8dbd4cc2ca|cfbc8890>
- Re-detect Flyte Admin version (#3510)
flyteorg/flyteGitHub
03/21/2023, 9:17 PMGitHub
03/21/2023, 9:21 PM<https://github.com/flyteorg/flytekit/tree/master|master>
by eapolinario
<https://github.com/flyteorg/flytekit/commit/a19043116f507d149c7ff7858ab97100717e51ee|a1904311>
- Make FlyteFile
compatible with Annotated[..., HashMethod]
(#1544)
flyteorg/flytekitGitHub
03/21/2023, 9:24 PM<https://github.com/flyteorg/flyteconsole/tree/master|master>
by jsonporter
<https://github.com/flyteorg/flyteconsole/commit/c968593b2385935ffd2f31b73a67d18decb55bf7|c968593b>
- fix: show correct i/o in details panel (#727)
flyteorg/flyteconsoleGitHub
03/21/2023, 9:52 PM<https://github.com/flyteorg/flyteidl/tree/master|master>
by hamersaw
<https://github.com/flyteorg/flyteidl/commit/bdb76306558b64645f095bc34b32ac487222fc3f|bdb76306>
- Create service for runtime metrics (#367)
flyteorg/flyteidlGitHub
03/21/2023, 9:56 PMGitHub
03/21/2023, 9:58 PM<https://github.com/flyteorg/flyte/tree/master|master>
by eapolinario
<https://github.com/flyteorg/flyte/commit/733e39549228854d85c5610f4e9f94f31b2332f1|733e3954>
- Update Flyte components (#3511)
flyteorg/flyteGitHub
03/21/2023, 9:58 PMGitHub
03/21/2023, 10:13 PM<https://github.com/flyteorg/flytepropeller/tree/master|master>
by hamersaw
<https://github.com/flyteorg/flytepropeller/commit/b4cde1f63873c88800e869f25070176ac068292a|b4cde1f6>
- bumping flyteplugins to 1.0.42 to pick up external cluster configuration for ray (#542)
flyteorg/flytepropellerGitHub
03/21/2023, 10:20 PMthen()
closure, an error is reported server side.
When this is run locally, this error is not reported and the output is successfully generated.
The error message when run on-cloud is the following:
failed at Node[n3-n0]. BindingResolutionError: Error binding Var [wf].[dataset], caused by: failed at Node[n0]. CausedByError: Failed to GetPrevious data from outputDir [<s3://union-oc-production-demo/metadata/propeller/zeryx-demo-development-a9vk7kbf8ptkdcdwqrq6/n0/data/0/outputs.pb>], caused by: path:<s3://union-oc-production-demo/metadata/propeller/zeryx-demo-development-a9vk7kbf8ptkdcdwqrq6/n0/data/0/outputs.pb>: not found
The node graph diagram displays the incorrect sequence of operations:
image▾
pyflyte run workflows/example.py train_mnist_model --n_epoch 1
7. then run the following, pointing to your remote cluster of choice: pyflyte --config ~/.uctl/config.yaml run -p your_project -d development --image zeryx1211/mnist _gpu:latest workflows/example.y train_mnist_model --n_epoch1
8. View the failed workflow and execution
9. Rerun the same with working_example from the gist, repeat steps 6-8
Screenshots
No response
Are you sure this issue hasn't been raised already?
☑︎ Yes
Have you read the Code of Conduct?
☑︎ Yes
flyteorg/flyteGitHub
03/21/2023, 10:54 PMGitHub
03/21/2023, 10:54 PM<https://github.com/flyteorg/flyte/tree/master|master>
by jeevb
<https://github.com/flyteorg/flyte/commit/a68d3b584e015a109f464f7478055f0df4208628|a68d3b58>
- Split flyte-binary services into http and grpc, and some cleanups (#3495)
flyteorg/flyteGitHub
03/21/2023, 11:06 PMGitHub
03/22/2023, 12:17 AMflytectl demo
sandbox was done
• Spin up flytectl demo cluster, confirm the existing migrations were all run, then run new migrations: confirm no alter commands were run.
• Create two new databases, run the old migrations on one, and the new migrations on the second: do a pg_dump and confirm they are identical.
The combination of these tests implies that the new migrations map one to one and onto the old migrations.
The tests were deleted but can be found here: https://gist.github.com/wild-endeavor/cf2336ebf059a087334a8d8e7efe35b1
The pg_dumps are legacy.txt and noop.txt
Tracking Issue
This is the first step in fulfilling flyteorg/flyte#3277
flyteorg/flyteadmin
GitHub Actions: Check Go Generate / Go Generate
GitHub Actions: Docker Build Images / Build Docker Image
✅ 2 other checks have passed
2/4 successful checksGitHub
03/22/2023, 9:31 AMis_true()
and is_false()
it would be nice to have a functionality to check whether None
is returned by a task. This could be useful in conditionals for example.
Goal: What should the final outcome look like, ideally?
The function is_none()
should simply return True
or False
and logically work like
if val is None:
return True
else:
return False
Describe alternatives you've considered
Either returning the string 'None'
or False
and explicitly check for that.
Propose: Link/Inline OR Additional context
No response
Are you sure this issue hasn't been raised already?
☑︎ Yes
Have you read the Code of Conduct?
☑︎ Yes
flyteorg/flyteGitHub
03/22/2023, 10:58 AMflytesnacks/development/human_in_the_loop.wf_dyn
Launch the above workflow and it should show Paused
for gate nodes.
How to fix
Dynamic nodes are included in subWorkflows
under compiledWorkflowClosure
instead of primary
. We only considered priamry
to check the compiled nodes and that's why the ui couldn't understand the gate nodes.
Another issue was that compiled nodes are not only based on execution node id. We should also compare with the metadata.specNodeId
Type
☑︎ Bug Fix
☐ Feature
☐ Plugin
Are all requirements met?
☐ Code completed
☐ Smoke tested
☐ Unit tests added
☐ Code documentation added
☐ Any pending items have an associated Issue
Complete description
How did you fix the bug, make the feature etc. Link to any design docs etc
Tracking Issue
_Remove the '_fixes_' keyword if there will be multiple PRs to fix the linked issue_
fixes flyteorg/flyte#3354
flyteorg/flyteconsole
GitHub Actions: Build & Push FlyteConsole Image
GitHub Actions: Get Release Tag
GitHub Actions: Generate Release
✅ 8 other checks have passed
8/11 successful checksGitHub
03/22/2023, 1:33 PM<https://github.com/flyteorg/flyteadmin/tree/master|master>
by hamersaw
<https://github.com/flyteorg/flyteadmin/commit/83e8694e04f4e75f4b8bf71f32de620fba375c39|83e8694e>
- Tracking reasons time-series (#540)
flyteorg/flyteadminGitHub
03/22/2023, 2:21 PMGitHub
03/22/2023, 2:33 PMGitHub
03/22/2023, 6:33 PMResource(ephemeral_storage=...)
request is relevant when the user writes data to the task pod's disk.
• flytesnacks: https://docs.flyte.org/projects/cookbook/en/latest/auto/deployment/customizing_resources.html
• flytekit:
• https://docs.flyte.org/projects/flytekit/en/latest/generated/flytekit.Resources.html#flytekit-resources
• https://docs.flyte.org/projects/flytekit/en/latest/generated/flytekit.types.file.FlyteFile.html#flytekit.types.file.FlyteFile.download
• https://docs.flyte.org/projects/flytekit/en/latest/generated/flytekit.types.directory.FlyteDirectory.html#flytekit.types.directory.FlyteDirectory: this is missing documentation on the download
method
• https://docs.flyte.org/projects/flytekit/en/latest/generated/flytekit.ExecutionParameters.html#flytekit.ExecutionParameters: this is missing documentation on the working_directory
property
Are you sure this issue hasn't been raised already?
☑︎ Yes
Have you read the Code of Conduct?
☑︎ Yes
flyteorg/flyteGitHub
03/22/2023, 6:58 PM<https://github.com/flyteorg/flyte/tree/master|master>
by wild-endeavor
<https://github.com/flyteorg/flyte/commit/7479b56888d34f2c131e1f16c313ad34bed90484|7479b568>
- Revert "Split flyte-binary services into http and grpc, and some cleanups (#3495)" (#3517)
flyteorg/flyteGitHub
03/22/2023, 7:07 PMGitHub
03/22/2023, 7:09 PMGitHub
03/22/2023, 7:15 PM<https://github.com/flyteorg/flyteconsole/tree/master|master>
by ursucarina
<https://github.com/flyteorg/flyteconsole/commit/35c3b8c992690590327e7d0dcdcc1ff89c729fa0|35c3b8c9>
- chore: task observability (#720)
flyteorg/flyteconsole