GitHub
06/16/2023, 11:40 PMGitHub
06/16/2023, 11:55 PM<https://github.com/flyteorg/flyteconsole/tree/master|master>
by jsonporter
<https://github.com/flyteorg/flyteconsole/commit/4783c39d7c16ae99f6f9b6d9162cf8ceea1fcbc7|4783c39d>
- chore: dparent null in alt paths (#775)
flyteorg/flyteconsoleGitHub
06/17/2023, 3:24 AM<https://github.com/flyteorg/flytesnacks/tree/master|master>
by pingsutw
<https://github.com/flyteorg/flytesnacks/commit/3d29c2d3601b996075fe4b2df425b305b0e3ba63|3d29c2d3>
- Fix "ray: command not found Error" with Kuberay v0.3.0+ (#1002)
flyteorg/flytesnacksGitHub
06/17/2023, 4:24 AMGitHub
06/17/2023, 8:39 PM<https://github.com/flyteorg/flytesnacks/tree/master|master>
by cosmicBboy
<https://github.com/flyteorg/flytesnacks/commit/2a127cf8e3988d0a8a310be5d3c8ec3ca1f2216e|2a127cf8>
- Doc for flyte agent (#971)
flyteorg/flytesnacksGitHub
06/17/2023, 8:39 PMGitHub
06/17/2023, 9:33 PMGitHub
06/19/2023, 5:29 AMflytectl register files -d development -p flytesnacks --archive flyte-package.tgz --version "0.0.8" --k8sServiceAccount flyte-user --assumableIamRole arn:aws:iam::<acount_name>:role/<role_name>
.
I can't launch that workflow, whenever I tried to launch that workflow, they returned this error:
Request failed with status code 500 failed to create workflow in propeller <http://FlyteWorkflow.flyte.lyft.com|FlyteWorkflow.flyte.lyft.com> "atsv5q7rp97m5fdg5kqs" is invalid: [metadata.annotations: Invalid value: "": name part must be non-empty, metadata.annotations: Invalid value: "": name part must consist of alphanumeric characters, '-', '_' or '.', and must start and end with an alphanumeric character (e.g. 'MyName', or 'my.name', or '123-abc', regex used for validation is '([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9]')]
Expected behavior
Launch Workflow successfully
Additional context to reproduce
1. pyflyte init my_project
2. cd my_project
3. pyflyte --pkgs workflows package
4. flytectl register files -d development -p flytesnacks --archive flyte-package.tgz --version "0.0.1" --k8sServiceAccount flyte-user --assumableIamRole arnawsiam:::role/
5. Execution workflow via Flyte admin or flytectl.
Screenshots
https://imgur.com/a/DlOYUG6
Are you sure this issue hasn't been raised already?
☑︎ Yes
Have you read the Code of Conduct?
☑︎ Yes
flyteorg/flyteGitHub
06/19/2023, 9:09 AM@task(task_config=Elastic(...))
one can perform training with torch elastic launch (torchrun
).
This works both locally as well as in a cluster with a kubeflow PyTorchJob
.
When executing a workflow locally, i.e. python workflow.py
, but setting e.g. Elastic(nnodes=2)
, the rendezvous of the workers will timeout because the workers wait for the non-existing workers from the non-existing 2nd node to join.
One would have to set the log level to debug in order to see that torch is waiting for the rendezvous to complete. By default, the workflow appears to not do anything.
I thins PR I add a warning log message that informs the user about this.
Type
☐ Bug Fix
☑︎ Feature
☐ Plugin
Are all requirements met?
☑︎ Code completed
☑︎ Smoke tested
☐ Unit tests added
☐ Code documentation added
☐ Any pending items have an associated Issue
Complete description
I check for an environment variable that is set by the kubeflow training operator. If this is not set but the user set nnodes>1
, the warning is emitted.
One could discuss whether we should just automatically switch to nnodes=1
if the environment variables for distributed training have not been set by the training operator but I found this too intrusive. Warning the user, however, should be done.
Tracking Issue
NA
Follow-up issue
NA
flyteorg/flytekit
✅ All checks have passed
30/30 successful checksGitHub
06/19/2023, 4:01 PMGitHub
06/19/2023, 4:01 PMGitHub
06/19/2023, 4:01 PMGitHub
06/19/2023, 4:29 PMGitHub
06/19/2023, 5:01 PM<https://github.com/flyteorg/flytekit/tree/master|master>
by fg91
<https://github.com/flyteorg/flytekit/commit/68ac1f5371de1dac96df3f4e9df7ed983cd1c0d5|68ac1f53>
- Warn when training locally with nnodes > 1 (#1697)
flyteorg/flytekitGitHub
06/19/2023, 9:06 PMfrom flytekit import task, workflow, ImageSpec
new_flytekit = "git+<https://github.com/flyteorg/flytekit@01f2d63221467face8d35ed0af5308bbb3e66eaa>"
image = ImageSpec(registry="pingsutw", packages=["pandas", new_flytekit], apt_packages=["git"], requirements="requirement.txt")
@task(container_image=image)
def bar():
print("hello")
print("hello")
@workflow
def wf() -> str:
bar()
return "hi"
if __name__ == '__main__':
wf()
Tracking Issue
NA
Follow-up issue
NA
flyteorg/flytekit
✅ All checks have passed
30/30 successful checksGitHub
06/20/2023, 10:39 AMGitHub
06/20/2023, 10:44 AM.github
directories across all repos under Flyte org. If we have a single `.github` repo under flyteorg, we can have all the common .github
files under one roof.
What if we do not do this?
Maintenance
Related component(s)
No response
Are you sure this issue hasn't been raised already?
☑︎ Yes
Have you read the Code of Conduct?
☑︎ Yes
flyteorg/flyteGitHub
06/20/2023, 11:22 AMfully standalone minimal environment for running Flyte
. One is using flytectl demo start
command and other is with flytectl sandbox start
command. Document for demo is here and document for sandbox is here.
Having these 2 different ways for the same thing is confusing for a new comer because of multiple reasons:
1. They don't know which is better or preferred way
2. Which is used by Flyte team internally?
3. Is any approach better or worse than the other?
4. Is there any plan to deprecate one of them? If yes then what's the plan / dates, etc?
This hard to find document says that demo
is lighter / better but there isn't any action or ticket to manage this document hence creating this ticket. Hopefully it is not a duplicate issue.
Are you sure this issue hasn't been raised already?
☑︎ Yes
Have you read the Code of Conduct?
☑︎ Yes
flyteorg/flyteGitHub
06/20/2023, 12:53 PMflytectl demo start
command fails with below error when a developer has Docker Desktop
which allows us to have a Kubernetes
local cluster. Kubernetes API server runs on port 6443
. So we have to either turn off local Kubernetes cluster (not a good option) or allow demo cluster to use some other port.
{"status":"Status: Downloaded newer image for <http://cr.flyte.org/flyteorg/flyte-sandbox-bundled:sha-1ae254f8683699b68ecddc89d775fc5d39cc3d84%22|cr.flyte.org/flyteorg/flyte-sandbox-bundled:sha-1ae254f8683699b68ecddc89d775fc5d39cc3d84">}
🧑🏭 booting Flyte-sandbox container
😬 Something went wrong: Failed to start Sandbox container 🐋, Please check your docker client and try again.
Error: Error response from daemon: driver failed programming external connectivity on endpoint flyte-sandbox (5eaf6609c86e6d818291e2f060e42f8002dfd4e288393da6e01109cfa5a60bd7): listen tcp4 0.0.0.0:6443: bind: address already in use
Requested feature:
Allow demo
cluster to use other port preferrably via config file (config.yaml or config-sandbox.yaml or some other file) so that if a developer has some other container / process using the same port, they can change it and quickly get the demo cluster up and running.
Environment details:
• Docker Desktop with Kubernetes enabled
• Mac with M1 chip
• Discussion about this issue on slack is here
flytectl version
{
"App": "flytectl",
"Build": "0cb2bfa",
"Version": "0.6.39",
"BuildTime": "2023-06-20 14:45:22.664715 +0200 CEST m=+0.016518584"
}
flyteorg/flyteGitHub
06/20/2023, 1:06 PMGitHub
06/20/2023, 4:03 PM_outpu_deck
to entrypoint.py
. we use entrypoint.py
only when we run the workflow remotely.
Type
☑︎ Bug Fix
☐ Feature
☐ Plugin
Are all requirements met?
☑︎ Code completed
☐ Smoke tested
☐ Unit tests added
☐ Code documentation added
☐ Any pending items have an associated Issue
Complete description
^^^
Tracking Issue
https://unionai.slack.com/archives/C02CMUNT4PQ/p1687268923664029
Follow-up issue
NA
flyteorg/flytekit
GitHub Actions: build-plugins (3.8, flytekit-hive)
GitHub Actions: build-plugins (3.8, flytekit-greatexpectations)
GitHub Actions: build-plugins (3.8, flytekit-envd)
GitHub Actions: build-plugins (3.8, flytekit-duckdb)
GitHub Actions: build-plugins (3.8, flytekit-dolt)
GitHub Actions: build-plugins (3.8, flytekit-deck-standard)
GitHub Actions: build-plugins (3.8, flytekit-dbt)
GitHub Actions: build-plugins (3.8, flytekit-data-fsspec)
GitHub Actions: build-plugins (3.8, flytekit-dask)
GitHub Actions: build-plugins (3.8, flytekit-bigquery)
GitHub Actions: build-plugins (3.8, flytekit-aws-sagemaker)
GitHub Actions: build-plugins (3.8, flytekit-aws-batch)
GitHub Actions: build-plugins (3.8, flytekit-aws-athena)
GitHub Actions: build (windows-latest, 3.11)
GitHub Actions: build (windows-latest, 3.9)
GitHub Actions: build (windows-latest, 3.8)
GitHub Actions: build (ubuntu-latest, 3.11)
GitHub Actions: build (ubuntu-latest, 3.10)
GitHub Actions: build (ubuntu-latest, 3.9)
GitHub Actions: build (ubuntu-latest, 3.8)
GitHub Actions: lint
GitHub Actions: Docs Warnings
GitHub Actions: docs
✅ 2 other checks have passed
2/25 successful checksGitHub
06/20/2023, 5:43 PM<https://github.com/flyteorg/flyteidl/tree/master|master>
by wild-endeavor
<https://github.com/flyteorg/flyteidl/commit/283472365c16c0e1e5974a68db98f6620525e90d|28347236>
- Add prefix to data upload (#416)
flyteorg/flyteidlGitHub
06/20/2023, 5:44 PMGitHub
06/20/2023, 6:05 PMnpm
dependencies of this project.
Changes included in this PR
• Changes to the following files to upgrade the vulnerable dependencies to a fixed version:
• package.json
Vulnerabilities that will be fixed
With an upgrade:
(*) Note that the real score may have changed since the PR was raised.
Commit messages
Package name: @semantic-release/npm The new version differs by 156 commits.
• 410b73d Merge pull request #581 from semantic-release/beta
• fc6fd18 Merge branch 'master' of github.com:semantic-release/npm into beta
• 93f2f73 chore(deps): lock file maintenance (#590)
• 3930bf1 chore(deps): update dependency semantic-release to v20.1.3 (#589)
• 4dae1ea build(deps): bump webpack from 5.75.0 to 5.76.1
• 0d914cb chore(deps): update dependency sinon to v15.0.2 (#587)
• dff7f90 chore(deps): update dependency dockerode to v3.3.5 (#586)
• c6e74d4 chore(deps): lock file maintenance (#585)
• c434c47 Merge pull request #582 from semantic-release/provenance
• a6aad6f ci(provenance): enabled provenance for publishing
• 5fa2115 chore(deps): update dependency semantic-release to v20.1.1 (#584)
• 9ad8402 chore(deps): lock file maintenance (#583)
• ed0b20f Merge branch 'master' of github.com:semantic-release/npm into beta
• 23eb1e3 chore(deps): lock file maintenance (#580)
• 3266f40 ci(release): configured the release job to use the latest lts version of node
• 725f044 chore(deps): updated semantic-release to the latest stable version
• 31bc1f3 Merge pull request #579 from semantic-release/npm-9
• 2a79f80 fix(deps): upgraded npm to v9
• c46c466 chore(deps): lock file maintenance (#573)
• f450a28 chore(deps): lock file maintenance (#572)
• 2b8ac2e Merge pull request #567 from semantic-release/sr-peer
• 1d0ed0f chore(deps): update dependency semantic-release to v20.1.0 (#571)
• 22e70ad feat(semantic-release-peer): raised the minimum peer requirement to the first version that supports loading esm plugins
• bd05849 chore(deps): update dependency semantic-release to v20.0.4 (#570)
See the full diff
Package name: jest The new version differs by 250 commits.
• be16e47 v27.0.0
• 63102ec chore: update changelog for release
• 564694a docs(blog): Jest 27 blog post (#11131)
• b68d91b feat(pretty-print): add option printBasicPrototype
(#11441)
• 2226742 chore: minor simplify format results error (#11432)
• 78eb25d chore: remove needless assign (#11433)
• 696c455 chore: update lockfile after publish
• e2eb9ae v27.0.0-next.11
• 3b253f8 Wait for closed resources to actually close before detecting open handles (#11429)
• 27bee72 fix: run GC before collecting open handles (#11278)
• 50451df feat: use fallback if prettier not found (#11400)
• 150dbd8 chore: update lockfile after publish
• 6f44529 v27.0.0-next.10
• cbcec7d Upgrade fsevents in jest-haste-map (#11428)
• 9633a26 feat: support reporters written in ESM (#11427)
• 59f42d8 fix: do not cache modules that throw during evaluation (#11263)
• 57e32e9 Detect open hand…
flyteorg/flyteconsole
GitHub Actions: Build & Push FlyteConsole Image
GitHub Actions: Get Release Tag
GitHub Actions: Generate Release
GitHub Actions: unit_tests_with_coverage
GitHub Actions: lint_project
DCO: DCO
✅ 3 other checks have passed
3/9 successful checksGitHub
06/20/2023, 6:09 PMimage▾
image▾
GitHub
06/20/2023, 8:26 PMGitHub
06/20/2023, 10:16 PMGitHub
06/21/2023, 5:16 AMBigQurey.job
Fixed https://flyte-org.slack.com/archives/CP2HDHKE1/p1683133259253529
Type
☑︎ Bug Fix
☐ Feature
☐ Plugin
Are all requirements met?
☑︎ Code completed
☑︎ Smoke tested
☐ Unit tests added
☐ Code documentation added
☐ Any pending items have an associated Issue
Complete description
^^^
Tracking Issue
https://flyte-org.slack.com/archives/CP2HDHKE1/p1683133259253529
Follow-up issue
NA
flyteorg/flytekit
✅ All checks have passed
30/30 successful checksGitHub
06/21/2023, 6:24 AMGitHub
06/21/2023, 3:51 PM<https://github.com/flyteorg/flyteconsole/tree/master|master>
by ursucarina
<https://github.com/flyteorg/flyteconsole/commit/4b579560d4d78b0b919a3dc62949850405d9f10d|4b579560>
- fix: launch plan icon (#777)
flyteorg/flyteconsole