GitHub
04/23/2023, 12:09 PMGitHub
04/23/2023, 7:27 PMfrom flytekitplugins.kfpytorch import Elastic
@task(
task_config=Elastic(
replicas=4,
nproc_per_node=4,
...
),
...
)
def train(...):
...
Type
☐ Bug Fix
☑︎ Feature
☐ Plugin
Are all requirements met?
☑︎ Code completed
☑︎ Smoke tested
☑︎ Unit tests added
☑︎ Code documentation added
☑︎ Any pending items have an associated Issue
Complete description
See flyteorg/flyte#3614 for motivation and a more detailed description.
Tracking Issue
flyteorg/flyte#3614
Follow-up issue
flyteorg/flytekit
✅ All checks have passed
30/30 successful checksGitHub
04/24/2023, 2:42 AM<https://github.com/flyteorg/flyteidl/tree/master|master>
by eapolinario
<https://github.com/flyteorg/flyteidl/commit/6a7b31438821190be535cc6bbcb6f2939e5a056a|6a7b3143>
- Feat: Add ElasticConfig
message type for torch elastic training (#394)
flyteorg/flyteidlGitHub
04/24/2023, 2:43 AMElasticConfig
message type for torch elastic training (#394)
flyteorg/flyteidlGitHub
04/24/2023, 3:53 AMimport pandas as pd
from flytekit import task, workflow, logger
from flytekit.deck.deck import log_on_deck
from flytekit.image_spec import ImageSpec
new_flytekit = "git+<https://github.com/flyteorg/flytekit@aab764d73871da58847206d868ca94e60242a5b3>"
image_spec = ImageSpec(packages=[new_flytekit],
apt_packages=["git"],
registry="pingsutw",
env={"FLYTE_SDK_LOGGING_LEVEL": "10"})
@task(disable_deck=False, container_image=image_spec)
@log_on_deck
def t1():
print("hello")
print("flyte")
logger.error("error")
logger.warning("warning")
logger.debug("debug")
print("pandas dataframe")
df = pd.DataFrame({"Name": ["Tom", "Joseph"], "Age": [1, 22]})
print(df)
return
@workflow
def wf():
t1()
if __name__ == '__main__':
wf()
image▾
GitHub
04/24/2023, 4:41 AMGitHub
04/24/2023, 6:00 AM<https://github.com/flyteorg/flyteplugins/tree/master|master>
by kumare3
<https://github.com/flyteorg/flyteplugins/commit/acc831a734195aba489c9b05a607a175cdb8d2c7|acc831a7>
- Feat: Configure elastic training in pytorch plugin (#343)
flyteorg/flytepluginsGitHub
04/24/2023, 6:32 AMGitHub
04/24/2023, 1:50 PMpaused --> ${some phase}
. However, if the user refreshes the page the phase is correctly labeled.
The correct behavior is that the phase displayed should always be up-to-date without a page refresh.
Expected behavior
Gate nodes should update without a page refresh after a user interacts and lets the node proceed.
Additional context to reproduce
No response
Screenshots
No response
Are you sure this issue hasn't been raised already?
☑︎ Yes
Have you read the Code of Conduct?
☑︎ Yes
flyteorg/flyteGitHub
04/24/2023, 4:00 PMGitHub
04/24/2023, 5:32 PMoperationId
for the execution and use the hover-index to render that operation as bold.
Screenshot 2023-04-24 at 10 30 17 AM▾
Screenshot 2023-04-24 at 10 30 17 AM▾
GitHub
04/24/2023, 5:41 PMGitHub
04/24/2023, 8:24 PMrecord_outputs
, although serializing into json might difficult. See https://github.com/flyteorg/flytekit/blob/master/plugins/flytekit-papermill/flytekitplugins/papermill/task.py#L294.
flyteorg/flyteGitHub
04/24/2023, 10:03 PMGitHub
04/24/2023, 11:29 PMpyflyte run my_workflow --my-json path/file.json
OR
pyflyte run my_workflow --my-json '{"x":"y"}'
Type
☐ Bug Fix
☑︎ Feature
☐ Plugin
Are all requirements met?
☑︎ Code completed
☑︎ Smoke tested
☑︎ Unit tests added
☑︎ Code documentation added
☑︎ Any pending items have an associated Issue
Tracking Issue
flyteorg/flyte#3371
flyteorg/flytekit
GitHub Actions: build-plugins (3.8, flytekit-greatexpectations)
GitHub Actions: build-plugins (3.8, flytekit-envd)
GitHub Actions: build-plugins (3.8, flytekit-duckdb)
GitHub Actions: build-plugins (3.8, flytekit-dolt)
GitHub Actions: build-plugins (3.8, flytekit-deck-standard)
GitHub Actions: build-plugins (3.8, flytekit-dbt)
GitHub Actions: build-plugins (3.8, flytekit-data-fsspec)
GitHub Actions: build-plugins (3.8, flytekit-dask)
GitHub Actions: build-plugins (3.8, flytekit-bigquery)
GitHub Actions: build-plugins (3.8, flytekit-aws-sagemaker)
GitHub Actions: build-plugins (3.8, flytekit-aws-batch)
GitHub Actions: build-plugins (3.8, flytekit-aws-athena)
GitHub Actions: lint
GitHub Actions: docs
GitHub Actions: Docs Warnings
✅ 2 other checks have passed
2/17 successful checksGitHub
04/25/2023, 5:06 AMnon-interruptible-node-selector-requirement
node selector if set. As such, these pods may be scheduled incorrectly on interruptible nodes.
Type
☑︎ Bug Fix
☐ Feature
☐ Plugin
Are all requirements met?
☑︎ Code completed
☑︎ Smoke tested
☑︎ Unit tests added
☐ Code documentation added
☐ Any pending items have an associated Issue
Complete description
Always apply node selector requirements for non-interruptible to spark driver pods.
flyteorg/flyteplugins
✅ All checks have passed
6/6 successful checksGitHub
04/25/2023, 5:08 AMScreenshot 2023-04-24 at 10 12 18 PM▾
GitHub
04/25/2023, 10:52 AMGitHub
04/25/2023, 5:04 PMGitHub
04/25/2023, 5:31 PMGitHub
04/25/2023, 5:50 PM<https://github.com/flyteorg/flyteplugins/tree/master|master>
by jeevb
<https://github.com/flyteorg/flyteplugins/commit/63e1e45de43c39d0bae7dfb973f0176a18228ecb|63e1e45d>
- Add non-interruptible node selector requirement to spark driver if set (#346)
flyteorg/flytepluginsGitHub
04/25/2023, 5:51 PMGitHub
04/25/2023, 6:37 PM<https://github.com/flyteorg/flyteconsole/tree/master|master>
by jsonporter
<https://github.com/flyteorg/flyteconsole/commit/957f9fa4e2406d34052f1ef8cace8287d121a691|957f9fa4>
- [Snyk] Upgrade eslint from 8.31.0 to 8.33.0 (#695)
flyteorg/flyteconsoleGitHub
04/25/2023, 6:37 PMrestrictDefaultExports
option to no-restricted-exports rule (#16785) (Nitin Kumar)
Documentation
• `17f4be2` docs: Fix examples in no-multiple-empty-lines rule (#16835) (jonz94)
• `9c7cfe3` docs: 'Source Code' content in 'Set up Development Environment' page (#16780) (Ben Perlmutter)
• `ede5c64` docs: Custom processors page (#16802) (Ben Perlmutter)
• `2620614` docs: Code of Conduct page (#16781) (Ben Perlmutter)
• `50a8efd` docs: report a sec vulnerability page (#16808) (Ben Perlmutter)
• `ed60afd` docs: Update page titles, section landing pages, and side TOC (#16760) (Ben Perlmutter)
• `333c712` docs: add background to code-path-diagrams for dark-mode (#16822) (Tanuj Kanti)
• `f5f7b9b` docs: Update README (GitHub Actions Bot)
• `2aa4f5f` docs: no-constant-condition: Add multi-comparison example (#16776) (Sebastian Simon)
• `40287db` docs: Remove Google Group icon (#16779) (Nicholas C. Zakas)
• `ea10ca5` docs: 'a .eslint' -> 'an .eslint' for consistency (#16809) (Ben Perlmutter)
• `3be0748` docs: add example for nodejs lintText api (#16789) (Siva K)
• `ce4f5ff` docs: Replace removed related rules with a valid rule (#16800) (Ville Saalo)
• 8.32.0 - 2023-01-15
Features
• `fc20f24` feat: add suggestions for redundant wrapping in prefer-regex-literals (#16658) (YeonJuan)
Bug Fixes
• `b4f8329` fix: ignore directives for no-fallthrough (#16757) (gfyoung)
Documentation
• `17b65ad` docs: IA Update page URL move (#16665) (Ben Perlmutter)
• `5981296` docs: fix theme switcher button (#16752) (Sam Chen)
• `6669413` docs: deploy prerelease docs under the /docs/next/
path (#16541) (Nitin Kumar)
• `78ecfe0` docs: use inline code for rule options name (#16768) (Percy Ma)
• `fc2ea59` docs: Update README (GitHub Actions Bot)
• `762a872` docs: Update README (GitHub Actions Bot)
Chores
• `2952d6e` chore: sync templates/*.md files with issue templates (#16758) (gfyoung)
• `3e34418` chore: Add new issues to triage project (#16740) (Nicholas C. Zakas)
• 8.31.0 - 2022-12-31
Read more
from eslint GitHub release notes Commit messages
Package name: eslint
• 13552c4 8.33.0
• ddeef02 Build: changelog update for 8.33.0
• 17f4be2 docs: Fix examples in no-multiple-empty-lines rule (#16835)
• 9c7cfe3 docs: 'Source Code' content in 'Set up Development Environment' page (#16780)
• ede5c64 docs: Custom processors page (#16802)
• 2620614 docs: Code of Conduct page (#16781)
• 50a8efd docs: report a sec vulnerability page (#16808)
• 2cc7954 feat: add restrictDefaultExports
option to no-restricted-exports rule (#16785)
• <https://snyk.i…
flyteorg/flyteconsole
GitHub Actions: Build & Push FlyteConsole Image
GitHub Actions: Get Release Tag
GitHub Actions: Generate Release
GitHub Actions: unit_tests_with_coverage
GitHub Actions: lint_project
DCO: DCO
✅ 3 other checks have passed
3/9 successful checksGitHub
04/25/2023, 9:01 PMGitHub
04/25/2023, 9:15 PMGitHub
04/25/2023, 9:51 PMGitHub
04/26/2023, 12:19 AMGitHub
04/26/2023, 5:36 AMGitHub
04/26/2023, 5:56 AM<https://github.com/flyteorg/flytekit/tree/master|master>
by eapolinario
<https://github.com/flyteorg/flytekit/commit/36fc15168d5923ecfb1c3640f9c2f8acfbe3eb50|36fc1516>
- pyflyte run now supports json/yaml files (#1606)
flyteorg/flytekit