GitHub
01/04/2023, 10:05 PM<https://github.com/flyteorg/flyteplugins/tree/master|master>
by hamersaw
<https://github.com/flyteorg/flyteplugins/commit/9eeb694e3871f25638ea20775f19a352c31fa926|9eeb694e>
- Add podUID as a template input in tasklog plugin (#304)
flyteorg/flytepluginsGitHub
01/04/2023, 10:06 PMGitHub
01/04/2023, 11:01 PMbuf
call so users do not need to manually install buf
to generate protos. The PR uses the existing approach of setting the user id but requires manually setting the buf cache directory. If not set it will default to the root users home (ie. /.cache
) which the caller likely does not have permissions to access. Therefore, we set it to /tmp/cache
which should be globally accessible.
Type
☐ Bug Fix
☑︎ Feature
☐ Plugin
Are all requirements met?
☑︎ Code completed
☑︎ Smoke tested
☐ Unit tests added
☑︎ Code documentation added
☐ Any pending items have an associated Issue
Complete description
^^^
Tracking Issue
flyteorg/flyte#3139
Follow-up issue
NA
flyteorg/flyteidlGitHub
01/05/2023, 12:19 AMimage▾
GitHub
01/05/2023, 12:21 AM<https://github.com/flyteorg/flytekit/tree/master|master>
by wild-endeavor
<https://github.com/flyteorg/flytekit/commit/be32ac73200f3a3474d09c589b9d076db02d202f|be32ac73>
- Update log level for structured dataset (#1394)
flyteorg/flytekitGitHub
01/05/2023, 12:32 AMGitHub
01/05/2023, 1:31 AM<https://github.com/flyteorg/flyteidl/tree/master|master>
by hamersaw
<https://github.com/flyteorg/flyteidl/commit/398c54f1961211f00b9c0597de54bfe7bd2b5b6d|398c54f1>
- dockerizing buf call (#356)
flyteorg/flyteidlGitHub
01/05/2023, 1:32 AM<https://github.com/flyteorg/flytepropeller/tree/master|master>
by hamersaw
<https://github.com/flyteorg/flytepropeller/commit/1feca6d2b1013a9ddbc7739affbc53801b746075|1feca6d2>
- Update flyteplugin version to v1.0.27 (#513)
flyteorg/flytepropellerGitHub
01/05/2023, 1:34 AM<https://github.com/flyteorg/flytesnacks/tree/master|master>
by pingsutw
<https://github.com/flyteorg/flytesnacks/commit/f404a20adbfd5d837bc0e4d7c1b66a110beb807a|f404a20a>
- Databricks plugin (#928)
flyteorg/flytesnacksGitHub
01/05/2023, 1:37 AMdask
plugin IDL (#339)
• fd05a13 added comments to the catalog reservation API (#355)
• 398c54f dockerizing buf call (#356)
flyteorg/flyteidlGitHub
01/05/2023, 2:23 AMGitHub
01/05/2023, 2:46 AMGitHub
01/05/2023, 3:00 PMGitHub
01/05/2023, 4:17 PM<https://github.com/flyteorg/flyteplugins/tree/master|master>
by hamersaw
<https://github.com/flyteorg/flyteplugins/commit/36c0b41bb5ed2633c4451162a144208abef563f3|36c0b41b>
- Add dask plugin #patch (#275)
flyteorg/flytepluginsGitHub
01/05/2023, 4:17 PMdask
cluster lifecycle.
Type
☐ Bug Fix
☐ Feature
☑︎ Plugin
Are all requirements met?
☑︎ Code completed
☐ Smoke tested
☑︎ Unit tests added
☑︎ Code documentation added
☑︎ Any pending items have an associated Issue
Complete description
The plugin works similar to the already existing spark
plugin. It uses the DaskJob
(docs) Custom Resource as a client pod which connects to the spun up cluster. This is similar to how the SparkApplication
Custom Resource works for the spark
plugin.
TODO before this can be merged:
☐ Use correct flyteidl
after flyteorg/flyteidl#339 is merged and released
Tracking Issue
flyteorg/flyte#427
Follow-up issue
NA
flyteorg/flyteplugins
✅ All checks have passed
6/6 successful checksGitHub
01/05/2023, 4:18 PMGitHub
01/05/2023, 5:33 PMraw-container
task type to not spam logs with useless warnings, analogous to #290
Type
☐ Bug Fix
☑︎ Feature
☐ Plugin
Are all requirements met?
☑︎ Code completed
☐ Smoke tested
☐ Unit tests added
☐ Code documentation added
☐ Any pending items have an associated Issue
Complete description
^^^
Tracking Issue
NA
Follow-up issue
NA
flyteorg/flytepluginsGitHub
01/05/2023, 5:47 PM<https://github.com/flyteorg/flytesnacks/tree/master|master>
by hamersaw
<https://github.com/flyteorg/flytesnacks/commit/f373a6f2373e91addfdb3ee51d7e4af0e334a1c0|f373a6f2>
- Update documentation with new logging template variable: podUID (#935)
flyteorg/flytesnacksGitHub
01/05/2023, 6:16 PMGitHub
01/05/2023, 6:27 PMflyte_misleading_status▾
GitHub
01/05/2023, 6:53 PM<https://github.com/flyteorg/flytekit/tree/master|master>
by wild-endeavor
<https://github.com/flyteorg/flytekit/commit/0dcda1dda1290077cbaf5b6143e0f226c4e9142a|0dcda1dd>
- Add Niels to code owners (#1404)
flyteorg/flytekitGitHub
01/05/2023, 8:05 PM<https://github.com/flyteorg/flyte/tree/master|master>
by cosmicBboy
<https://github.com/flyteorg/flyte/commit/10285d9044d57464df7170d1f2f3ea0ce13b982d|10285d90>
- Add note that flytekit supports python 3.7 - 3.10 (#3210)
flyteorg/flyteGitHub
01/05/2023, 8:09 PMGitHub
01/05/2023, 9:34 PM<https://github.com/flyteorg/flytekit/tree/master|master>
by wild-endeavor
<https://github.com/flyteorg/flytekit/commit/d0089176870b0834d30ca39ef20808a1ad5036fa|d0089176>
- Signal use (#1398)
flyteorg/flytekitGitHub
01/05/2023, 11:51 PM<https://github.com/flyteorg/flytekit/tree/master|master>
by wild-endeavor
<https://github.com/flyteorg/flytekit/commit/8c7208282c8226099debe2f065a36590b35c89c6|8c720828>
- User Documentation Proposal (#1200)
flyteorg/flytekitGitHub
01/06/2023, 2:54 AMGitHub
01/06/2023, 8:15 AMdef generate_backfill_workflow(
start_date: datetime, end_date: datetime, base_lp: LaunchPlan
) -> Workflow:
if base_lp.schedule is None:
raise ValueError("Backfill can only be created for scheduled launchplans")
if isinstance(base_lp.schedule, CronSchedule):
pass
else:
raise NotImplementedError("The launchplan schedule needs to be a cron schedule")
if start_date >= end_date:
raise ValueError("Start date should be greater than end date")
print(f"Generating backfill for {start_date} to {end_date}")
wf = Workflow(name=f"backfill-{base_lp.name}")
lp_iter = croniter(
base_lp.schedule.cron_schedule.schedule, start_time=start_date, ret_type=datetime
)
while True:
next_start_date = lp_iter.get_next()
if next_start_date > end_date:
break
print(f"Adding -> {next_start_date}")
wf.add_launch_plan(base_lp, kickoff_time=next_start_date)
`
Are you sure this issue hasn't been raised already?
☑︎ Yes
Have you read the Code of Conduct?
☑︎ Yes
flyteorg/flyteGitHub
01/06/2023, 9:51 AMimage▾
GitHub
01/06/2023, 3:27 PMGitHub
01/06/2023, 6:04 PM