1 new commit pushed to

<https://github.com/flyteorg/flyteplugins/tree/master|master>

by hamersaw

<https://github.com/flyteorg/flyteplugins/commit/9eeb694e3871f25638ea20775f19a352c31fa926|9eeb694e>

- Add podUID as a template input in tasklog plugin (#304) flyteorg/flyteplugins

Release - v1.0.27 New release published by flyte-bot Changelog • 9eeb694 Add podUID as a template input in tasklog plugin (#304) flyteorg/flyteplugins

#356 Dockerize buf call Pull request opened by hamersaw TL;DR Dockerizing the

buf

call so users do not need to manually install

buf

to generate protos. The PR uses the existing approach of setting the user id but requires manually setting the buf cache directory. If not set it will default to the root users home (ie.

/.cache

) which the caller likely does not have permissions to access. Therefore, we set it to

/tmp/cache

which should be globally accessible. Type ☐ Bug Fix ☑︎ Feature ☐ Plugin Are all requirements met? ☑︎ Code completed ☑︎ Smoke tested ☐ Unit tests added ☑︎ Code documentation added ☐ Any pending items have an associated Issue Complete description ^^^ Tracking Issue flyteorg/flyte#3139 Follow-up issue NA flyteorg/flyteidl

#935 Update documentation with new logging template variable: podUID Pull request opened by goyalankit Added the new podUID template variable in docs: flyteorg/flytesnacks

1 new commit pushed to

<https://github.com/flyteorg/flytekit/tree/master|master>

by wild-endeavor

<https://github.com/flyteorg/flytekit/commit/be32ac73200f3a3474d09c589b9d076db02d202f|be32ac73>

- Update log level for structured dataset (#1394) flyteorg/flytekit

#513 Update flyteplugin version to v1.0.27 Pull request opened by goyalankit TL;DR Update flyteplugin version to pull the changes that adds podUID as an a new template variable Type ☐ Bug Fix ☑︎ Feature ☐ Plugin Are all requirements met? ☑︎ Code completed ☐ Smoke tested ☑︎ Unit tests added ☑︎ Code documentation added ☐ Any pending items have an associated Issue Complete description How did you fix the bug, make the feature etc. Link to any design docs etc Tracking Issue _Remove the '_fixes_' keyword if there will be multiple PRs to fix the linked issue_ fixes https://github.com/flyteorg/flyte/issues/ Follow-up issue NA OR https://github.com/flyteorg/flyte/issues/ flyteorg/flytepropeller

1 new commit pushed to

<https://github.com/flyteorg/flyteidl/tree/master|master>

by hamersaw

<https://github.com/flyteorg/flyteidl/commit/398c54f1961211f00b9c0597de54bfe7bd2b5b6d|398c54f1>

- dockerizing buf call (#356) flyteorg/flyteidl

1 new commit pushed to

<https://github.com/flyteorg/flytepropeller/tree/master|master>

by hamersaw

<https://github.com/flyteorg/flytepropeller/commit/1feca6d2b1013a9ddbc7739affbc53801b746075|1feca6d2>

- Update flyteplugin version to v1.0.27 (#513) flyteorg/flytepropeller

1 new commit pushed to

<https://github.com/flyteorg/flytesnacks/tree/master|master>

by pingsutw

<https://github.com/flyteorg/flytesnacks/commit/f404a20adbfd5d837bc0e4d7c1b66a110beb807a|f404a20a>

- Databricks plugin (#928) flyteorg/flytesnacks

Release - v1.3.2 New release published by flyte-bot Changelog • 8681a86 Add inital

dask

plugin IDL (#339) • fd05a13 added comments to the catalog reservation API (#355) • 398c54f dockerizing buf call (#356) flyteorg/flyteidl

Release - v1.1.58 New release published by flyte-bot Changelog • 1feca6d Update flyteplugin version to v1.0.27 (#513) flyteorg/flytepropeller

Release - flytesnacks v0.3.157 New release published by flyte-bot Changelog • f404a20 Databricks plugin (#928) flyteorg/flytesnacks

#3210 Add note that flytekit supports python 3.7 - 3.10 Pull request opened by cosmicBboy Signed-off-by: Niels Bantilan niels.bantilan@gmail.com Signed-off-by: Niels Bantilan niels.bantilan@gmail.com Tracking issue Describe your changes Check all the applicable boxes ☐ I updated the documentation accordingly. ☐ All new and existing tests passed. ☐ All commits are signed-off. Screenshots Note to reviewers flyteorg/flyte

1 new commit pushed to

<https://github.com/flyteorg/flyteplugins/tree/master|master>

by hamersaw

<https://github.com/flyteorg/flyteplugins/commit/36c0b41bb5ed2633c4451162a144208abef563f3|36c0b41b>

- Add dask plugin #patch (#275) flyteorg/flyteplugins

#275 Add dask plugin #patch Pull request opened by bstadlbauer on <!date^1657179476^{date_short}|2022-07-07T07:37:56Z> TL;DR This PR adds a backend `dask` plugin using the `dask-kubernetes` operator to manage the

dask

cluster lifecycle. Type ☐ Bug Fix ☐ Feature ☑︎ Plugin Are all requirements met? ☑︎ Code completed ☐ Smoke tested ☑︎ Unit tests added ☑︎ Code documentation added ☑︎ Any pending items have an associated Issue Complete description The plugin works similar to the already existing

spark

plugin. It uses the

DaskJob

(docs) Custom Resource as a client pod which connects to the spun up cluster. This is similar to how the

SparkApplication

Custom Resource works for the

spark

plugin. TODO before this can be merged: ☐ Use correct

flyteidl

after flyteorg/flyteidl#339 is merged and released Tracking Issue flyteorg/flyte#427 Follow-up issue NA flyteorg/flyteplugins ✅ All checks have passed 6/6 successful checks

Release - v1.0.28 New release published by flyte-bot Changelog • 36c0b41 Add dask plugin #patch (#275) flyteorg/flyteplugins

#305 added raw-container to registered task types Pull request opened by flixr TL;DR Add the

raw-container

task type to not spam logs with useless warnings, analogous to #290 Type ☐ Bug Fix ☑︎ Feature ☐ Plugin Are all requirements met? ☑︎ Code completed ☐ Smoke tested ☐ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description ^^^ Tracking Issue NA Follow-up issue NA flyteorg/flyteplugins

1 new commit pushed to

<https://github.com/flyteorg/flytesnacks/tree/master|master>

by hamersaw

<https://github.com/flyteorg/flytesnacks/commit/f373a6f2373e91addfdb3ee51d7e4af0e334a1c0|f373a6f2>

- Update documentation with new logging template variable: podUID (#935) flyteorg/flytesnacks

#1404 Add Niels to code owners Pull request opened by pingsutw Signed-off-by: Kevin Su pingsutw@apache.org TL;DR Adding @cosmicBboy flyteorg/flytekit

#3211 [UI Feature] improve status display Issue created by flixr Motivation: Why do you think this is important? I missed that some tasks were not actually running because the status shows running, but it is actually still queued. Goal: What should the final outcome look like, ideally? The status should show more info, like queued, etc... Propose: Link/Inline OR Additional context No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

1 new commit pushed to

<https://github.com/flyteorg/flytekit/tree/master|master>

by wild-endeavor

<https://github.com/flyteorg/flytekit/commit/0dcda1dda1290077cbaf5b6143e0f226c4e9142a|0dcda1dd>

- Add Niels to code owners (#1404) flyteorg/flytekit

1 new commit pushed to

<https://github.com/flyteorg/flyte/tree/master|master>

by cosmicBboy

<https://github.com/flyteorg/flyte/commit/10285d9044d57464df7170d1f2f3ea0ce13b982d|10285d90>

- Add note that flytekit supports python 3.7 - 3.10 (#3210) flyteorg/flyte

Release - flytesnacks v0.3.158 New release published by flyte-bot Changelog • f373a6f Update documentation with new logging template variable: podUID (#935) flyteorg/flytesnacks

1 new commit pushed to

<https://github.com/flyteorg/flytekit/tree/master|master>

by wild-endeavor

<https://github.com/flyteorg/flytekit/commit/d0089176870b0834d30ca39ef20808a1ad5036fa|d0089176>

- Signal use (#1398) flyteorg/flytekit

1 new commit pushed to

<https://github.com/flyteorg/flytekit/tree/master|master>

by wild-endeavor

<https://github.com/flyteorg/flytekit/commit/8c7208282c8226099debe2f065a36590b35c89c6|8c720828>

- User Documentation Proposal (#1200) flyteorg/flytekit

#936 Update lp_schedules.py Pull request opened by cosmicBboy Signed-off-by: Niels Bantilan niels.bantilan@gmail.com flyteorg/flytesnacks

#3212 [Core] Backfill and look-back Issue created by frelyf Description Users want to know how to run backfill jobs. The docs should include an example of how to trigger a launch plan across a range of dates. The launch plan should take datetime as an input so that the range of dates can be applied onto it. This is different from dynamic and map tasks because backfill jobs likely want to run on their own nodes and want to be evaluated at compile-time. A similar use case is for workflows that want look-back. When users run daily jobs, it is convenient to quickly evaluate the state of the last couple of days. If the data paths of the previous days' outputs is well known, it is a simple gsutil ls (or the like on other file systems) to check this. If a previous day is missing its output data, then that day should be processed and that workflow will proceed with the heavy compute. Being able to put the workflow iterator on a cron schedule would make this possible. I haven't been able to prove this case yet, but believe it is technically possible. Method to iterate over dates on a launch plan and return a workflow: `

def generate_backfill_workflow(
    start_date: datetime, end_date: datetime, base_lp: LaunchPlan
) -> Workflow:

    if base_lp.schedule is None:
        raise ValueError("Backfill can only be created for scheduled launchplans")

    if isinstance(base_lp.schedule, CronSchedule):
        pass
    else:
        raise NotImplementedError("The launchplan schedule needs to be a cron schedule")

    if start_date >= end_date:
        raise ValueError("Start date should be greater than end date")

    print(f"Generating backfill for {start_date} to {end_date}")
    wf = Workflow(name=f"backfill-{base_lp.name}")
    lp_iter = croniter(
        base_lp.schedule.cron_schedule.schedule, start_time=start_date, ret_type=datetime
    )
    while True:
        next_start_date = lp_iter.get_next()
        if next_start_date > end_date:
            break
        print(f"Adding -> {next_start_date}")
        wf.add_launch_plan(base_lp, kickoff_time=next_start_date)

` Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#306 Add env FLYTE_FAIL_ON_ERROR to aws batch job Pull request opened by pingsutw Signed-off-by: Kevin Su pingsutw@apache.org TL;DR Add env FLYTE_FAIL_ON_ERROR to aws batch job, so the flytekit will return error code once the job fail. Type ☑︎ Bug Fix ☐ Feature ☐ Plugin Are all requirements met? ☑︎ Code completed ☑︎ Smoke tested ☐ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description Tracking Issue https://flyte-org.slack.com/archives/C01P3B761A6/p1664972219043289 Follow-up issue NA flyteorg/flyteplugins ✅ All checks have passed 6/6 successful checks

#1405 fix remote API reference Pull request opened by cosmicBboy Signed-off-by: Niels Bantilan niels.bantilan@gmail.com TL;DR Please replace this text with a description of what this PR accomplishes. Type ☐ Docs ☐ Bug Fix ☐ Feature ☐ Plugin Are all requirements met? ☐ Code completed ☐ Smoke tested ☐ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description How did you fix the bug, make the feature etc. Link to any design docs etc Tracking Issue https://github.com/flyteorg/flyte/issues/ Follow-up issue NA OR https://github.com/flyteorg/flyte/issues/ flyteorg/flytekit

#3125 [Core feature] [flytekit] Add signaling experience to FlyteRemote Issue created by wild-endeavor Task This added the human-in-the-loop signaling experience provided for by the backend here and here. We need to add to the FlyteRemote experience however, allowing users to set and fulfill signals in running executions using just the FlyteRemote interface. Part of #208 Misc Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte