#3742 [FlyteCTL Feature] Allow seed project description to be set Issue created by rupurt Describe the feature/command for FlyteCTL Currently you can only set the name of the seed project as a string flyte/cmd/single/config.go Line 27 in </flyteorg/flyte/commit/14171fd84d171a609e619a214fa36695fc79990b|14171fd> #2503 Provide a possible output or UX example Support the current implementation + the ability to set a description With Description

flyte:
  admin:
    seedProjects:
      - name: myproject
        description: My project description

Without Description

flyte:
  admin:
    seedProjects:
      - koi

Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#3743 [Housekeeping] Improve logging around cluster sync loops Issue created by EngHabu Describe the issue Cluster sync loop produce 0 logs (some debug logs) in case of successes which makes it harder to understand and track changes to the templates and config. What if we do not do this? N/A Related component(s) No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#573 Add logs and stats to sync loop Pull request opened by EngHabu TL;DR Add info logging for when a sync loop starts and ends and add stats at the end of the sync loop to the log. Type ☐ Bug Fix ☑︎ Feature ☐ Plugin Are all requirements met? ☑︎ Code completed ☐ Smoke tested ☐ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Tracking Issue fixes flyteorg/flyte#3743 flyteorg/flyteadmin GitHub Actions: Build & Push Flyteadmin Image GitHub Actions: Goreleaser GitHub Actions: Build & Push Flytescheduler Image GitHub Actions: Bump Version ✅ 10 other checks have passed 10/14 successful checks

#3744 [BUG] Can't create project from file in demo Issue created by rupurt Describe the bug Given a

flyte

demo cluster When I try to create a project from a file Then it should be created in the

demo

cluster But it currently panics with an invalid memory address

> flytectl create project --file ./flyte/projects/myproject.yaml
INFO[0000] [0] Couldn't find a config file []. Relying on env vars and pflags.
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x147b463]

goroutine 1 [running]:
<http://github.com/flyteorg/flytectl/cmd/core.CommandContext.AdminClient(...)|github.com/flyteorg/flytectl/cmd/core.CommandContext.AdminClient(...)>
        /home/runner/work/flytectl/flytectl/cmd/core/cmd_ctx.go:57
<http://github.com/flyteorg/flytectl/cmd/create.createProjectsCommand({0x2563718|github.com/flyteorg/flytectl/cmd/create.createProjectsCommand({0x2563718>, 0xc00005c228}, {0xc000ec5b40?, 0x100000049c165?, 0x25633c0?}, {0x0, {0x0, 0x0}, {0x0, 0x0}, ...})
        /home/runner/work/flytectl/flytectl/cmd/create/project.go:62 +0xe3
<http://github.com/flyteorg/flytectl/cmd/core.generateCommandFunc.func1(0xc000af2f00|github.com/flyteorg/flytectl/cmd/core.generateCommandFunc.func1(0xc000af2f00>?, {0xc000a41620, 0x0, 0x2})
        /home/runner/work/flytectl/flytectl/cmd/core/cmd.go:70 +0x93d
<http://github.com/spf13/cobra.(*Command).execute(0xc000af2f00|github.com/spf13/cobra.(*Command).execute(0xc000af2f00>, {0xc000a41600, 0x2, 0x2})
        /home/runner/go/pkg/mod/github.com/spf13/cobra@v1.4.0/command.go:856 +0x67c
<http://github.com/spf13/cobra.(*Command).ExecuteC(0xc000a6e780)|github.com/spf13/cobra.(*Command).ExecuteC(0xc000a6e780)>
        /home/runner/go/pkg/mod/github.com/spf13/cobra@v1.4.0/command.go:974 +0x3bd
<http://github.com/spf13/cobra.(*Command).Execute(...)|github.com/spf13/cobra.(*Command).Execute(...)>
        /home/runner/go/pkg/mod/github.com/spf13/cobra@v1.4.0/command.go:902
<http://github.com/flyteorg/flytectl/cmd.ExecuteCmd()|github.com/flyteorg/flytectl/cmd.ExecuteCmd()>
        /home/runner/work/flytectl/flytectl/cmd/root.go:137 +0x1e
main.main()
        /home/runner/work/flytectl/flytectl/main.go:12 +0x1d

Expected behavior A project to be created from the file Additional context to reproduce No response Screenshots No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#3745 [Housekeeping] flytekit to support generic collections in 3.9+ Issue created by wild-endeavor Support builtins generics Implement https://peps.python.org/pep-0585/ basically, so that in python 3.9+ users don't have to do

typing.List

and can just use

list

Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#1673 Remote file access wip Pull request opened by wild-endeavor on <!date^1685757002^{date_short}|2023-06-03T01:50:02Z> TL;DR Please replace this text with a description of what this PR accomplishes. Type ☐ Bug Fix ☐ Feature ☐ Plugin Are all requirements met? ☐ Code completed ☐ Smoke tested ☐ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description How did you fix the bug, make the feature etc. Link to any design docs etc Tracking Issue https://github.com/flyteorg/flyte/issues/ Follow-up issue NA OR https://github.com/flyteorg/flyte/issues/ flyteorg/flytekit GitHub Actions: build-plugins (3.11, flytekit-sqlalchemy) GitHub Actions: build-plugins (3.11, flytekit-snowflake) GitHub Actions: build-plugins (3.11, flytekit-polars) GitHub Actions: build-plugins (3.11, flytekit-papermill) GitHub Actions: build-plugins (3.11, flytekit-pandera) GitHub Actions: build-plugins (3.11, flytekit-kf-tensorflow) GitHub Actions: build-plugins (3.11, flytekit-kf-pytorch) GitHub Actions: build-plugins (3.11, flytekit-kf-mpi) GitHub Actions: build-plugins (3.11, flytekit-k8s-pod) GitHub Actions: build-plugins (3.11, flytekit-hive) GitHub Actions: build-plugins (3.11, flytekit-envd) GitHub Actions: build-plugins (3.11, flytekit-duckdb) GitHub Actions: build-plugins (3.11, flytekit-dolt) GitHub Actions: build-plugins (3.11, flytekit-deck-standard) GitHub Actions: build-plugins (3.11, flytekit-dbt) GitHub Actions: build-plugins (3.11, flytekit-data-fsspec) GitHub Actions: build-plugins (3.11, flytekit-dask) GitHub Actions: build-plugins (3.11, flytekit-bigquery) GitHub Actions: build-plugins (3.11, flytekit-aws-sagemaker) GitHub Actions: build-plugins (3.11, flytekit-aws-batch) GitHub Actions: build-plugins (3.11, flytekit-aws-athena) GitHub Actions: build-plugins (3.10, flytekit-whylogs) GitHub Actions: build-plugins (3.10, flytekit-vaex) GitHub Actions: build-plugins (3.10, flytekit-sqlalchemy) GitHub Actions: build-plugins (3.10, flytekit-spark) GitHub Actions: build-plugins (3.10, flytekit-snowflake) GitHub Actions: build-plugins (3.10, flytekit-ray) GitHub Actions: build-plugins (3.10, flytekit-polars) GitHub Actions: build-plugins (3.10, flytekit-papermill) GitHub Actions: build-plugins (3.10, flytekit-pandera) ✅ No checks have passed 0/30 successful checks

#355 [Bigquery] Add support for impersonation of GSA bound to task's KSA Pull request opened by jeevb Addresses: flyteorg/flyte#3736 Incorporates lost changes from #161 with some some modern bits. Smoke tested on GCP test cluster with flytesnacks BQ workflow. In the second snapshot, the correct GSA,

userflyterole

, is used. flyteorg/flyteplugins Codecov: 62.57% (-0.24%) compared to c499a48 Codecov: 27.53% of diff hit (target 62.80%) ✅ 5 other checks have passed 5/7 successful checks

#226 Support converters to binding collection &amp; maps Pull request opened by andresgomezfrr TL;DR Please replace this text with a description of what this PR accomplishes. Add support on the Java<->Scala converters to convert SdkBindingData created using BindingMap and BindingCollections. Type ☑︎ Bug Fix ☐ Feature ☐ Plugin Are all requirements met? ☐ Code completed ☐ Smoke tested ☑︎ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue flyteorg/flytekit-java ✅ All checks have passed 3/3 successful checks

1 new commit pushed to

<https://github.com/flyteorg/flytekit-java/tree/master|master>

by andresgomezfrr

<https://github.com/flyteorg/flytekit-java/commit/a165cf1d6d966563f832a241de8a02b6f3b61cd7|a165cf1d>

- Support converters to binding collection & maps (#226) flyteorg/flytekit-java

1 new commit pushed to

<https://github.com/flyteorg/flytekit-java/tree/master|master>

by github-actions[bot]

<https://github.com/flyteorg/flytekit-java/commit/35d6b75fb1d9722517b705cd1a269be8434e4c8c|35d6b75f>

- [maven-release-plugin] prepare release 0.4.17 flyteorg/flytekit-java

1 new commit pushed to

<https://github.com/flyteorg/flytekit-java/tree/master|master>

by github-actions[bot]

<https://github.com/flyteorg/flytekit-java/commit/943ac342841d79eae38ad9e4d9895434eae53926|943ac342>

- [maven-release-plugin] prepare for next development iteration flyteorg/flytekit-java

#227 Support converters to binding collection &amp; maps (toJava) Pull request opened by andresgomezfrr TL;DR Follow-up PR of #226 to fix the

toJava()

of bindingCollection and bindingMap Type ☑︎ Bug Fix ☐ Feature ☐ Plugin Are all requirements met? ☐ Code completed ☐ Smoke tested ☑︎ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue flyteorg/flytekit-java ✅ All checks have passed 3/3 successful checks

1 new commit pushed to

<https://github.com/flyteorg/flytekit-java/tree/master|master>

by andresgomezfrr

<https://github.com/flyteorg/flytekit-java/commit/342714b999bbc0b22be2d19eeaea61ea7d9866a5|342714b9>

- Support converters to binding collection & maps (toJava) (#227) flyteorg/flytekit-java

1 new commit pushed to

<https://github.com/flyteorg/flytekit-java/tree/master|master>

by andresgomezfrr

<https://github.com/flyteorg/flytekit-java/commit/510ec1a710be364dd1bc57c9f9e33ea2cc8e1f1a|510ec1a7>

- fix: upgrade com.google.errorprone:error_prone_annotations from 2.18.0 to 2.19.1 (#225) flyteorg/flytekit-java

1 new commit pushed to

<https://github.com/flyteorg/flytekit-java/tree/master|master>

by andresgomezfrr

<https://github.com/flyteorg/flytekit-java/commit/a5eb4f79955a56ae4324c836af8b7498131e80e5|a5eb4f79>

- fix: upgrade info.picocli:picocli from 4.7.1 to 4.7.2 (#224) flyteorg/flytekit-java

#224 [Snyk] Upgrade info.picocli:picocli from 4.7.1 to 4.7.2 Pull request opened by snyk-bot Snyk has created this PR to upgrade info.picocli:picocli from 4.7.1 to 4.7.2. ℹ️ Keep your dependencies up-to-date. This makes it easier to fix existing vulnerabilities and to more quickly identify and fix newly disclosed vulnerabilities when they affect your project. * * * • The recommended version is 1 version ahead of your current version. • The recommended version was released 22 days ago, on 2023-04-09. * * * Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open upgrade PRs. For more information: https://camo.githubusercontent.com/642b96ab7f84300036f39fc3423b3b3c84f4083cfed45a0cc61153c83c977151/68747470733a2f2f6170692e7365676d656e742e696f2f76312f706978656c2f747261636b3f646174613d65794a33636d6c305a55746c65534936496e4a79576d785a634564485932527954485a7362306c596430645563566734576b4652546e4e434f5545774969776959573576626e6c746233567a535751694f69497a4e7a64695a6d45775a6930354d5467774c54526b4f446774595455774f4330314d6d4a6a4e54466d4f5463304e444d694c434a6c646d567564434936496c425349485a705a58646c5a434973496e42796233426c636e52705a584d694f6e736963484a4a5a434936496a4d334e324a6d5954426d4c546b784f4441744e4751344f4331684e5441344c545579596d4d314d5759354e7a51304d794a3966513d3d 🧐 View latest project report 🛠️ Adjust upgrade PR settings 🔕 Ignore this dependency or unsubscribe from future upgrade PRs flyteorg/flytekit-java ✅ All checks have passed 3/3 successful checks

1 new commit pushed to

<https://github.com/flyteorg/flytekit-java/tree/master|master>

by andresgomezfrr

<https://github.com/flyteorg/flytekit-java/commit/440047611c551451c626eaffd0f949f13ada8995|44004761>

- fix: upgrade com.google.api.grpc:proto-google-common-protos from 2.14.2 to 2.14.3 (#223) flyteorg/flytekit-java

#223 [Snyk] Upgrade com.google.api.grpc:proto-google-common-protos from 2.14.2 to 2.14.3 Pull request opened by snyk-bot Snyk has created this PR to upgrade com.google.api.grpc:proto-google-common-protos from 2.14.2 to 2.14.3. ℹ️ Keep your dependencies up-to-date. This makes it easier to fix existing vulnerabilities and to more quickly identify and fix newly disclosed vulnerabilities when they affect your project. * * * • The recommended version is 1 version ahead of your current version. • The recommended version was released 22 days ago, on 2023-03-14. * * * Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open upgrade PRs. For more information: https://camo.githubusercontent.com/34240eee664fe1d5c9566e1627948fe1ffa5ec6fae14486315c8ca68b6ee6bd0/68747470733a2f2f6170692e7365676d656e742e696f2f76312f706978656c2f747261636b3f646174613d65794a33636d6c305a55746c65534936496e4a79576d785a634564485932527954485a7362306c596430645563566734576b4652546e4e434f5545774969776959573576626e6c746233567a535751694f6949314d7a6c6d5a57526a4e433168596a4d7a4c54526c59546b74596a4a694f53307a4f574a6d59325a6b4d6d45324d6d49694c434a6c646d567564434936496c425349485a705a58646c5a434973496e42796233426c636e52705a584d694f6e736963484a4a5a434936496a557a4f575a6c5a474d304c5746694d7a4d744e4756684f5331694d6d49354c544d35596d5a6a5a6d51795954597959694a3966513d3d 🧐 View latest project report 🛠️ Adjust upgrade PR settings 🔕 Ignore this dependency or unsubscribe from future upgrade PRs flyteorg/flytekit-java ✅ All checks have passed 3/3 successful checks

#3746 [FlyteCTL Feature] Support passing envs when creating executions Issue created by honnix Describe the feature/command for FlyteCTL Currently

flytectl

does not support passing envs when creating executions, which

pyflyte

does: flyteorg/flytekit#1617 Provide a possible output or UX example Envs can be passed to flyteadmin when creating executions. This could either be achieved in the same as

pyflyte

or utilizing

flytectl

execution spec file. Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#408 Support envs when creating execution Pull request opened by honnix TL;DR Support envs when creating execution. Type ☐ Bug Fix ☑︎ Feature ☐ Plugin Are all requirements met? ☑︎ Code completed ☑︎ Smoke tested ☑︎ Unit tests added ☑︎ Code documentation added ☐ Any pending items have an associated Issue Complete description As described by flyteorg/flyte#3746,

flytectl

currently does not support passing envs to flyteadmin when creating executions. This PR adds the support by utilizing execution spec file. Note that this works differently than

pyflyte

, which is on one hand not ideal, but OTOH given the existence of execution spec file in

flytectl

, using the file instead of a plain json string is less error prone and more traceable. Tracking Issue Closes flyteorg/flyte#3746 Follow-up issue NA flyteorg/flytectl GitHub Actions: Goreleaser GitHub Actions: Generate documentation GitHub Actions: Bump Version ✅ 7 other checks have passed 7/10 successful checks

1 new commit pushed to

<https://github.com/flyteorg/flytekit-java/tree/master|master>

by github-actions[bot]

<https://github.com/flyteorg/flytekit-java/commit/5d953197da82c02d7c31adbfd3bb65f6d6919431|5d953197>

- [maven-release-plugin] prepare release 0.4.18 flyteorg/flytekit-java

1 new commit pushed to

<https://github.com/flyteorg/flytekit-java/tree/master|master>

by github-actions[bot]

<https://github.com/flyteorg/flytekit-java/commit/8076468ed22180e75adef19da74769631b9ca6c7|8076468e>

- [maven-release-plugin] prepare for next development iteration flyteorg/flytekit-java

#3747 [BUG] remote sync map_tasks Issue created by lauralindy Describe the bug Can't seem to use FlyteRemote to sync workflows/tasks with map_task

_InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
	status = StatusCode.NOT_FOUND
	details = "missing entity of type TASK with identifier project:"flytetester" domain:"development" name:"MAP_TASK_NAME" version:"VERSION" "
	debug_error_string = "UNKNOWN:Error received from peer ipv4:10.34.2.243:443 {grpc_message:"missing entity of type TASK with identifier project:\"flytetester\" domain:\"development\" name:\"MAP_TASK_NAME\" version:\"VERSION\" ", grpc_status:5, created_time:"2023-06-02T15:32:48.383186-07:00"}"
>

Expected behavior expect to be able to see the map_task tasks' info Additional context to reproduce

task_execution = client.get_task_execution(flytekit.models.core.identifier.TaskExecutionIdentifier(
    task_id=task_identifier, 
    node_execution_id=node_exe_identifier,
    retry_attempt=0
))
flyte_task_exe = flytekit.remote.executions.FlyteTaskExecution.promote_from_model(task_execution)
remote.sync_task_execution(
    execution=flyte_task_exe,
)

where task_identifier and node_exe_identifier is for the map_task or call

synced = remote.sync(
    execution=execution,
    sync_nodes=True
)

on the workflow level. Screenshots No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

Pending review on flyteorg/flyteidl - Manage reminder [#297] Proposal : Settings proto (pmahindrakar-oss) 5 months stale · a year old · Waiting on _ @Haytham Abuelfutuh, katrogan_

Pending review on flyteorg/flytekit-python-template - Manage reminder [#40] ImageSpec overhaul (zeryx) 6 days stale · 6 days old · Waiting on _ @Niels Bantilan_

1 new commit pushed to

<https://github.com/flyteorg/flyteadmin/tree/master|master>

by EngHabu

<https://github.com/flyteorg/flyteadmin/commit/949f5a25e5f1fdca96e9908fde6506d1ff12a2b8|949f5a25>

- Add logs and stats to sync loop (#573) flyteorg/flyteadmin

Release - v1.1.105 New release published by flyte-bot Changelog • 949f5a2 Add logs and stats to sync loop (#573) flyteorg/flyteadmin

#3748 [BUG] Optional type transform not working in dynamic workflow Issue created by jganbat-fn Describe the bug Optional type transformation on output is not working during dynamic workflow compilation. Following example workflow:

from __future__ import absolute_import, division, print_function
from typing import Optional

from flytekit import task, workflow, dynamic
from flytekit.types.file import FlyteFile

@task
def hello(myfile: Optional[FlyteFile]) -> Optional[FlyteFile]:
    return None

@dynamic
def dyn_hello(myfile: Optional[FlyteFile]) -> Optional[FlyteFile]:
    return hello(myfile=myfile)


@workflow
def child_workflow(myfile: Optional[FlyteFile]) -> Optional[FlyteFile]:
    hello_output = dyn_hello(myfile=myfile)
    return hello_output

when executed showing the following issue in FlytePropeller

E0601 21:21:14.327422       1 workers.go:102] error syncing 'flyte-sandbox/anwfrlcbl62scwxzsnh7': 0: [User] malformed dynamic workflow, caused by: Collected Errors: 2
	Error 0: Code: MismatchingTypes, Node Id: end-node, Description: Variable [o1] (type [union_type:<variants:<blob:<> structure:<tag:"FlyteFilePath" > > variants:<simple:NONE structure:<tag:"none" > > > ]) doesn't match expected type [union_type:<variants:<blob:<> > variants:<simple:NONE > > ].
	Error 1: Code: ParameterNotBound, Node Id: end-node, Description: Parameter not bound [o1].

Flytekit version:

v1.4

Flytepropeller version:

v1.1.91

Expected behavior Optional output type is correctly transformed and routed to downstream tasks or returned. relevant slack conversation is here https://flyte-org.slack.com/archives/CP2HDHKE1/p1685654534312309 Additional context to reproduce Register and run the example workflow described above in sandbox. Screenshots No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

1 new commit pushed to

<https://github.com/flyteorg/flytectl/tree/master|master>

by eapolinario

<https://github.com/flyteorg/flytectl/commit/0cb2bfaf436a09d58c1ae9c1389eacb6eab8a856|0cb2bfaf>

- Support envs when creating execution (#408) flyteorg/flytectl

#409 Update documentation Pull request opened by flyte-bot Update documentation • Auto-generated by [flyte-bot] flyteorg/flytectl ✅ All checks have passed 2/2 successful checks