#578 [Feature] flytectl cross-domain commands Issue created by wild-endeavor Motivation: Why do you think this is important? For certain commands in flyte-cli like update-workflow-meta need to have the domain specified. This is fine but usually, when users are archiving a workflow, they want to do it for all domains. We should have an option in flytectl to do this, a --all-domains switch or something. Goal: What should the final outcome look like, ideally? See above. Describe alternatives you've considered None. Flyte component ☐ Overall ☐ Flyte Setup and Installation scripts ☐ Flyte Documentation ☐ Flyte communication (slack/email etc) ☐ FlytePropeller ☐ FlyteIDL (Flyte specification language) ☐ Flytekit (Python SDK) ☐ FlyteAdmin (Control Plane service) ☐ FlytePlugins ☐ DataCatalog ☐ FlyteStdlib (common libraries) ☐ FlyteConsole (UI) ☑︎ Other GitHub repo(s) flytectl flyteorg/flyte

#622 [Feature] Flyteadmin cluster resource controller should perform deletion changes Issue created by katrogan Motivation: Why do you think this is important? If a user adds a template to the cluster resource controller and then later on removes it, there's no automated process for picking up the deletion and applying it as a patch to existing resources. Goal: What should the final outcome look like, ideally? Deleting a custom template should result in the state of the world appearing as if the template had never existed. Describe alternatives you've considered The current implementation requires system administrators to go through and apply a patch to remove.deleted k8s resource attributes. This is a manual process and potentially error-prone. Flyte component ☐ Overall ☐ Flyte Setup and Installation scripts ☐ Flyte Documentation ☐ Flyte communication (slack/email etc) ☐ FlytePropeller ☐ FlyteIDL (Flyte specification language) ☐ Flytekit (Python SDK) ☑︎ FlyteAdmin (Control Plane service) ☐ FlytePlugins ☐ DataCatalog ☐ FlyteStdlib (common libraries) ☐ FlyteConsole (UI) ☐ Other [Optional] Propose: Link/Inline If you have ideas about the implementation please propose the change. If inline keep it short, if larger then you link to an external document. Additional context N/A Is this a blocker for you to adopt Flyte N/A flyteorg/flyte

#629 [Feature] Add a configurable limit for the number of active launch plans per workflow Issue created by katrogan Motivation: Why do you think this is important? It's possible for a stray user to activate many launch plans, all with schedules and potentially DDOS flyte. Configurable limits should be in place cap the number of schedules. Goal: What should the final outcome look like, ideally? Each workflow should have a max permitted number of active launch plans associated with it. Describe alternatives you've considered Per-project cap on active workflows (this is less granular) Flyte component ☐ Overall ☐ Flyte Setup and Installation scripts ☐ Flyte Documentation ☐ Flyte communication (slack/email etc) ☐ FlytePropeller ☐ FlyteIDL (Flyte specification language) ☐ Flytekit (Python SDK) ☑︎ FlyteAdmin (Control Plane service) ☐ FlytePlugins ☐ DataCatalog ☐ FlyteStdlib (common libraries) ☐ FlyteConsole (UI) ☐ Other [Optional] Propose: Link/Inline All summarized above Additional context N/A Is this a blocker for you to adopt Flyte N/A flyteorg/flyte

#638 [Feature] Task & Workflow interface evolution to allow ease of sharability. Issue created by kumare3 Motivation: Why do you think this is important? Flyte tasks and workflows can be shared across projects. A desirable property is to use the latest version of a task when using a shared platform task. this would allow platform owners to deploy new versions of tasks without needing to re-compile all the workflows and still get newer versions Goal: What should the final outcome look like, ideally? USers can keep using new versions of a task (at registration time) and get the latest copy of the task even when the interface was evolved. This could be done like python functions with default values. Describe alternatives you've considered 2 alternatives: 1. all users lock a version of the task locally. they manually update the version and update the corresponding usage 2. for every new release of shared task, we recompile all the workflows - this may not always be possible, but we can introdce default values at usage point - This is extremely complex to do correctly Flyte component ☐ Overall ☐ Flyte Setup and Installation scripts ☐ Flyte Documentation ☐ Flyte communication (slack/email etc) ☐ FlytePropeller ☑︎ FlyteIDL (Flyte specification language) ☑︎ Flytekit (Python SDK) ☐ FlyteAdmin (Control Plane service) ☐ FlytePlugins ☐ DataCatalog ☐ FlyteStdlib (common libraries) ☐ FlyteConsole (UI) ☐ Other [Optional] Propose: Link/Inline RFP to be done flyteorg/flyte

#641 [Feature] Use generic cron expressions in FlyteAdmin Issue created by katrogan Motivation: Why do you think this is important? Currently FlyteAdmin uses CloudWatch cron expressions which ties us to a specific cloud provider rather than convention. Furthermore flyteorg/flyteidl#83 added a richer means of encoding cron expressions. FlyteAdmin should read this field, in addition to the deprecated

string cron_expression

one so that sdk users can eventually move to using the richer

CronSchedule

message. Goal: What should the final outcome look like, ideally? FlyteAdmin should support reading from the

CronSchedule

field in Schedule messages and the sdk should expose this to end users to configure (and default to 0 as the offset always given that it is used specifically by Styx). Describe alternatives you've considered the current impl Flyte component ☐ Overall ☐ Flyte Setup and Installation scripts ☐ Flyte Documentation ☐ Flyte communication (slack/email etc) ☐ FlytePropeller ☑︎ FlyteIDL (Flyte specification language) ☐ Flytekit (Python SDK) ☑︎ FlyteAdmin (Control Plane service) ☐ FlytePlugins ☐ DataCatalog ☐ FlyteStdlib (common libraries) ☐ FlyteConsole (UI) ☐ Other [Optional] Propose: Link/Inline If you have ideas about the implementation please propose the change. If inline keep it short, if larger then you link to an external document. Additional context Related #529 Is this a blocker for you to adopt Flyte N/A flyteorg/flyte

#670 [BUG] Extensive regression testing suite for the compiler Issue created by EngHabu Describe the bug Here Expected behavior A clear and concise description of what you expected to happen. Flyte component ☐ Overall ☐ Flyte Setup and Installation scripts ☐ Flyte Documentation ☐ Flyte communication (slack/email etc) ☑︎ FlytePropeller ☐ FlyteIDL (Flyte specification language) ☐ Flytekit (Python SDK) ☐ FlyteAdmin (Control Plane service) ☐ FlytePlugins ☐ DataCatalog ☐ FlyteStdlib (common libraries) ☐ FlyteConsole (UI) ☐ Other flyteorg/flyte

#675 [Feature] internal k8s error messages are actionable for general users Issue created by slai Motivation: Why do you think this is important? When an error occurs with k8s starting or running a task, the resulting error message in Flyte Console is often cryptic and unactionable. Some examples - •

Pod reported success despite being OOMKilled

, when a process in the task pod was OOMKilled but not pyflyte so task still completes. Reporting success despite a bad condition like OOMKilled is confusing - did the task succeed or not? What were the memory request/limits set? Which process was OOMKilled? •

containers with unready status: [execution_id]|context deadline exceeded

, generally when a container image cannot be pulled from the registry. This is a combination of an unclear k8s message (why is the container unready?) and a Go specific error that a general user wouldn't understand (context deadline exceeded, i.e. timed out waiting). What should the user do? •

[3/3] currentAttempt done. Last Error: SYSTEM::object [execution_id] terminated in the background, manually

, generally when the task pod was on an instance that was spot pre-empted or otherwise removed. What does 'currentAttempt done' mean, what does 'terminated in the background, manually' mean - what manually terminated it? This error is actually benign in most cases because the retry should succeed, but the message gives no indication of that Goal: What should the final outcome look like, ideally? The error message should strike a balance between - • being understandable for the general user (little k8s knowledge) • contain enough info for an engineer to troubleshoot • be actionable without being too prescriptive and eliminating other causes For example, the first error message above could be -

Task ended with one or more processes killed due to lack of memory. Current memory request is X, limit is Y. Process killed was Z. Consider increasing the memory request/limit of this task.

Alternatively, maybe an error code that a user can look up elsewhere with more info would be a better way to keep the necessary k8s detail in one place, and the general user explanation in another. Flyte component ☐ Overall ☐ Flyte Setup and Installation scripts ☑︎ Flyte Documentation ☐ Flyte communication (slack/email etc) ☐ FlytePropeller ☐ FlyteIDL (Flyte specification language) ☐ Flytekit (Python SDK) ☐ FlyteAdmin (Control Plane service) ☑︎ FlytePlugins ☐ DataCatalog ☐ FlyteStdlib (common libraries) ☑︎ FlyteConsole (UI) ☐ Other Additional context See https://flyte-org.slack.com/archives/CNMKCU6FR/p1611329101012800 for some further discussion. Also seems like #512 and #535 are similar issues. Is this a blocker for you to adopt Flyte Nope, already a user :) flyteorg/flyte

#698 [Docs] Separate Flyte Admin Service.proto into multiple files Issue created by kumare3 Motivation: Why do you think this is important? Currently the entire service API specification is one file. It would be much better to split it into multiple files, as this with automatically separate the generated documentation here More than documentation, it is also easier to reason about the service and eventually if required decompose into multiple processes (though this is not a strict requirement) Goal: What should the final outcome look like, ideally? Separate documents for execution_service.proto workflow_service.proto task_service.proto etc Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered. Flyte component ☐ Overall ☐ Flyte Setup and Installation scripts ☑︎ Flyte Documentation ☐ Flyte communication (slack/email etc) ☐ FlytePropeller ☑︎ FlyteIDL (Flyte specification language) ☐ Flytekit (Python SDK) ☐ FlyteAdmin (Control Plane service) ☐ FlytePlugins ☐ DataCatalog ☐ FlyteStdlib (common libraries) ☐ FlyteConsole (UI) ☐ Other flyteorg/flyte

#705 [Feature] [Plugin] Redshift Issue created by EngHabu Motivation: Why do you think this is important? Redshift is a common database favored for small databases (e.g. for analytics) due to its low latency query execution. Goal: What should the final outcome look like, ideally? An easy to use FlyteKit plugin for users to write Redshift queries and a FlytePlugins backend plugin to send and monitor jobs. AWS Redshift Data API FlytePlugins WebAPI AWS Athena Plugin FlyteKit Hive Plugin Flyte component ☐ Overall ☐ Flyte Setup and Installation scripts ☐ Flyte Documentation ☐ Flyte communication (slack/email etc) ☐ FlytePropeller ☐ FlyteIDL (Flyte specification language) ☑︎ Flytekit (Python SDK) ☐ FlyteAdmin (Control Plane service) ☑︎ FlytePlugins ☐ DataCatalog ☐ FlyteStdlib (common libraries) ☐ FlyteConsole (UI) ☐ Other flyteorg/flyte

#706 [Feature] [Plugins] Data Catalog Issue created by EngHabu Motivation: Why do you think this is important? DataCatalog is Flyte's service to provide data lineage, tracking and memoization for workflow executions. Goal: What should the final outcome look like, ideally? A flytekit and a backend webAPI plugins to retrieve and push artifacts to DataCatalog. Not an exact example:

@workflow
def my_wf():
    # pull data from datacatalog
    a = datacatalog.query(tag="abc")
    x = my_task(input=a)

    # push data to datacatalog
    datacatalog.publish(dataset=x, tag="xyz")

FlytePlugins WebAPI AWS Athena Plugin Flyte component ☐ Overall ☐ Flyte Setup and Installation scripts ☐ Flyte Documentation ☐ Flyte communication (slack/email etc) ☐ FlytePropeller ☐ FlyteIDL (Flyte specification language) ☑︎ Flytekit (Python SDK) ☐ FlyteAdmin (Control Plane service) ☑︎ FlytePlugins ☐ DataCatalog ☐ FlyteStdlib (common libraries) ☐ FlyteConsole (UI) ☐ Other flyteorg/flyte

#707 [Feature] [Plugins] EMR Hive/Spark Issue created by EngHabu Motivation: Why do you think this is important? AWS EMR offers a Hadoop managed service to run hive and containerized spark workloads that promises scalability and less management overhead. Goal: What should the final outcome look like, ideally? backend webAPI plugins to run hive and spark workloads on AWS EMR. Ideally these plugins should not require flytekit work and should be able to leverage existing spark/hive protos to target EMR. Example for AWS Athena plugin AWS EMR Spark Example Current k8s-based Spark Plugin Current Hive plugin Flyte component ☐ Overall ☐ Flyte Setup and Installation scripts ☐ Flyte Documentation ☐ Flyte communication (slack/email etc) ☐ FlytePropeller ☐ FlyteIDL (Flyte specification language) ☑︎ Flytekit (Python SDK) ☐ FlyteAdmin (Control Plane service) ☑︎ FlytePlugins ☐ DataCatalog ☐ FlyteStdlib (common libraries) ☐ FlyteConsole (UI) ☐ Other flyteorg/flyte

#710 [Feature] [FlyteCtl] CRUD-* operations for Datacatalog Issue created by EngHabu Motivation: Why do you think this is important? Flytectl is flyte's CLI to interact with the backend. It's quickly adding a ton of features. One set of missing operations is interactions with DataCatalog service. Flyte component ☐ Overall ☐ Flyte Setup and Installation scripts ☐ Flyte Documentation ☐ Flyte communication (slack/email etc) ☐ FlytePropeller ☐ FlyteIDL (Flyte specification language) ☐ Flytekit (Python SDK) ☐ FlyteAdmin (Control Plane service) ☐ FlytePlugins ☐ DataCatalog ☐ FlyteStdlib (common libraries) ☐ FlyteConsole (UI) ☑︎ FlyteCtl ☐ Other flyteorg/flyte

#740 [BUG] Make the error message reported to Admin include the original error in upload error cases Issue created by wild-endeavor If there's an error in the execution of a task, it should produce an errors.pb file. This is fine. This errors.pb file is then uploaded to S3, using the aws cli. If there's an error, the error message shown in the UI doesn't show it, it only shows the error uploading the file. https://kubernetes.io/docs/tasks/debug-application-cluster/determine-reason-pod-failure/#customizing-the-termination-message flyteorg/flytekit#368 flyteorg/flyte

#770 [Core Feature]Automatic deduction of Cache Version for certain task types Issue created by kumare3 Motivation: Why do you think this is important? Flytekit today relies on the Cache Version to be specified always. This can be error prone. But, in some cases deducing the Cache Version is completely possible. For example in the case of SQL Queries or Instance tasks, or sagemaker built-in algorithm tasks etc. Flytekit should allow some task types to automatically deduce the cache version Goal: What should the final outcome look like, ideally? Users should not need to specify cache version of certain task types. It should be automatically de-ducted. But, cacheable should be optional. Describe alternatives you've considered Implement the cache version on the user side. This is complicated and more error prone flyteorg/flyte

#786 [Flytectl Feature] Parallel to kubectl explain Issue created by wild-endeavor kubectl has an explain, super detailed help. We should consider having something like this as well.

kubectl explain deployment
KIND:     Deployment
VERSION:  apps/v1

DESCRIPTION:
     Deployment enables declarative updates for Pods and ReplicaSets.

FIELDS:
   apiVersion   <string>
     APIVersion defines the versioned schema of this representation of an
     object. Servers should convert recognized schemas to the latest internal
     value, and may reject unrecognized values. More info:
     <https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources>

   kind <string>
     Kind is a string value representing the REST resource this object
     represents. Servers may infer this from the endpoint the client submits
     requests to. Cannot be updated. In CamelCase. More info:
     <https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds>

   metadata     <Object>
     Standard object metadata.

   spec <Object>
     Specification of the desired behavior of the Deployment.

   status       <Object>
     Most recently observed status of the Deployment.

flyteorg/flyte

#791 [Flytekit] FlyteFile and FlyteDirectory randomness needs to be fixed Issue created by wild-endeavor This is for @cosmicBboy, whom I cannot assign. The Flyte IDL has a type/corresponding literal that's supposed to represent offloaded binary data. These are weird names for users. Instead, flytekit takes a single part blob to mean a file and a multi-part blob to mean a directory. There is a transformer that will help translate between a file on the filesystem and Blob type. The contents are uploaded to S3 (or gcs or whatever) when converting to a Literal and they're downloaded to the os FS when converting to a Python file. In both cases, a call to get a random path is used https://github.com/flyteorg/flytekit/blob/master/flytekit/types/file/file.py#L244 and https://github.com/flyteorg/flytekit/blob/master/flytekit/types/file/file.py#L261. This is fine but if a user repeatedly calls one or the other, new paths should not be returned... we should memoize the path in the object the first time each of these is used. flyteorg/flyte

#797 [Feature] [Flytekit] Make testing mechanisms work with launch plans Issue created by wild-endeavor Because of the way the

LaunchPlan

class does the patching, overwriting the execute function effectively, it doesn't work on launch plans today (which don't have an

execute

function because *call*'s are dispatched to the underlying workflow). Can you try to think of a way around this? Also as part of this, let's rename the

task_mock

function as well, it's already a misnomer because it's already capable of patching workflows. Think of an appropriate name, but also, let's keep the

task_mock

alias in

flytekit/__init__.py

so that we don't break existing users. flyteorg/flyte

#817 [Plugin] BigQuery plugin Issue created by kanterov Why would this plugin be helpful to the Flyte community BigQuery is a fully-managed, serverless data warehouse that enables scalable analysis over petabytes of data. It is a Platform as a Service that supports querying using ANSI SQL. It also has built-in machine learning capabilities. • query job task flyteorg/flyteplugins#161 • copy job task • load job task • extract job task Type of Plugin ☐ Python/Java interface only plugin ☑︎ Web Service (e.g. AWS Sagemaker, GCP DataFlow, Qubole etc...) ☐ Kubernetes Operator (e.g. TfOperator, SparkOperator, FlinkK8sOperator, etc...) ☐ Customized Plugin using native kubernetes constructs ☐ Other Can you help us with the implementation? ☑︎ Yes ☐ No flyteorg/flyte

#818 [Plugin] Google Dataflow Issue created by kanterov Why would this plugin be helpful to the Flyte community Google Cloud Dataflow is a fully managed service for executing Apache Beam pipelines within the Google Cloud Platform ecosystem. Type of Plugin ☐ Python/Java interface only plugin ☑︎ Web Service (e.g. AWS Sagemaker, GCP DataFlow, Qubole etc...) ☐ Kubernetes Operator (e.g. TfOperator, SparkOperator, FlinkK8sOperator, etc...) ☐ Customized Plugin using native kubernetes constructs ☐ Other Can you help us with the implementation? ☑︎ Yes ☐ No flyteorg/flyte

#828 [Housekeeping] Clean up function names in marshal_utils Issue created by katrogan Describe the Issue A lot of different permutations for marshaling structs and protos exist in https://github.com/flyteorg/flyteplugins/blob/master/go/tasks/pluginmachinery/utils/marshal_utils.go. It's probably worth revisiting the naming of all these functions to make them less confusing What if we do not do this? marshall_utils utils stay confusing Related component Plugins flyteorg/flyte

#839 [Core Feature]Migrate the general Datacatalog go-client to flyteidl Issue created by kumare3 Motivation: Why do you think this is important? Datacatalog is a gRPC service. Currently the only go-client for datacatalog is in fytepropeller. But there is motivation in the community to create a simple CLI using flytectl. For this it would be great to have a central place to house the datacatalog go-client. FlyteIDL already houses the gRPC protobuf definition. Goal: What should the final outcome look like, ideally? FlyteCTL can simply introduce a dependency and the client should work Proposal There is a PR that started some of this work - https://github.com/flyteorg/flyteidl/pull/2/files FlytePropeller uses a higher level client for its own usage - https://github.com/flyteorg/flytepropeller/tree/master/pkg/controller/nodes/task/catalog FlyteAdmins client can be used as a template https://github.com/flyteorg/flyteidl/tree/master/clients/go/admin flyteorg/flyte

#854 [BUG] Flytekit entity patch command interferes with mock.patch Issue created by wild-endeavor Describe the bug This short test file will reproduce the issue. If you copy paste this into a test file, each of the two tests will succeed on its own. However if you run it together as a file in a pytest command, it will break.

import pytest
from mock import patch as _system_patch

from flytekit.core.task import task
from flytekit.core.testing import patch as flyte_patch
from flytekit.core.workflow import ImperativeWorkflow, workflow


@task
def t1(a: str) -> str:
    return a + " world"


wb = ImperativeWorkflow(name="my.workflow")
wb.add_workflow_input("in1", str)
node = wb.add_entity(t1, a=wb.inputs["in1"])
wb.add_workflow_output("from_n0t1", node.outputs["o0"])


@_system_patch("flytekit.core.workflow.ImperativeWorkflow.execute")
def test_return_none_errors(mock_execute):
    mock_execute.return_value = None
    with pytest.raises(Exception):
        wb(in1="hello")


@flyte_patch(wb)
def test_imperative_patching(mock_wb):
    mock_wb.return_value = "hi"

    @workflow
    def my_functional_wf(a: str) -> str:
        x = wb(in1=a)
        return x

    assert my_functional_wf(a="hello") == "hi"

It is unclear what is happening. Somehow, the

flyte_patch

patching logic is interfering with the normal

mock.patch

. The mere presence of the second test, basically makes it seem like the

mock.patch

isn't there on the first test. Basically

ImperativeWorkflow.execute

is not patched and returns what it normally would return, and hence the exception isn't raised. If you copy paste the

wb

workflow and use a new variable for it for the second test, then things pass. This will take some digging into the mock library to understand how they're interfering with each other. flyteorg/flyte

#878 [Core Feature] Allow Some Dynamic Sub-workflows to Fail Issue created by michaels-lyft Motivation: Why do you think this is important? We have a lot of scenarios where there's a dynamic_task that executes a list of sub-workflows or sub-dynamic_tasks. Today, when one of these sub-executions fail, it would cause the parent dynamic_task to fail even with

FAIL_AFTER_EXECUTABLE_NODES_COMPLETE

enabled on the parent workflow or

allowed_failure_ratio

enabled on the dynamic_task (which only works with sub-python_tasks). This aborts any of the dependent downstream tasks automatically. But sometimes, we would like to allow certain % of the tasks to fail since we may not always need 100% of the output. Goal: What should the final outcome look like, ideally? There should be an option to allow a % of the sub-workflows or sub-dynamic_tasks to fail. Describe alternatives you've considered Gone through the discussions at #191. Tried using

FAIL_AFTER_EXECUTABLE_NODES_COMPLETE

and

allowed_failure_ratio

but neither works in this scenario. [Optional] Propose: Link/Inline OR Additional context I can think of two ways to do this: 1. Expand

allowed_failure_ratio

to include sub-workflows and sub-dynamic_tasks 2. Add something similar to a

try-catch

in the dynamic_task that we could use to catch any sub-execution failures and make the parent task succeed (as suggested here). flyteorg/flyte

#897 [BUG] Add retries in flytekit for network related errors Issue created by migueltol22 Describe the bug Intermittent network issues are to be expected. In the case of receiving a network related error response flytekit should retry the same request (e.g. serialization, executing workflow, listing workflows, etc). Example Error to retry on:

[2021-04-07T04:45:51Z] 	status = StatusCode.UNAVAILABLE
[2021-04-07T04:45:51Z] 	details = "upstream connect error or disconnect/reset before headers. reset reason: connection failure"
[2021-04-07T04:45:51Z] 	debug_error_string = "{"created":"@1617770750.462629552","description":"Error received from peer","file":"src/core/lib/surface/call.cc","file_line":1039,"grpc_message":"upstream connect error or disconnect/reset before headers. reset reason: connection failure","grpc_status":14}"

Expected behavior On Network related error responses Flytekit should retry the request. This retry should follow a specified retry policy whether that be exponential back offs, incremental intervals, randomization or some other retry policy. [Optional] Additional context To Reproduce Steps to reproduce the behavior: 1. 2. Screenshots If applicable, add screenshots to help explain your problem. flyteorg/flyte

#612 Add supportTaskTypes for agentservice without write it in config twice. Pull request opened by Future-Outlier TL;DR As title. Note: Please merge the code below, so that the code will work! flyteorg/flyteplugins#398 Type ☐ Bug Fix ☑︎ Feature ☑︎ Plugin Are all requirements met? ☑︎ Code completed ☐ Smoke tested ☑︎ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description Right now, the agent-service section doesn't need to add supportedTaskTypes anymore. We can get it by "default-for-task-types" from tasks. Tracking Issue flyteorg/flyteplugins#398 flyteorg/flytepropeller GitHub Actions: Build & Push Flytepropeller Image GitHub Actions: Goreleaser GitHub Actions: Bump Version ✅ 11 other checks have passed 11/14 successful checks

#398 Add supportTaskTypes for agentservice without write it in config twice. Pull request opened by Future-Outlier TL;DR As title. Type ☐ Bug Fix ☑︎ Feature ☑︎ Plugin Are all requirements met? ☑︎ Code completed ☐ Smoke tested ☑︎ Unit tests added ☐ Code documentation added ☐ Any pending items have an associated Issue Complete description Right now, the agent-service section doesn't need to add supportedTaskTypes anymore. We can get it by "default-for-task-types" from tasks. Tracking Issue flyteorg/flytepropeller#612 flyteorg/flyteplugins ✅ All checks have passed 7/7 successful checks

#943 [Housekeeping] Supporting archival for launchplans and tasks Issue closed as not planned by github-actions[bot] flyteorg/flyte

#1002 [flytekit-java] Support WorkflowNode Issue created by kanterov Why would this be helpful to the Flyte community WorkflowNode allows referencing sub-workflows or other launch plans. WorkflowNode should work with local execution as well. It should be possible to reference both

SdkWofkflow

,

SdkLaunchPlan

, and

SdkRemoteWorkflow

(to be added, similarly to

SdkRemoteTask

). flyteorg/flyte

#1003 [flytekit-java] Support dynamic workflows Issue created by kanterov Why would this plugin be helpful to the Flyte community Dynamic workflows allow to access input values during graph construction. In flytepropeller, they are implemented as tasks that produce workflow graphs, however, from the user perspective they are workflows. Presumably, the interface can look like this:

public abstract class SdkDynamicWorkflow<InputT> {
  public abstract void expand(SdkWorkflowBuilder builder, InputT input);
}

flyteorg/flyte

#1004 [flytekit-java] Add type parameter to SdkBindingData Issue created by kanterov Why would this be helpful to the Flyte community

SdkBindingData

should become

SdkBindingData<T>

, where

T

should reflect the corresponding Java or Scala type. This will allow surface type information only available in workflow graph as type information embedded into Java/Scala types. We should update all existing methods to preserve type information where it's possible, for instance:

class SdkBindingData {
  static SdkBindingData<String> literalOfString(String value);
}

flyteorg/flyte