#2261 [BUG] pyflyte fails with import error with python version 3.9.10 Issue created by AlekhyaSasi Describe the bug When packaging the workflow, with Python version 3.9.10 & protobuf version 3.19.4,

pyflyte

fails with

ImportError: cannot import name '_message' from 'google.protobuf.pyext' (/opt/homebrew/lib/python3.9/site-packages/google/protobuf/pyext/__init__.py)

This seems to work with Python version 3.9.7. Expected behavior pyflyte should succeed packaging. Additional context to reproduce After completing the

build

part of getting-started: https://docs.flyte.org/en/latest/getting_started/build.html 1.

brew install flyteorg/homebrew-tap/flytectl

2.

flytectl sandbox start --source .

3.

flytectl sandbox exec -- docker build . --tag "myapp:v1"

4.

pyflyte --pkgs flyte.workflows package --image "myapp:v1"

Screenshots Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#2257 [Docs] Improve FlyteKit configuration documentation Issue created by AlekhyaSasi Description We need to include detailed documentation for all the configuration options available for flytekit. https://docs.flyte.org/projects/flytekit/en/latest/configuration.html Slack archives Edit: The original intent of this issue is to better document how to write flyte.config toml files (see flytekit unit tests), as there's no place in the flytesnacks/flytekit API reference that shows what all the valid values are. The purpose of this issue is to create a page or pages on what the different configuration options are. Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#2243 [BUG] Flyteadmin should check launch plans when an execution of the same name already exists Issue created by katrogan Describe the bug Currently, creating an execution within a project and domain with an already claimed name returns "AlreadyExists" even if the corresponding execution was triggered by a completely different launch plan. FlyteAdmin should differentiate between duplicate executions and naming collisions. Expected behavior An AlreadyExists error should clarify whether the existing execution was launched using the same launch plan. Additional context to reproduce No response Screenshots No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#2252 [BUG] flytectl: Partial errors during register files is considered success Issue created by ariefrahmansyah Describe the bug In the middle of

flytectl register files

, some file registration is failed. These files are in the middle of the queue.

$ flytectl register files ...

...
25_pipeline.workflows.workflow.workflow.pipeline_workflow_3.pb            | Success | AlreadyExists
26_pipeline.workflows.workflow.workflow.pipeline_dry_run_workflow_2.pb    | Failed  | Error registering file due to rpc error: code = InvalidArgument desc = workflow with different structure already exists with id resource_type:WORKFLOW...
27_pipeline.workflows.workflow.workflow.pipeline_dry_run_workflow_3.pb    | Success | AlreadyExists
28_pipeline.workflows.workflow.launchplan.pipeline_workflow_schedule_3.pb | Success | AlreadyExists
...

The example snippet above returns a non-0 exit code and no error is displayed. Note that I also didn't use

--continueOnError

flag. Expected behavior I think the bug is due to this looping condition: https://github.com/flyteorg/flytectl/blob/v0.5.7/cmd/register/files.go#L140. The fix depends on how flytectl handles this. Is flytectl: 1. Stop registering files, return non-0 exit code, or 2. Continue registering remaining files, but still return non-0 exit code I prefer option 1 since it's expected behaviour if the user didn't specify

--continueOnError

flag. Additional context to reproduce No response Screenshots No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#2242 [Housekeeping] Single message for execution parameters Issue created by katrogan Describe the issue Currently execution-time parameters such as • labels • annotations • security context • raw output data config • max parallelism • notifications can all be specified in the launch plan spec, the create execution request spec, and soon, as a matchable attribute. It would be useful to consolidate all of these parameters into a single, re-usable protobuf message definition that will make adding new parameters a breeze. What if we do not do this? Any new execution parameters will have to be added piecemeal to the LaunchPlanSpec, ExecutionSpec and WorkflowExecutionConfig matchable entity. Adding the single message will require deprecating all the current fields but this would be nice to clean-up before release 1.0.0 Related component(s) flyteidl Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#2222 [Docs] Grammatical issues in files generated from helm Issue created by AlekhyaSasi Description #2169 (comment) For example: Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#2213 [FlyteCTL Feature] Support get execution --launch_spec flag Issue created by ckiosidis Describe the feature/command for FlyteCTL Users should be able to extract the Launch Spec in a way that is usable from the create execution command.

flytectl get execution xyz --launch_spec -p projectabc -d production > exec.yaml
flytectl create execution -p projectabc -d DEVELOPMENT --execFile exec.yaml

Provide a possible output or UX example The launch spec in a way that is readable and understandable by the

flytectl create execution ...

Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#2207 [Core feature] Improve FlyteRemote printing Issue created by wild-endeavor Motivation: Why do you think this is important? Improve printing Goal: What should the final outcome look like, ideally? Pretty. Describe alternatives you've considered None Propose: Link/Inline OR Additional context No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#2193 [Core feature] Add async API for getting entire workflow state including node execution Issue created by pmahindrakar-oss Motivation: Why do you think this is important? As a flyte API user for getting execution details, they have to be aware about following additional API's • ListNodeExecutions • ListTaskExecutions • GetNodeExecutionData Along with the regular GetExecution API Also there is structure to how these API calls are to be made based on the parent child relationships between the node executions and also if a node is dynamic or subworflow node. This complexity is implemented in all the clients right now including UI and the CLI. Would be great to support one async API to gather all this info on the Admin side. Create a async process in admin, that consolidates the entire state into (a set of ) protobuf messages (that can be chained) on completion of an execution. The API, then should return this object. For completed workflows the return can be really fast. For running workflows, we can actually perform the queries. Another option could be to lazily cache as workflows are invoked Goal: What should the final outcome look like, ideally? Async Admin API to trigger gather of overall state of the execution and using the same or different API to get the state details Describe alternatives you've considered Current way exists to gather this info in UI and CLI which is complex Propose: Link/Inline OR Additional context No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#2186 [BUG] Possible phase overwrite Issue created by katrogan Describe the bug Currently when flyteadmin receives an execution event it first fetches the existing execution record and validates the event is making progress on a valid phase change. However, in the interim a subsequent event could be sent and a possible race condition occurs where one is recorded even though the first event could presumably be written last. So you could get something likee db: RUNNING event 1: ABORTING event 2: ABORTED event 2 is written after event 1 validation succeeds but before event 1 is fully recorded. so the final state is incorrectly ABORTING. Expected behavior Database writes should use transactions to guarantee phase transition constraints. Additional context to reproduce No response Screenshots No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#2142 [Core feature] Flytectl should generate matchable attribute template files Issue created by katrogan Motivation: Why do you think this is important? The guidance from flytectl says that to update a matchable attribute, a user should get the attribute, have flytectl write it as a yaml file, update the yaml file and then use the flytectl update command with the filepath as an arg. For example: https://github.com/flyteorg/flytectl/blob/master/cmd/update/matchable_plugin_override.go#L17 This leads to problems where no existing matchable attribute exists. Because the update API is an upsert, creating a matchable attribute in the first place becomes very difficult. Requiring users to know the protobuf structure in order to hand generate a yaml file is undesirable. It would be much simpler if flytectl could generate a template file that a user could modify and then pass to the update command. Goal: What should the final outcome look like, ideally? It should be simple to create a matchable attribute to pass when calling flytectl update Describe alternatives you've considered current implementation Propose: Link/Inline OR Additional context No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#2140 [Core feature] Update cookie cutter to include a github workflow Issue created by EngHabu Motivation: Why do you think this is important? Flyte comes with a Cookie Cutter repo to bootstrap a flyte repo for our users. One thing that might help users validate workflows quicker is automatically setting up a workflow that does some basic validations... Goal: What should the final outcome look like, ideally? An OOB workflow that comes with cookie cutter repo to test the registration flow. Describe alternatives you've considered Docs Propose: Link/Inline OR Additional context No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#2123 The Flyte console is available to anyone who has a DNS address. Issue created by anmyachev This behavior can be observed when deploying Flyte to AWS using OPTA. Is there a way to deploy Flyte to the cloud so that only a limited number of people have access to the console? With passwords, VPC, or whatever... flyteorg/flyte

#2120 [Housekeeping] Support protoc-gen-doc to generate json tags with camel case Issue created by pmahindrakar-oss Describe the issue Current version of protoc-gen-doc being used generates the snake case version of the fields for json marshalling

type ClusterConfig struct {
	state         protoimpl.MessageState
	sizeCache     protoimpl.SizeCache
	unknownFields protoimpl.UnknownFields

	// List of plugins that this cluster can run.
	EnabledPlugins []string `protobuf:"bytes,1,rep,name=enabled_plugins,json=enabledPlugins,proto3" json:"enabled_plugins,omitempty"`
	// List of projects for which this cluster can schedule executions.
	EnabledProjects []string `protobuf:"bytes,2,rep,name=enabled_projects,json=enabledProjects,proto3" json:"enabled_projects,omitempty"`
}

Would be great if this behavior is customizable. Currently the flytectl uses camelcase for all its commandline flags In some cases where we want to allow users to read/write such configs from json/yaml files and pass them to cli, then in the existing behavior it results in empty attributes due to this diff. eg : while defining the commandline flag counterpart for the above ClusterConfig we define this way to match up with the json tag in the generated definition of ClusterConfig

type UpdateClusterConfig struct {
	DryRun            bool   `json:"dryRun" pflag:",execute command without making any modifications."`
	ClusterConfigFile string `json:"clusterConfigFile" pflag:",file containing the update cluster config parameters."`
	EnabledPlugins    []string `json:"enabled_plugins" pflag:",comma separated list of plugins to be enabled on the cluster."`
	EnabledProjects   []string `json:"enabled_projects" pflag:",comma separated list of projects to be enabled on the cluster."`
}

Expected to keep the cli config using camel case and still be able to use ClusterConfig to write a yaml file in camelCase

type UpdateClusterConfig struct {
	DryRun            bool   `json:"dryRun" pflag:",execute command without making any modifications."`
	ClusterConfigFile string `json:"clusterConfigFile" pflag:",file containing the update cluster config parameters."`
	EnabledPlugins    []string `json:"enabledPlugins" pflag:",comma separated list of plugins to be enabled on the cluster."`
	EnabledProjects   []string `json:"enabledProjects" pflag:",comma separated list of projects to be enabled on the cluster."`
}

What if we do not do this? mixture of camel case and snake case or add customized unmarshalling Related component(s) No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#2115 [Housekeeping] Handle dependencies dropping support for python 3.7 Issue created by eapolinario Describe the issue Both pandas (starting on version 1.4.0) and numpy (starting on version 1.22.0) dropped support for python 3.7, which is supposed to reach EOL only on June-2023 (according to https://www.python.org/dev/peps/pep-0537/#id27). This trend will probably apply to other libraries as well. What if we do not do this? In CI we strive to support the combination of latest versions of components. If we restrict the versions of numpy and pandas we won't be able to catch errors in newer versions. Also, security reviews will flag the docker images we maintain for each version, pointing to these outdated versions. Related component(s) flytekit Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#2108 [BUG][Flytekit] `FlyteWorkflowExecution` `inputs` and `outputs` property return `None` Issue created by mdjong1 Describe the bug When using

FlyteRemote

with the

fetch_workflow_execution

function we can correctly retrieve most of the information related to a workflow execution (i.e. version, closure information). However, when we try to access the inputs and output from here the result is a

None

, despite knowing/seeing that the execution has inputs & outputs in the Flyte Console. Expected behavior I would expect that when I call

.inputs

or

.outputs

on a

FlyteWorkflowExecution

it would

Returns the inputs to the execution in the standard python format as dictated by the type engine.

(taken from docstring) as opposed to returning a

None

when inputs and outputs are actually set for the specific workflow execution. For cases when no inputs or outputs are set perhaps an empty dict would make more sense here as well. Additional context to reproduce

remote = FlyteRemote(flyte_admin_url=_FLYTE_HOST, insecure=False)

execution = remote.fetch_workflow_execution(project=_PROJECT_NAME, domain=_PROJECT_DOMAIN, name=_EXECUTION_NAME)

print(execution.inputs)  # prints None

Screenshots Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#2103 [BUG] Failed to run spark job when using flytekit remote Issue created by pingsutw Describe the bug I am able to run spark jobs when using flytectl to register the workflow and trigger them on the flyte console. However, I got the below error messages when using the flytekit remote client to execute the workflow.

E0126 09:21:12.800044       1 workers.go:102] error syncing 'flytesnacks-development/f4fbbb51f24e6436695b': failed at 
Node[n0]. RuntimeExecutionError: failed during plugin execution, caused by: failed to execute handle for plugin [spark]: 
[BadTaskSpecification] invalid TaskSpecification [fields:{key:"sparkConf" value:{struct_value:{fields:
{key:"spark.driver.cores" value:{string_value:"1"}} fields:{key:"spark.driver.memory" value:{string_value:"1000M"}} fields:
{key:"spark.executor.cores" value:{string_value:"1"}} fields:{key:"spark.executor.instances" value:{string_value:"2"}} fields:
{key:"spark.executor.memory" value:{string_value:"1000M"}}}}}]., caused by: either MainApplicationFile or MainClass must 
be set

Expected behavior Remote clients should be able to execute the spark workflow Additional context to reproduce

from flytekit.remote import FlyteRemote
import spark_dataframe.workflows.example

remote = FlyteRemote(
        flyte_admin_url=localhost:30081,
        insecure=True,
        default_project="flytesnacks",
        default_domain="development",
        image_config=ImageConfig(
            default_image=Image(name="default", fqn="spark", tag="v1")
        )
    )
remote.execute(spark_dataframe.workflows.example.my_smart_schema, inputs={}, version=version, wait=False)

Screenshots Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#2101 [FlyteCTL Feature] Create Launch plan command Issue created by kumare3 Describe the feature/command for FlyteCTL It is not possible to create a launchplan from the CLI today. This is important as users may want o create launchplans and this shows that it possible async Provide a possible output or UX example

flytectl create launchplan --flle lp.yaml

Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#2097 [BUG] pyflyte: PermissionError: [Errno 13] Permission denied: '/tmp/flyte/20220124_215844' Issue created by anmyachev Describe the bug Failed command:

pyflyte --pkgs myapp.workflows package --image myapp:v1 -f

. The problem can be reproduced on a machine with multiple users. After the first user creates

tmp/flyte

folder, the second one will not be able to create anything in it (in case of different permissions). The problem part of the tmp path -

flyte/

. It seems to me that this line should be changed: https://github.com/flyteorg/flytekit/blob/05653c889a0f8f96a4f2424f8d475b9c95e36d74/flytekit/core/data_persistence.py#L428 Expected behavior Users on the same machine with different permissions should be able to work with

pyflyte

. Additional context to reproduce No response Screenshots No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#2079 [Docs] How to display Flyte metrics stored in the admin DB Issue created by veggiemonk Description As we discussed with @Ketan (kumare3), we were thinking about a way to extract metadata from the database. This could be useful for reporting and resources usage optimisation. Few things that we find cumbersome: A) Getting the data. Right now it leaves in the admin DB where production data are, and access control is more restricted. B) The schema of the DB is not so obvious. Some column have a name that is not so self explanatory. C) The queries are difficult to get right as they query a DAG. • One possibility for us at Spotify is to sync the data from the DB to BigQuery and do the analytics there. • Another possibility is to modify flyte-admin to add a command that can run a SQL query and expose the metrics to Prometheus. As the reporting metrics are not needed in near real time, modifying flyte-admin is a bit of work but with proper guidance/documentation people could contribute their queries upstream. Let me know what do you think? Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#2050 [Core feature] Project labels doesn't reflect on kubernetes namespace Issue created by yindia Motivation: Why do you think this is important? Currently if we create/update any project with labels then those labels only stay in flyte ecosystem, But it would be great if we can pass those labels in k8s namespaces. It's help full for management of different project in single cluster Goal: What should the final outcome look like, ideally? • Create project with labels • Project labels should reflect on Kubernetes namespace Describe alternatives you've considered NA Propose: Link/Inline OR Additional context No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#2025 [BUG] spark backend flyteplugin does not update pod security context Issue created by akumor Describe the bug The SparkApplication object created in Kubernetes by executing a Spark Flyte task with the spark-on-k8s-operator fails to include the configured pod security context. Expected behavior When

plugins.k8s.default-pod-security-context

is configured for flytepropeller, I expect to see that configuration reflected in

.spec.driver.podSecurityContext

and

.spec.executor.podSecurityContext

of the SparkApplication Kubernetes objects created from running Flyte tasks. Additional context to reproduce 1. Ensure you are running a kubernetes cluster with the spark-on-k8s-operator 2. Apply a configuration file for flytepropeller that includes `default-pod-security-context`:

plugins:
  k8s:
    default-cpus: 100m
    default-memory: 100Mi
    default-labels:
      <http://app.kubernetes.io/name|app.kubernetes.io/name>: flyte
    default-pod-security-context:
      sysctls:
        - name: net.ipv4.tcp_synack_retries
          value: "2"
  spark:
    # -- Spark default configuration
    spark-config-default:
      # We override the default credentials chain provider for Hadoop so that
      # it can use the serviceAccount based IAM role or ec2 metadata based.
      # This is more in line with how AWS works
      - spark.hadoop.fs.s3a.aws.credentials.provider: "com.amazonaws.auth.DefaultAWSCredentialsProviderChain"
      - spark.hadoop.mapreduce.fileoutputcommitter.algorithm.version: "2"
      - spark.master: "<k8s://https://kubernetes.default.svc:443>"

3. Run a workflow containing a spark task similar to:

@task(
    task_config=Spark(
        spark_conf={
            "spark.driver.memory": "1000M",
            "spark.executor.memory": "1000M",
            "spark.executor.cores": "1",
            "spark.executor.instances": "2",
            "spark.kubernetes.namespace": "flyte",
            "spark.kubernetes.driver.limit.cores": "1",
            "spark.kubernetes.executor.limit.cores": "1",
        }
    )
)
def spark_test() -> str:
    partitions = 50
    print("Starting Spark with Partitions: {}".format(partitions))
    n = 100000 * partitions
    sess = flytekit.current_context().spark_session
    count = (
        sess.sparkContext.parallelize(range(1, n + 1), partitions).map(f).reduce(add)
    )
    pi_val = 4.0 * count / n
    print("Pi val is :{}".format(pi_val))
    return f"Pi val is: {pi_val}"

4. Review the resulting SparkApplication object to see the missing podSecurityContext with:

$ kubectl get sparkapplication <workflow>-n0-0 -o yaml

Screenshots No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#2017 [FlyteCTL Bug] failed to perform fast register on sandbox Issue created by wenshutang Describe the feature/command for FlyteCTL I'm having issues when attempting to fast register Packinging using pyflyte

pyflyte -c sandbox.config --pkgs basic_wfs serialize --local-source-root ./ --image basic_wfs:v0 fast workflows -f _pb_output/

Fast register

flytectl register files  _pb_output/* -d development  -p piccadilly --version fastv2.0 Error: please check your Storage Config. It failed while uploading the source code. Failed to write data [56b] to path [fast/fastv2.0-fast42058067df98d17a434dc57fa62e2f3d.tar.gz].: PutObject, putting object: SerializationError: failed to decode REST XML response
        status code: 200, request id: 
caused by: XML syntax error on line 1: element <link> closed by </head>

I set up my flyte sandbox cluster using all default settings.

admin:
  # For GRPC endpoints you might want to use dns:///flyte.myexample.com
  endpoint: dns:///localhost:30081
  authType: Pkce
  insecure: true
logger:
  show-source: true
  level: 0
storage:
  connection:
    access-key: minio
    auth-type: accesskey
    disable-ssl: true
    endpoint: <http://localhost:30084>
    region: us-east-1
    secret-key: miniostorage
  type: minio
  container: "my-s3-bucket"

What am I missing here? Provide a possible output or UX example To properly build and fast register workflows to tighten iteration loops Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#2016 [BUG] FLytectl pagination should use server returned token instead of page number Issue created by pmahindrakar-oss Describe the bug Currently flytectl computes the token using Page number and limit provided by the user from the commandline https://github.com/flyteorg/flytectl/blob/64d399291fb6ed326c867eb5258fbf68de75f8ca/pkg/filters/util.go#L61 This will cause issues if the token implementation changes in flyteadmin. Instead flytectl should rely on the token returned in the ListResponse for flyte entities and have the user pass it directly for fetching next set of results. Expected behavior • flytectl get launchplans/worflows/tasks/projects --filter.limit O/p : Use the --filter.token server-returned-token to fetch the next set of results • flytectl get launchplans/worflows/tasks/projects --token server-returned-token-from-previous-query --filter.limit Additional context to reproduce No response Screenshots No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#2010 [Core feature] [FlyteAdmin] Cache completed workflow execution status to speed up UI pages Issue created by kumare3 Motivation: Why do you think this is important? Once a workflow reaches a terminal state, it does not get updated. Thus, we can simply cache the workflow execution status and some basic information in an LRU cache on each server. This should speed up access to these pages in Flyteconsole tremendously. Goal: What should the final outcome look like, ideally? The UI project/workflows page loads extremely fast. Describe alternatives you've considered Client side caching. This is already done, but we could ammortize the cache across multiple user sessions Propose: Link/Inline OR Additional context No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#1998 [Core feature] No stacktraces Issue created by sonjaer Motivation: Why do you think this is important? I find it hard to debug/understand where these logged errors comes from without a stacktrace

json: {
exec_id: "cz7vntanluzg2fck4yim"
src: "protobuf_store.go:77"
}
level: "error"
msg: "Failed to write to the raw store [<gs://flyte-production-storage/metadata/payment-risk-datasets/production/cz7vntanluzg2fck4yim/inputs>] Error: Failed to write data [172b] to path [metadata/payment-risk-datasets/production/cz7vntanluzg2fck4yim/inputs].: googleapi: Error 409: The metadata for object "metadata/payment-risk-datasets/production/cz7vntanluzg2fck4yim/inputs" was edited during the operation. Please try again., conflict"
ts: "2021-12-27T05:33:19Z"

json: {
exec_id: "so5a7vr6wrndjvk7h3g7"
node: "ffum5wni"
src: "node_execution_manager.go:120"
}
level: "error"
msg: "failed to fetch node execution for the parent node: project:"daily-sessions" domain:"production" name:"so5a7vr6wrndjvk7h3g7"  fivcmqky with err%!(EXTRA *errors.flyteAdminErrorImpl=unexpected error type for: dial tcp 10.201.108.5:5432: connect: connection refused)"
ts: "2021-12-28T02:21:24Z"

``` I am not very familiar with Go but it would be great to see a stacktrace being logged. Goal: What should the final outcome look like, ideally? Stacktrace when logging the error Describe alternatives you've considered None Propose: Link/Inline OR Additional context No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#1988 [Housekeeping] Publish multi arch images for flytesnacks Issue created by yindia Describe the issue The issue is releated to M1, We need multi arch images for flytesnacks example. What if we do not do this? flytesnacks example will not work on m1 Related component(s) No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#1968 [FlyteCTL Feature] Generate random version when register workflows Issue created by pingsutw Describe the feature/command for FlyteCTL Generate random version if users don't specify a version

(flytekit-3.9) ➜  flytectl git:(uuid) ✗  bin/flytectl register files <https://github.com/flyteorg/flytesnacks/releases/download/v0.2.226/snacks-cookbook-integrations-aws-athena.tar.gz> --archive -p flytesnacks -d development --config /Users/kevin/.flyte/config.yaml
 -------------------------------------------------------------------------------------------------------------- -------- ------------------------------------------------- 
| NAME (7)                                                                                                     | STATUS | ADDITIONAL INFO                                 |
 -------------------------------------------------------------------------------------------------------------- -------- ------------------------------------------------- 
| /tmp/register414934295/snacks-cookbook-integrations-aws-athena/0_sql.athena.no_io_1.pb                       | Failed | Error registering file due to rpc error: code = |
|                                                                                                              |        | InvalidArgument desc = missing version          |
 -------------------------------------------------------------------------------------------------------------- -------- ------------------------------------------------- 
| /tmp/register414934295/snacks-cookbook-integrations-aws-athena/1_athena.athena.no_io_wf_2.pb                 | Failed | Error registering file due to rpc error: code = |
|                                                                                                              |        | InvalidArgument desc = missing version          |
 -------------------------------------------------------------------------------------------------------------- -------- ------------------------------------------------- 
| /tmp/register414934295/snacks-cookbook-integrations-aws-athena/2_athena.athena.no_io_wf_3.pb                 | Failed | Error registering file due to rpc error: code = |
|                                                                                                              |        | InvalidArgument desc = missing version          |
 -------------------------------------------------------------------------------------------------------------- -------- ------------------------------------------------- 
| /tmp/register414934295/snacks-cookbook-integrations-aws-athena/3_sql.athena.w_io_1.pb                        | Failed | Error registering file due to rpc error: code = |
|                                                                                                              |        | InvalidArgument desc = missing version          |
 -------------------------------------------------------------------------------------------------------------- -------- ------------------------------------------------- 
| /tmp/register414934295/snacks-cookbook-integrations-aws-athena/4_athena.athena.manipulate_athena_schema_1.pb | Failed | Error registering file due to rpc error: code = |
|                                                                                                              |        | InvalidArgument desc = missing version          |
 -------------------------------------------------------------------------------------------------------------- -------- ------------------------------------------------- 
| /tmp/register414934295/snacks-cookbook-integrations-aws-athena/5_athena.athena.full_athena_wf_2.pb           | Failed | Error registering file due to rpc error: code = |
|                                                                                                              |        | InvalidArgument desc = missing version          |
 -------------------------------------------------------------------------------------------------------------- -------- ------------------------------------------------- 
| /tmp/register414934295/snacks-cookbook-integrations-aws-athena/6_athena.athena.full_athena_wf_3.pb           | Failed | Error registering file due to rpc error: code = |
|                                                                                                              |        | InvalidArgument desc = missing version          |
 -------------------------------------------------------------------------------------------------------------- -------- ------------------------------------------------- 
7 rows
Error: rpc error: code = InvalidArgument desc = missing version

Provide a possible output or UX example Improve UX when registering workflows Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#1958 [Housekeeping] Support gorm.ErrRecordNotFound as part of ErrorTransformer Issue created by pmahindrakar-oss Describe the issue With Gorm V2, gorm.ErrRecordNotFound error is thrown as part of tx.Error object. Add support for ErrRecordNotFound as part of the Errortransformer class More details are part of this discussion flyteorg/flyteadmin#287 (comment) What if we do not do this? Code duplication Related component(s) No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte

#1955 [Housekeeping] Support validations for gorm v2 Issue created by pmahindrakar-oss Describe the issue After upgrading to gorm V2 , the validations library is no longer supported The library uses v1 gorm https://github.com/qor/validations/blob/master/callbacks.go Its used in flyteadmin for gorm validations. https://github.com/flyteorg/flyteadmin/blame/master/pkg/repositories/config/postgres.go#L80 What if we do not do this? Add support for gorm v2 in the library or find another alternative Related component(s) No response Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte