Are there any examples of flyte scheduling workloads on CoiledCloud ?

Another question about passing secrets to Flyte workflow executions. I have a service that kicks off Flyte workflows on behalf of many users. In the Flyte workflow, I would like to access my service and identify which user is making the request. I was looking at the secrets documentation and it's not clear if that's going to fit use cases where different secrets need to be passed in for different executions. E.g. if user A runs workflow A, and user B runs workflow B, we want each workflow to receive a different token. I see that I can probably pass a token as an input to the Flyte task, but then I am concerned that the input would be stored in Flyte or accidentally logged. Any recommendations?

We have a conditional branch that takes the output of two tasks, and, whether they are true or false, takes the output of two different tasks to process in a final task. Once in a while, the inputs from the boolean checks are being bound to the branch instead of the other tasks (once or twice out of a couple hundred). This makes flyte’s input converter fail. Does anyone else have the case where the variable being conditionally evaluated is not the same variable being passed into the nested tasks? Happy to share more logging as needed. Running on GKE.

Traceback (most recent call last):

      File "/root/.varner_dp/venv/3.7.1/lib/python3.9/site-packages/flytekit/exceptions/scopes.py", line 165, in system_entry_point
        return wrapped(*args, **kwargs)
      File "/root/.varner_dp/venv/3.7.1/lib/python3.9/site-packages/flytekit/core/base_task.py", line 472, in dispatch_execute
        native_inputs = TypeEngine.literal_map_to_kwargs(exec_ctx, input_literal_map, self.python_interface.inputs)
      File "/root/.varner_dp/venv/3.7.1/lib/python3.9/site-packages/flytekit/core/type_engine.py", line 800, in literal_map_to_kwargs
        return {k: TypeEngine.to_python_value(ctx, lm.literals[k], python_types[k]) for k, v in lm.literals.items()}
      File "/root/.varner_dp/venv/3.7.1/lib/python3.9/site-packages/flytekit/core/type_engine.py", line 800, in <dictcomp>
        return {k: TypeEngine.to_python_value(ctx, lm.literals[k], python_types[k]) for k, v in lm.literals.items()}

Message:

    'n2.o0'

SYSTEM ERROR! Contact platform administrators.

Optional FlyteFile in dataclass In the following code example, my test_task_2 returns a dataclass with an Optional[FlyteFile] type. In test_task_1 I am directly returning an Optional[FlyteFile], which works fine, however in test_task_2 only the first non-optional file shows up on the remote. If the first example works, I would also expect, that my TaskOutput.file2 is uploaded correctly. Is this expected behavior / are optional types in dataclasses not supported? I am using flytekit 1.2.7 locally. Thanks for help

from dataclasses import dataclass
from typing import Optional

from dataclasses_json import dataclass_json
from flytekit import workflow, task
from flytekit.types.file import FlyteFile


@dataclass_json
@dataclass
class TaskOutput:
    file1: FlyteFile
    file2: Optional[FlyteFile]


@task
def test_task_1() -> Optional[FlyteFile]:
    with open("file11.txt", "w+") as f:
        f.write("")
    return FlyteFile("file11.txt", remote_path="<s3://test-bucket/tmp/file11.txt>")


@task
def test_task_2() -> TaskOutput:
    with open("file21.txt", "w+") as f:
        f.write("")
    with open("file22.txt", "w+") as f:
        f.write("")

    return TaskOutput(
        FlyteFile("file21.txt", remote_path="<s3://test-bucket/tmp/file21.txt>"),
        FlyteFile("file22.txt", remote_path="<s3://test-bucket/tmp/file22.txt>")
    )


@workflow
def test_workflow():
    test_task_1()
    test_task_2()


if __name__ == '__main__':
    test_workflow()

Hello! I have a workflow which chains together several sub-workflows, including two

reference_launch_plans

and a

dynamic

workflow. When I try to view this workflow in the Flyte console, everything shows up as unknown (Node ID’s, type, workflow/task name, etc.), and the graph doesn’t render. When I try to get execution details using

flytectl get execution --details

, I get this error:

{"json":{},"level":"info","msg":"Retrieved 1 executions","ts":"2022-11-22T10:25:18-08:00"}
Error: unsupported literal scalar type *core.Scalar_Union
{"json":{},"level":"error","msg":"unsupported literal scalar type *core.Scalar_Union","ts":"2022-11-22T10:27:16-08:00"}

Is what I’m trying to do here something that isn’t supported?

Is there any way to enforce access controls on a per project/domain basis in terms of who is allowed to register entities/trigger executions? We were wondering if it was possible to limit access to running in the production domain using the auth/security features in Flyte but couldn't find anything in the docs

Hi Flyte Fellows, I have a question about Flytectl Auth. There are 2 major concerns when testing flytectl auth over Pkce which I will summarize in 🧵 :

Hi everyone. I'm new to using flyte. I'm using wsl and have installed flytectl using curl. How do I address "flytectl command not found?

Hi all. I originally had flyte working, but had to delete the cluster and reinstall. I passed the deployment stage and set it using this console (https://birdejo.live). Unfortunately the console doesnt seem to work. Any ideas on how to diagnose what the issue is?

Is there a way to allow tasks to fail? I have the following setup:

@dynamic
def base_workflow(config: Config):
    for i in [1..10]:
        task_a = a(i=i)
        task_b = b(i=i)
        task_c = c(i=i)
        task_a >> task_b >> task_c

This all starts nicely 10x in parallel (as desired), but as soon as one of the tasks fail (lets say, task_a fails for i=3), they all abort. Is there a Flyte-native way to allow i=3 to fail, but let the others complete?