Hi, is there a way to get try / catch behavior in a Flyte workflow. I have a workflow where one of the tasks can fail to run. If it fails to run, I would like to perform different steps vs when it succeeds. Something like this:

@workflow
def wf():
    try:
        output0 = t0()
    except Exception:
        return handle_failure(...)
    
    return handle_success(...)

One way I can think of achieving this behavior is to move the try/catch inside

t0

, and conditioning on the output. Ideally I would like to reflect in the state of the workflow execution that t0 failed. Is there a better way to do this?

Hi Community, I am new to Flyte and trying to run the

flytectl demo start

following the starter guide However I keep hitting error

Authenticated requests get a higher rate limit. Check out the documentation for more details.

And it doesn’t show what documentation I should follow (not in the starter guide) could you please help advise?

Hi, We've tried to setup flyte on an EKS cluster. On checking with

kubectl get pods -n flyte

, the output is as shown below:

NAME                                              READY   STATUS                   RESTARTS          AGE
flyte-deps-contour-contour-cd4d956d9-2xlwk        1/1     Running                  0                 19d
flyte-deps-contour-envoy-244xb                    2/2     Running                  0                 19d
flyte-deps-contour-envoy-rbwp7                    2/2     Running                  0                 19d
flyte-deps-kubernetes-dashboard-8b7d858b7-x9ncd   1/1     Running                  0                 19d
flyte-pod-webhook-85ff4b5f9d-t9l2f                1/1     Running                  0                 19d
flyteadmin-b46f578c7-qww88                        1/1     Running                  0                 17d
flyteconsole-77f665cc84-8gmv7                     1/1     Running                  0                 19d
flytepropeller-655664f9f9-6c69v                   1/1     Running                  0                 14d
flytepropeller-655664f9f9-l67tf                   0/1     Error                    0                 14d
flytepropeller-655664f9f9-zz2c2                   0/1     Error                    0                 19d
flytescheduler-6db897d545-mlntd                   0/1     ContainerStatusUnknown   1                 14d
flytescheduler-6db897d545-mmhtl                   1/1     Running                  563 (8m45s ago)   10d
minio-5cbbccf6cd-bm6ms                            1/1     Running                  0                 17d
postgres-565f59484-qcnjq                          1/1     Running                  0                 19d
syncresources-7b4b44f7cc-p4h9j                    1/1     Running                  0                 19d

But when we run any command to run/register a workflow

pyflyte run ........  etc.

, we're getting the below error:

Traceback (most recent call last):
  File "/home/tarun/.local/bin/pyflyte", line 5, in <module>
    from flytekit.clis.sdk_in_container.pyflyte import main
  File "/home/tarun/.local/lib/python3.8/site-packages/flytekit/__init__.py", line 259, in <module>
    load_implicit_plugins()
  File "/home/tarun/.local/lib/python3.8/site-packages/flytekit/__init__.py", line 253, in load_implicit_plugins
    discovered_plugins = entry_points(group="flytekit.plugins")
TypeError: entry_points() got an unexpected keyword argument 'group'

Can someone explain, what might be the cause of this error ?

Hi, i'm trying to configure a secret to be able to pull our docker image from a private registry when starting a workflow. Ie i'd like to configure the

imagePullSecrets

field when starting pods. I've look at the secrets docs but i'm not sure this is what i'm looking for. As i understand, this will provide secret inside the tasks so, during the execution of the tasks not during the schedule of the pods, am i right ? 1/ does the secret shall be global or one by namespace (ie flyte's project ?) 2/ can i configure this secret once for all workflows ? Thanks !

Hi! I'm trying to run the example in the docs here: https://docs.flyte.org/projects/cookbook/en/latest/getting_started/analytics.html# When I register and run on the demo sandbox I get this error:

[1/1] currentAttempt done. Last Error: USER::Pod failed. No message received from kubernetes.
[ass5zvhclkmtr9hd55ks-n0-0] terminated with exit code (1). Reason [Error]. Message: 
trap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/root/workflows/ex_analytics.py", line 4, in <module>
    import pycountry
ModuleNotFoundError: No module named 'pycountry'
Traceback (most recent call last):
  File "/usr/local/bin/pyflyte-fast-execute", line 8, in <module>
    sys.exit(fast_execute_task_cmd())
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/flytekit/bin/entrypoint.py", line 513, in fast_execute_task_cmd
    subprocess.run(cmd, check=True)
  File "/usr/local/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['pyflyte-execute', '--inputs', '<s3://my-s3-bucket/metadata/propeller/flytesnacks-development-ass5zvhclkmtr9hd55ks/n0/data/inputs.pb>', '--output-prefix', '<s3://my-s3-bucket/metadata/propeller/flytesnacks-development-ass5zvhclkmtr9hd55ks/n0/data/0>', '--raw-output-data-prefix', '<s3://my-s3-bucket/data/7x/ass5zvhclkmtr9hd55ks-n0-0>', '--checkpoint-path', '<s3://my-s3-bucket/data/7x/ass5zvhclkmtr9hd55ks-n0-0/_flytecheckpoints>', '--prev-checkpoint', '""', '--dynamic-addl-distro', '<s3://my-s3-bucket/flytesnacks/development/TPFR2TGNNPMEAIZGLYXBVJ46S4======/scriptmode.tar.gz>', '--dynamic-dest-dir', '/root', '--resolver', 'flytekit.core.python_auto_container.default_task_resolver', '--', 'task-module', 'workflows.ex_analytics', 'task-name', 'clean_data']' returned non-zero exit status 1.
.

Anybody understand this?

How to

map volumes

through flyte from a directory inside VM to a directory inside Container and vice versa.(Bind mount)

Is there an easy way to specify a that a task should recieve the serialized version of an output from a previous task? Like say I have a train task that outputs a

nn.Module

with the pytorch plugin, but I want a downstream task to receive the filepath that the

nn.Module

is saved to. Is there a way to do that?

Hi I have a question about

@dynamic

workflows. If I create a dynamic workflow that invokes a set of tasks and one of those tasks fails but all the others succeeds, would it try to invoke the same set of tasks if I retrigger that same dynamic workflow? My situation is the following: • I want to compile a dbt project to read all the nodes in the dbt DAG and translate them to flyte tasks • I do not want to reprocess earlier nodes in the DAG if one of the nodes fail.

Hi everyone, I have question related to workflows that have many tasks that are executed in a different system (for example, hive or spark), Does flyte create one pod for each task? I am trying to study the performance impact of this situation when we have high fanout workflows of this kind. any pointers will be helpful.

Are folks using Flyte to do hyperparameter optimization to the scale of 1.4k tasks in a single workflow? We are hitting error from K8S request entity too large, where Fyte propeller OSS did make a patch explicitly for this.