Hey, I am trying to use the

clientSecret

authflow to be able to build and register workflows in our GitHub Actions CI builds. I looked at the CI docs here, and getting the following error both locally and in our CI when I run the same command. Has anyone experienced anything similar? Flytectl config available in comments

☁  ~  flytectl get projects
Error: rpc error: code = Unauthenticated desc = transport: oauth2: cannot fetch token: 401 Unauthorized
Response: {"error":"invalid_client","error_description":"Client authentication failed (e.g., unknown client, no client authentication included, or unsupported authentication method)."}

Hi, I'm seeing a very weird error in my workflow (see thread). Propeller is complaining that there is no input to a node in the workflow. When I look into the gcs bucket I see this in the

<gs://mybucket/metadata/propeller/myproject-myns-myexecid/n1/data/>

path:

data
|  0/
|  | start-node/
|  |    |  0/
|  |    |  | outputs.pb
|  
|  inputs.pb

i.e. there's no

n0/

directory under

data/0/

. When I look into

data/0/start-node/0/outputs.pb

it looks to be the correct input to the entire workflow. I'm running

flytekit 1.5.0

and deployed from helm

flyte-core:1.5.0

. I tried reverting to

1.4.0

in the core helm chart but it didn't help. Any idea why this is happening?

I need to conditionally run some tasks based on an output of a previous task. I saw that using

if

in my @workflow to conditionally run these tasks is not supported because of limitations related to DAG and/or serialization I tried with a conditional (

from flytekit import conditional

), when I do

conditional('somename').if_(mybool).then(mytask(myinputs=myinputs))

it ignores my conditional with no representation in the graph UI. When I add an

.else_().then(anytask())

it gets represented in graphs UI but errors with

AttributeError: 'FlyteBranchNode' object has no attribute 'interface'

after the run already begins I tried with

@task

and with

@dynamic

What is the correct way to conditionally run these multiple tasks based an a boolean output of another task? Note that these tasks also have dependencies on each other, one of them produces outputs and the rest receive it as input

Does anyone have experience with TLS/SSL between the Ingress and Flyte? 🔒 🔒 I’m currently trying to get a reference implementation to work for deploying Flyte on GCP with the GKE Ingress controller (instead of nginx) wich would allow us to use Google Identity Aware Proxy (IAP) for authentication. IAP requires the user to login “at the load balancer”, before a request reaches the backend, instead of being redirected to the login screen by the backend. Because this is more secure it’s a requirement by our company (and actually also my previous one) to use IAP. To make gRPC backends work with the GKE Ingress, HTTP2 needs to be enabled for the backend. Flyte already does so in the gcp helm values. Using HTTP2 between the load balancer and the backend requires TLS between lb and backend. So I’m trying to deploy Flyte with this config:

configmap:
  adminServer:
    server:
      httpPort: 8088
      grpcPort: 8089
      security:
        secure: false #true
        ssl:
          certificateFile: "/etc/tls/cert.pem"
          keyFile: "/etc/tls/key.pem"

Certificate is self-signed and mounted into the flyteadmin pod via a secret. I can’t talk to flyteadmin though and would appreciate help getting this to work 🙂 Details in 🧵

hi everybody, has anybody running on a mac m1 managed to run the tensorflow cookbook example?

Hey! Why is it necessary to install flytekit in virtual environment. ??? Flytekit not found error occurs even if the module is installed. But it works fine if flytekit is installed in venv and that activated venv is used for workflows and programs.

I have two questions: 1. How can I monitor the std out and std err while the task is running ? 2. Does

pyflyte register

not use the docker image when its packaging the workflow. We have some custom dependencies for the workflow to execute that aren’t present in the development environment.

Hello, How to configure S3 and Postgres from Environment variables in Kubernetes cluster?

I'm having issues when running a fast-registered workflow. It looks like flytekit (0.30.3) can't find the code archive:

[1/1] currentAttempt done. Last Error: USER::Pod failed. No message received from kubernetes.

[vej152kn4b-n1-0] terminated with exit code (1). Reason [Error]. Message: 
tar: /root/8143d4634b6d53c26072284ce429e69af1278102-fast1-fast32b90af686df921bd623ae6df68f5c48.tar.gz: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
Traceback (most recent call last):
  File "/opt/venv/bin/pyflyte-fast-execute", line 8, in <module>
    sys.exit(fast_execute_task_cmd())
  File "/opt/venv/lib/python3.9/site-packages/click/core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "/opt/venv/lib/python3.9/site-packages/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "/opt/venv/lib/python3.9/site-packages/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/venv/lib/python3.9/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/opt/venv/lib/python3.9/site-packages/flytekit/bin/entrypoint.py", line 467, in fast_execute_task_cmd
    _download_distribution(additional_distribution, dest_dir)
  File "/opt/venv/lib/python3.9/site-packages/flytekit/tools/fast_registration.py", line 112, in download_distribution
    result.check_returncode()
  File "/usr/lib/python3.9/subprocess.py", line 460, in check_returncode
    raise CalledProcessError(self.returncode, self.args, self.stdout,
subprocess.CalledProcessError: Command '['tar', '-xvf', '/root/8143d4634b6d53c26072284ce429e69af1278102-fast1-fast32b90af686df921bd623ae6df68f5c48.tar.gz', '-C', '/root']' returned non-zero exit status 2.
.

Here's what's executing from the pod spec

spec:
  containers:
  - args:
    - pyflyte-fast-execute
    - --additional-distribution
    - <gs://my-bucket/fast/8143d4634b6d53c26072284ce429e69af1278102-fast1-fast32b90af686df921bd623ae6df68f5c48.tar.gz>
...

I checked the bucket and the object exists so uploading during registration seems to be working correctly. Perhaps something wrong with the download or the path it is copied to in the container. Any ideas?

Hi all! 👋 I've started exploring flyte recently but I don't seem to be able to run workflows on my sandbox environment. I've created a sandbox environment using the command

flytectl sandbox start

and exported the env vars for kube and flyte configuration. I'm trying to run the hello world example from flytesnacks and I've created a virtual env using poetry, in which I installed pyflyte (1.0.2). I can run it locally no problem, but when I add the

remote

flag, I get a connection reset by peer error. Any ideas what is wrong on my setup? I'm on WSL2