Quick question about the type system: Is there a technical reason why a tuple type transformer cannot be implemented easily? This is somehow something that is regularly met with disbelief and a bit of frustration when our engineers stumble on this. This is not meant as criticism, I’d rather like to know whether there is a known technical reason why this is complicated.

Hi, Im running into an issue when trying to set up a dev environment by following this guide: https://github.com/flyteorg/flyte/pull/3811 I have the demo environment with Flyte running (all Pods Running with

flytectl demo start --dev

), and was able to

make compile

and test all the different components. Now on the final step when running

flyte start --config flyte_local.yaml

it fails, due to wanting to access

/home/{user}/.flyte/k3s/k3s.yaml

which is a broken symlink (as described here as well: https://github.com/flyteorg/flyte/issues/3645)

❯ flyte start --config flyte_local.yaml
INFO[0000] Using config file: [flyte_local.yaml]        
{"json":{"src":"viper.go:398"},"level":"debug","msg":"Config section [plugins] updated. No update handler registered.","ts":"2023-06-30T14:39:46+02:00"}
...
{"json":{"src":"viper.go:398"},"level":"debug","msg":"Config section [webhook] updated. No update handler registered.","ts":"2023-06-30T14:39:46+02:00"}
{"json":{"src":"start.go:63"},"level":"info","msg":"Running Database Migrations...","ts":"2023-06-30T14:39:46+02:00"}
{"json":{"src":"start.go:124"},"level":"error","msg":"Failed to create controller manager. error building Kubernetes Clientset: Error building kubeconfig: stat /home/user/.flyte/k3s/k3s.yaml: no such file or directory","ts":"2023-06-30T14:39:46+02:00"}
{"json":{"src":"start.go:185"},"level":"panic","msg":"Failed to start Propeller, err: error building Kubernetes Clientset: Error building kubeconfig: stat /home/user/.flyte/k3s/k3s.yaml: no such file or directory","ts":"2023-06-30T14:39:46+02:00"}
panic: (*logrus.Entry) 0xc000510380

goroutine 61 [running]:
<http://github.com/sirupsen/logrus.(*Entry).log(0xc000510310|github.com/sirupsen/logrus.(*Entry).log(0xc000510310>, 0x0, {0xc0014de0a0, 0x9f})
        /root/go/pkg/mod/github.com/sirupsen/logrus@v1.8.1/entry.go:259 +0x487
<http://github.com/sirupsen/logrus.(*Entry).Log(0xc000510310|github.com/sirupsen/logrus.(*Entry).Log(0xc000510310>, 0x0, {0xc00105be68?, 0x1?, 0x1?})
        /root/go/pkg/mod/github.com/sirupsen/logrus@v1.8.1/entry.go:293 +0x4f
<http://github.com/sirupsen/logrus.(*Entry).Logf(0xc000510310|github.com/sirupsen/logrus.(*Entry).Logf(0xc000510310>, 0x0, {0x3145be9?, 0x0?}, {0xc000a8e230?, 0x0?, 0x0?})
        /root/go/pkg/mod/github.com/sirupsen/logrus@v1.8.1/entry.go:338 +0x85
<http://github.com/sirupsen/logrus.(*Entry).Panicf(0x3fdac40|github.com/sirupsen/logrus.(*Entry).Panicf(0x3fdac40>?, {0x3145be9?, 0x416947?}, {0xc000a8e230?, 0x2a74b80?, 0x1?})
        /root/go/pkg/mod/github.com/sirupsen/logrus@v1.8.1/entry.go:376 +0x34
<http://github.com/flyteorg/flytestdlib/logger.Panicf({0x3fdac40|github.com/flyteorg/flytestdlib/logger.Panicf({0x3fdac40>?, 0xc000c55310?}, {0x3145be9, 0x22}, {0xc000a8e230, 0x1, 0x1})
        /root/go/pkg/mod/github.com/flyteorg/flytestdlib@v1.0.17/logger/logger.go:188 +0x64
<http://github.com/flyteorg/flyte/cmd/single.glob..func4.2()|github.com/flyteorg/flyte/cmd/single.glob..func4.2()>
        /home/user/git/flyte/flyte/cmd/single/start.go:185 +0xbe
<http://golang.org/x/sync/errgroup.(*Group).Go.func1()|golang.org/x/sync/errgroup.(*Group).Go.func1()>
        /root/go/pkg/mod/golang.org/x/sync@v0.1.0/errgroup/errgroup.go:75 +0x64
created by <http://golang.org/x/sync/errgroup.(*Group).Go|golang.org/x/sync/errgroup.(*Group).Go>
        /root/go/pkg/mod/golang.org/x/sync@v0.1.0/errgroup/errgroup.go:72 +0xa5

hey, all. no matter which commands I try my

flytectl

consistently fails to discover where config lives (

~/.flyte/config.yaml

). the only way i've gotten it to work is by supplying

--file

arg manually – but I thought that was the point of having a default location:

> flytectl config init
INFO[0000] [0] Couldn't find a config file []. Relying on env vars and pflags.
Init flytectl config file at [/Users/kevinschaich/.flyte/config.yaml]
                                                                                                          
> flytectl config discover
INFO[0000] [0] Couldn't find a config file []. Relying on env vars and pflags.

just me or has anyone else encountered? did I forget to set an env variable somewhere? I'm on an M1 Mac – Ventura

13.2

,

flytectl v0.6.27

So I’m stuck on S3 access - I created a service account called

flyte-executor

with the

flyte-user-role

(which has full s3 access) attached as an annotation and running Flyte executions with this service account, but it’s giving me PutObject access denied error. This service account is in the project+domain namespace. What am I doing wrong?

Hi All, we are deploying flyte to our kubernetes cluster. We are at the SSL Certificate step. We don’t want to expose flyte to outside of our organization. But, it looks like the SSL Certificate for GKE is only for external HTTPS. Are there other options we can use so that we don’t expose flyte to outside world? Appreciate any helps from the community.

Can we also easily create integrations with external apps as easily as prefect on Flyte too?

Hi everyone.. any familiar with this error running a workflow? [1/1] currentAttempt done. Last Error: USER::Pod failed. No message received from kubernetes. [a8crkvcv4mzm69kk5hp5-n0-0] terminated with exit code (1). Reason [Error]. Message: exec /opt/env/bin/pyflyte-execute: exec format error .

Hello, I am trying to configure auth across azure AD, I added the following as configuration:

auth:
    enabled: true
    oidc:
      baseUrl: "<https://login.microsoftonline.com/TENANT_ID/v2.0>"
      clientId: CLIENT_ID
      clientSecret: CLIENT_SECRET
      scopes:
        - openid
        - email
        - profile
    internal:
      clientSecret: CLIENT_SECRET
      clientSecretHash: CLIENT_SECRET_HASHED
    flyteClient:
      clientId: CLIENT_ID
      redirectUri: "<http://localhost:53593/callback>"
      scopes:
        - all
    authorizedUris:
    - <https://login.microsoftonline.com/TENANT_ID/oauth2/v2.0>
    - <https://my.domain.com>

When trying to login I can walk through azure login step, but then receive a 403 from my.domain.com/callback?code=XXXX After an increase of the log level, I can see the following :

{"json":{"src":"handlers.go:238"},"level":"debug","msg":"Running authentication gRPC interceptor","ts":"2023-06-27T14:31:08Z"}
{"json":{"src":"token.go:83"},"level":"debug","msg":"Could not retrieve bearer token from metadata rpc error: code = Unauthenticated desc = Request unauthenticated with Bearer","ts":"2023-06-27T14:31:08Z"}
{"json":{"src":"handlers.go:248"},"level":"info","msg":"Failed to parse Access Token from context. Will attempt to find IDToken. Error: [JWT_VERIFICATION_FAILED] Could not retrieve bearer token from metadata, caused by: rpc error: code = Unauthenticated desc = Request unauthenticated with Bearer","ts":"2023-06-27T14:31:08Z"}
{"json":{"src":"token.go:103"},"level":"debug","msg":"Could not retrieve id token from metadata rpc error: code = Unauthenticated desc = Request unauthenticated with IDToken","ts":"2023-06-27T14:31:08Z"}
{"json":{"src":"cookie.go:79"},"level":"info","msg":"Could not detect existing cookie [flyte_idt]. Error: http: named cookie not present","ts":"2023-06-27T14:31:08Z"}
{"json":{"src":"handlers.go:65"},"level":"error","msg":"Failed to retrieve tokens from request, redirecting to login handler. Error: [EMPTY_OAUTH_TOKEN] Failure to retrieve cookie [flyte_idt], caused by: http: named cookie not present","ts":"2023-06-27T14:31:08Z"}
{"json":{"src":"handlers.go:121"},"level":"debug","msg":"Setting CSRF state cookie to tb9f2xhb2y and state to 2419390fb3ddca455183ba94811c3c6a3a9d988b99536691fde913716e22cd65\n","ts":"2023-06-27T14:31:08Z"}
{"json":{"src":"composite_workqueue.go:88"},"level":"debug","msg":"Subqueue handler batch round","ts":"2023-06-27T14:31:08Z"}
{"json":{"src":"composite_workqueue.go:98"},"level":"debug","msg":"Dynamically configured batch size [-1]","ts":"2023-06-27T14:31:08Z"}
{"json":{"src":"composite_workqueue.go:129"},"level":"debug","msg":"Exiting SubQueue handler batch round","ts":"2023-06-27T14:31:08Z"}
{"json":{"src":"handlers.go:141"},"level":"debug","msg":"Running callback handler... for RequestURI /callback?code=XXXXXX

I am behind an azure/application-gateway as ingress, and as far as I know appgw does not support gRPC, can it be the issue? Or should I look somewhere else?

Has anyone figured out how to get sphinx-autodoc to work with Flyte task/workflow decorators? I found that creating a Sphinx extension (similar to what Celery did here) worked, but am curious if anyone else has dealt with this

Hi, I have a flyte cluster on AWS with the manual setup. I'm having trouble registering workflows, for example trying to register the examples with

flytectl register examples -d development -p flytesnacks

I'm getting:

Error: example 0xc0006ae770 failed to register rpc error: code = Internal desc = failed to write marshaled workflow [resource_type:WORKFLOW project:"flytesnacks" domain:"development" name:"blast.blastx_example.blast_wf" version:"v0.3.81" ] to storage <s3://flyte-cluster-bucket/metadata/admin/flytesnacks/development/blast.blastx_example.blast_wf/v0.3.81> with err Failed to write data [3170b] to path [metadata/admin/flytesnacks/development/blast.blastx_example.blast_wf/v0.3.81].: PutObject, putting object: WebIdentityErr: failed to retrieve credentials
caused by: ValidationError: Request ARN is invalid

But I can't seem to find which ARN this error is referring to