https://flyte.org logo
#flyte-deployment
Title
# flyte-deployment
c

Cody Scandore

08/15/2023, 1:48 PM
I'm able to deploy the
flyte-binary
chart on EKS. When I try to add
auth
the container goes to CrashLoopBackoff.
Copy code
Events:
  Type     Reason     Age                    From               Message
  ----     ------     ----                   ----               -------
  Normal   Scheduled  6m49s                  default-scheduler  Successfully assigned flyte/flyte-flyte-binary-6479b586cc-sb75s to ip-192-xxx-xx-xxx.us-west-2.compute.internal
  Normal   Pulled     6m48s                  kubelet            Container image "postgres:15-alpine" already present on machine
  Normal   Created    6m48s                  kubelet            Created container wait-for-db
  Normal   Started    6m48s                  kubelet            Started container wait-for-db
  Normal   Started    6m47s                  kubelet            Started container gen-admin-auth-secret
  Normal   Created    6m47s                  kubelet            Created container gen-admin-auth-secret
  Normal   Pulled     6m47s                  kubelet            Container image "<http://cr.flyte.org/flyteorg/flyte-binary-release:v1.5.0|cr.flyte.org/flyteorg/flyte-binary-release:v1.5.0>" already present on machine
  Normal   Started    6m45s                  kubelet            Started container flyte
  Warning  Unhealthy  6m18s (x3 over 6m38s)  kubelet            Liveness probe failed: Get "<http://192.168.37.60:8088/healthcheck>": dial tcp 192.168.37.60:8088: connect: connection refused
  Normal   Killing    6m18s                  kubelet            Container flyte failed liveness probe, will be restarted
  Normal   Pulled     5m48s (x2 over 6m45s)  kubelet            Container image "<http://cr.flyte.org/flyteorg/flyte-binary-release:v1.5.0|cr.flyte.org/flyteorg/flyte-binary-release:v1.5.0>" already present on machine
  Normal   Created    5m48s (x2 over 6m45s)  kubelet            Created container flyte
  Warning  Unhealthy  107s (x52 over 6m44s)  kubelet            Readiness probe failed: Get "<http://192.168.37.60:8088/healthcheck>": dial tcp 192.168.37.60:8088: connect: connection refused
Copy code
...
{"json":{"src":"composite_workqueue.go:98"},"level":"debug","msg":"Dynamically configured batch size [-1]","ts":"2023-08-15T13:46:21Z"}
{"json":{"src":"composite_workqueue.go:129"},"level":"debug","msg":"Exiting SubQueue handler batch round","ts":"2023-08-15T13:46:21Z"}
{"json":{"src":"composite_workqueue.go:88"},"level":"debug","msg":"Subqueue handler batch round","ts":"2023-08-15T13:46:22Z"}
{"json":{"src":"composite_workqueue.go:98"},"level":"debug","msg":"Dynamically configured batch size [-1]","ts":"2023-08-15T13:46:22Z"}
{"json":{"src":"composite_workqueue.go:129"},"level":"debug","msg":"Exiting SubQueue handler batch round","ts":"2023-08-15T13:46:22Z"}
d

David Espejo (he/him)

08/15/2023, 3:00 PM
can you share the anonymized
values
you're using?
c

Cody Scandore

08/15/2023, 3:05 PM
Copy code
auth:
    enabled: true
    oidc:
      baseUrl: <https://login.microsoftonline.com/><my-client-id>/oauth2/v2.0/authorize
      clientId: 0xxxxx5-6xxx-4xxx-bxxx-3cxxxxxx
      clientSecret: 'Kxxxx~EqDxxxxxxxxxxxxxlxx~.Ddxxx'
    internal:
      clientSecret: 'monkeyJAX32'
      clientSecretHash: 'bW9ua2V5SkFYMzIK'
    authorizedUris:
      - <https://my-webapp.com>
d

David Espejo (he/him)

08/15/2023, 3:19 PM
is the
clientSecretHash
here dummy data? otherwise, I get a very different output for that
clientSecret
c

Cody Scandore

08/15/2023, 3:24 PM
They're real values. I used bcrypt but I had some errors launching the
gen-admin-auth-secret
container, along the lines of
Copy code
1 error(s) decoding:

* error decoding 'appAuth.selfAuthServer.staticClients[flytepropeller].client_secret': illegal base64 data at input byte 80

goroutine 1 [running]:
main.main()
        /flyteorg/build/cmd/main.go:13 +0x85
This value was generated from base64 on linux
Copy code
echo 'monkeyJAX32' | base64
and the deployment seemed (?) to progress further along.
d

David Espejo (he/him)

08/15/2023, 3:38 PM
I got the following using bcrypt:
JDJiJDA2JEp2dldEYk1ZZDB6ZWJJVnozRWIvM2VzcDE0NFJtRHh1QVdJakJLQzFyZDVMTjNQSGFlWkF1
c

Cody Scandore

08/15/2023, 3:47 PM
Okay, it seems like it's still crash looping with that value
Copy code
Normal   Killing    92s                 kubelet            Container flyte failed liveness probe, will be restarted
  Normal   Pulled     62s (x2 over 116s)  kubelet            Container image "<http://cr.flyte.org/flyteorg/flyte-binary-release:v1.5.0|cr.flyte.org/flyteorg/flyte-binary-release:v1.5.0>" already present on machine
  Normal   Created    62s (x2 over 116s)  kubelet            Created container flyte
  Warning  Unhealthy  62s (x9 over 115s)  kubelet            Readiness probe failed: Get "<http://192.168.37.60:8088/healthcheck>": dial tcp 192.168.37.60:8088: connect: connection refused
  Normal   Started    61s (x2 over 116s)  kubelet            Started container flyte
d

David Espejo (he/him)

08/15/2023, 3:59 PM
can you try with
baseUrl: <https://login.microsoftonline.com/><my-client-id>/oauth2/
?
c

Cody Scandore

08/15/2023, 4:02 PM
Same, failing readiness. Not much helpful information in the pod logs, as far as I can tell.
d

David Espejo (he/him)

08/15/2023, 4:41 PM
has it worked in the past?
c

Cody Scandore

08/15/2023, 4:41 PM
No
Everything works well without
auth
, fails with auth enabled.
9 Views