Hello, we have been keeping an eye on Flyte since a while and decided to run a POC today, but notice...

glamorous-diamond-15887

11/06/2023, 6:40 PM

Hello, we have been keeping an eye on Flyte since a while and decided to run a POC today, but noticed that the GCP deployment page on Flyte docs has been changed - it no longer has the detailed steps that were there earlier. Is it possible to get those detailed instructions back?

👍 1

average-finland-92144

11/06/2023, 8:45 PM

Hi @glamorous-diamond-15887 your question arrived just on time. I just pushed this PR that adds a Flyte reference implementation for GCP using Terraform. It'd be great if you can use it, test it and report any findings. It should take you from an empty GCP project to a flyte-binary instance including Ingress and SSL.

🎉 3

cool-lifeguard-49380

11/06/2023, 9:22 PM

In case you have questions i can also try to help, have been running flyte on gcp for a while (core not binary but still ..)

strong-plumber-41198

11/17/2023, 10:11 AM

Hi there, I work with @glamorous-diamond-15887, we have followed the code provided and have the infrastructure setup, however we seem to have come stuck in the final steps. We’ve run the helm install command and from that after running the command:

kubectl get ingress -n flyte

shows 2 NGINX resources:

flyte-binary-grpc

and

flyte-binary-http

. However, it’s the last step where we should update the config.yaml file. When I open the file it is empty. Is that supposed to be the case? Also, once that step is completed, are there any other steps to get the console up and running?

cool-lifeguard-49380

11/17/2023, 10:41 AM

The UI/flyteconsole should be up after deploying the helm chart with the ingresses. No need for config files for that 🤔

average-finland-92144

11/17/2023, 10:50 AM

@strong-plumber-41198 yes, you should have two Ingress resources, both using the same IP address, the one you'd use to create an A record in your DNS. Then the hostname is what you'd enter in your local

config

file, but this is only needed for the CLI. Going to

https:://<your-flyte-fqdn>/console

should take you to the UI (once the new A record is propagated)

strong-plumber-41198

11/17/2023, 11:04 AM

ah ok, thanks @average-finland-92144 and @cool-lifeguard-49380 I think we are having issues with the subdomain delegation, so it’s not a Flyte issue, I can update once these have been resolved

👍 1

strong-plumber-41198

11/17/2023, 12:51 PM

I believe the domain error has now been resolved, but I am getting a

NET::ERR_CERT_AUTHORITY_INVALID

error at the url, I am getting the following as the cert-issuer:

Copy code

Jake@192 flyte % kubectl describe issuer letsencrypt-production -n flyte
Name:         letsencrypt-production
Namespace:    flyte
Labels:       <none>
Annotations:  <none>
API Version:  cert-manager.io/v1
Kind:         Issuer
Metadata:
  Creation Timestamp:  2023-11-14T15:14:51Z
  Generation:          1
  Resource Version:    3817
  UID:                 b9fcc83e-fc45-47c2-9540-489f68d56f52
Spec:
  Acme:
    Email:  noreply@flyte.org
    Private Key Secret Ref:
      Name:  letsencrypt-production
    Server:  <https://acme-v02.api.letsencrypt.org/directory>
    Solvers:
      http01:
        Ingress:
          Ingress Class Name:  nginx
Status:
  Acme:
    Last Private Key Hash:  EzBocC0eIPkoKo/0fiT14YVOeSbLQJVfrbUXCAsUWEo=
    Last Registered Email:  noreply@flyte.org
    Uri:                    <https://acme-v02.api.letsencrypt.org/acme/acct/1413684816>
  Conditions:
    Last Transition Time:  2023-11-14T15:17:54Z
    Message:               The ACME account was registered with the ACME server
    Observed Generation:   1
    Reason:                ACMEAccountRegistered
    Status:                True
    Type:                  Ready
Events:                    <none>

cool-lifeguard-49380

11/17/2023, 1:05 PM

Can you please take a look at the Certificate resource in the flyte namespace?

strong-plumber-41198

11/17/2023, 1:07 PM

Copy code

Jake@192 flyte % kubectl get certificate -n flyte
NAME               READY   SECRET             AGE
flyte-secret-tls   False   flyte-secret-tls   22h

cool-lifeguard-49380

11/17/2023, 1:15 PM

READY false is an issue, you can maybe

kubectl describe Certificate …

to figure out why?

strong-plumber-41198

11/17/2023, 2:03 PM

Copy code

Jake@192 flyte % kubectl describe certificate -n flyte
Name:         flyte-secret-tls
Namespace:    flyte
Labels:       <http://app.kubernetes.io/instance=flyte-binary|app.kubernetes.io/instance=flyte-binary>
              <http://app.kubernetes.io/managed-by=Helm|app.kubernetes.io/managed-by=Helm>
              <http://app.kubernetes.io/name=flyte-binary|app.kubernetes.io/name=flyte-binary>
              <http://app.kubernetes.io/version=1.16.0|app.kubernetes.io/version=1.16.0>
              <http://helm.sh/chart=flyte-binary-v1.10.0|helm.sh/chart=flyte-binary-v1.10.0>
Annotations:  <http://acme.cert-manager.io/http01-override-ingress-name|acme.cert-manager.io/http01-override-ingress-name>: flyte-binary-grpc
              <http://cert-manager.io/issue-temporary-certificate|cert-manager.io/issue-temporary-certificate>: true
API Version:  <http://cert-manager.io/v1|cert-manager.io/v1>
Kind:         Certificate
Metadata:
  Creation Timestamp:  2023-11-16T14:28:41Z
  Generation:          1
  Owner References:
    API Version:           <http://networking.k8s.io/v1|networking.k8s.io/v1>
    Block Owner Deletion:  true
    Controller:            true
    Kind:                  Ingress
    Name:                  flyte-binary-grpc
    UID:                   6da305d1-d8a7-48e1-93e1-e3ff73912cd3
  Resource Version:        1549186
  UID:                     e1626685-9ffc-4d6b-beca-1c5bbaab3484
Spec:
  Dns Names:
    flyte.gcp.passby.dev
  Issuer Ref:
    Group:      <http://cert-manager.io|cert-manager.io>
    Kind:       Issuer
    Name:       letsencrypt-production
  Secret Name:  flyte-secret-tls
  Usages:
    digital signature
    key encipherment
Status:
  Conditions:
    Last Transition Time:        2023-11-16T14:28:41Z
    Message:                     Issuing certificate as Existing issued Secret is not up to date for spec: [spec.dnsNames]
    Observed Generation:         1
    Reason:                      SecretMismatch
    Status:                      True
    Type:                        Issuing
    Last Transition Time:        2023-11-16T14:28:41Z
    Message:                     Issuing certificate as Existing issued Secret is not up to date for spec: [spec.dnsNames]
    Observed Generation:         1
    Reason:                      SecretMismatch
    Status:                      False
    Type:                        Ready
  Next Private Key Secret Name:  flyte-secret-tls-2wpgq
  Not After:                     2024-02-13T12:08:46Z
  Not Before:                    2023-11-15T12:08:46Z
  Renewal Time:                  2024-01-14T12:08:46Z
Events:                          <none>

average-finland-92144

11/17/2023, 2:25 PM

also, could you describe the

certificaterequest

strong-plumber-41198

11/17/2023, 2:34 PM

Copy code

Jake@192 flyte % kubectl describe certificaterequest -n flyte
Name:         flyte-secret-tls-1
Namespace:    flyte
Labels:       <http://app.kubernetes.io/instance=flyte-binary|app.kubernetes.io/instance=flyte-binary>
              <http://app.kubernetes.io/managed-by=Helm|app.kubernetes.io/managed-by=Helm>
              <http://app.kubernetes.io/name=flyte-binary|app.kubernetes.io/name=flyte-binary>
              <http://app.kubernetes.io/version=1.16.0|app.kubernetes.io/version=1.16.0>
              <http://helm.sh/chart=flyte-binary-v1.10.0|helm.sh/chart=flyte-binary-v1.10.0>
Annotations:  <http://acme.cert-manager.io/http01-override-ingress-name|acme.cert-manager.io/http01-override-ingress-name>: flyte-binary-grpc
              <http://cert-manager.io/certificate-name|cert-manager.io/certificate-name>: flyte-secret-tls
              <http://cert-manager.io/certificate-revision|cert-manager.io/certificate-revision>: 1
              <http://cert-manager.io/issue-temporary-certificate|cert-manager.io/issue-temporary-certificate>: true
              <http://cert-manager.io/private-key-secret-name|cert-manager.io/private-key-secret-name>: flyte-secret-tls-2wpgq
API Version:  <http://cert-manager.io/v1|cert-manager.io/v1>
Kind:         CertificateRequest
Metadata:
  Creation Timestamp:  2023-11-16T14:28:41Z
  Generation:          1
  Owner References:
    API Version:           <http://cert-manager.io/v1|cert-manager.io/v1>
    Block Owner Deletion:  true
    Controller:            true
    Kind:                  Certificate
    Name:                  flyte-secret-tls
    UID:                   e1626685-9ffc-4d6b-beca-1c5bbaab3484
  Resource Version:        1549201
  UID:                     98436c86-8238-488d-9d2d-c264e6974b4f
Spec:
  Extra:
    <http://authentication.kubernetes.io/pod-name|authentication.kubernetes.io/pod-name>:
      cert-manager-6856dc897b-wqdms
    <http://authentication.kubernetes.io/pod-uid|authentication.kubernetes.io/pod-uid>:
      df4b6b50-0b2b-4ac4-816d-8f11a4b25f06
  Groups:
    system:serviceaccounts
    system:serviceaccounts:cert-manager
    system:authenticated
  Issuer Ref:
    Group:  <http://cert-manager.io|cert-manager.io>
    Kind:   Issuer
    Name:   letsencrypt-production
  Request:  LS0tLS1CRUdJTiBDRVJUSUZJQ0FURSBSRVFVRVNULS0tLS0KTUlJQ2hEQ0NBV3dDQVFBd0FEQ0NBU0l3RFFZSktvWklodmNOQVFFQkJRQURnZ0VQQURDQ0FRb0NnZ0VCQUtvdAphQ3U0N2RFcnpibnBnYmRIRnEwMStlZXVjN3BEZHBTM0dkWVl0bUhJNHRRM2ZVVW9McEpYck05dnMwT2hNWEM2Ck9sMU1QVCt3bXlLQlovQ3hsRTEzZndWeGpqTmdKb2tTQmhVV0xEck93akhaSlU2eVFiMVhOTHZiYzJXaUdSc2sKVnMvWFJ3eWs3TFdZamROcnAvb3pJYmh1VWZEQ0JnQTB1U1h1OHZlbW9zM0VwMUV0MDJ5TXY2MzJIZ0s2M1ViWAp3NG9VMHFIWFNWSEIrSFFyTTR2a3dLYjFHUjdwZWFqVDg1T1ZyNXlEZkZRc0JlcjVObm9TdW96QW5lYWprNk85CjgvYmsrWmM0VWN0bVVDeGpxMnZrbkZ1N0VTRThMOGZZRHVudlozTTJxaFk0LzNaVXR0RGgwc1pFS3VIVHhKTUcKSitubUJaUlNSOWhjWGh0cFJsY0NBd0VBQWFBL01EMEdDU3FHU0liM0RRRUpEakV3TUM0d0h3WURWUjBSQkJndwpGb0lVWm14NWRHVXVaMk53TG5CaGMzTmllUzVrWlhZd0N3WURWUjBQQkFRREFnV2dNQTBHQ1NxR1NJYjNEUUVCCkN3VUFBNElCQVFBektRRXo5RWVsb3pHZk5mS2VXWUxoWlpsU1FnLzZWTDN0d1poN3h2S2JRZ1NsaCt4VlBjdEkKZ3FWcllTVEg3eTFGOGx4Q0d4T3dzU3p6RFBkUWN1eXRITXZHdTB5SmxxOVhaUmVYZzcwaGJLdDN5QWV5OXNpawpoU0h3cXVzWUxLNXc1Z1NZSGdhWTNyaUR1MEFyL2UxMUN3aFlPTFpkNjBNRmR2aVMxZzkydmY0eHBUOFgwUVNGCjZ4aDI0QjRsZnZTR21HK1JzaVRYa0JYeTd4OEtMQnlVU2I4OHpJRVZ1TStKNjJyOUlueVFjQlRJUEpvUlkwa2cKL09Md09wWUlKbklnZmsrd2RBZ1MyTXk3bTlsVXRNRjZaR1BjdDRFWmQ4WnZLK2p3VFRyTkVxWlc5L212WkhnOQpBd2FpM0pJeU83K1MvUXhlSHFtY3hxU1RzVlFjeHNvVQotLS0tLUVORCBDRVJUSUZJQ0FURSBSRVFVRVNULS0tLS0K
  UID:      00f6ba69-15ce-4a36-a4b6-b69b5ac6122c
  Usages:
    digital signature
    key encipherment
  Username:  system:serviceaccount:cert-manager:cert-manager
Status:
  Conditions:
    Last Transition Time:  2023-11-16T14:28:41Z
    Message:               Certificate request has been approved by <http://cert-manager.io|cert-manager.io>
    Reason:                <http://cert-manager.io|cert-manager.io>
    Status:                True
    Type:                  Approved
    Last Transition Time:  2023-11-16T14:28:41Z
    Message:               Waiting on certificate issuance from order flyte/flyte-secret-tls-1-2359579067: "pending"
    Reason:                Pending
    Status:                False
    Type:                  Ready
Events:                    <none>

cool-lifeguard-49380

11/17/2023, 2:35 PM

It’s pending apparently. If everything was right, this should complete in a few seconds.

cool-lifeguard-49380

11/17/2023, 2:36 PM

Can you please make sure the DNS settings are correct? While insecure, you should be able to see the cloud console also without TLS

strong-plumber-41198

11/17/2023, 3:10 PM

I uninstall the flyte-binary deployment and reinstalled and it is working now, thanks for all the help @cool-lifeguard-49380

🚀 2

cool-lifeguard-49380

11/17/2023, 3:10 PM

Awesome

13 Views

Open in Slack

Previous Next

Flyte

Flyte enables production-grade orchestration for machine learning workflows and data processing created to accelerate local workflows to production.