How do you change imagePullPolicy for existing deployments? By default, its ifnotpresent, but I would like to change it to always for all of my potential docker images/containers. Do I have to use https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#alwayspullimages?

I'm having some trouble configuring a new installation of flyte, but I think I'm close. I installed flyte with opta using the documentation here https://docs.flyte.org/en/latest/deployment/aws/opta.html#deployment-aws-opta I'm using an external-ssl-cert and am able to access the flyte console. My two problems: 1. I don't think I am using the correct endpoint for flytectl. I thought it should be the subdomain I access the console through but that didn't work. After trying a few things I was able to get it working by pointing flytectl directly to the flyteadmin service's load balancer on port 81. I used 

kubectl -n flyte get services flyteadmin

 to find it. Is this the correct way to do it? 2. I am having trouble configuring authentication with google cloud. Using https://docs.flyte.org/en/latest/deployment/cluster_config/auth_setup.html#deployment-cluster-config-auth-setup I did the following + Setup my google cloud OAuth2 Client Credential + Ran 

kubectl edit secret -n flyte flyte-admin-secrets

 and added the client secret + Ran 

kubectl edit configmap -n flyte flyte-admin-config

 updated the config according to the docs + Restarted flyteadmin with 

kubectl rollout restart deployment/flyteadmin -n flyte

I didn't get everything wrong because when I visited the flyte console it redirected me to google to login before going to the dashboard. However when I tried to run a workflow the new execution just hung with status unknown. I also was unable to connect with flytectl no matter what I tried. I'm not sure what I'm doing wrong here. Any help is much appreciated.

Hi Team, I have noticed when users are logged in to Flyte Console via Auth0, any workflows launched from Console go to an

UNKNOWN

state. I was wondering what could be possible cause for this? Do we need to inform any service other than Flyte Admin about the Auth? For Example Flyte Propeller? CC: @Prafulla Mahindrakar

Hi, I am very new to the Flyte and trying to learn how flyte compiles its workflow and there’s a cli: pyflyte serialize, but I would not get it work with an example project, created via pyflyte init. It will be great if anyone can share some insights. Here is my question in the SO: https://stackoverflow.com/questions/76298549/how-to-run-pyflyte-serialize-workflows

Hi team.. Can

dynamic

be cached?

Hello folks! Suppose we have Flyte workflow with some failed map_task. At the start, this task submits a lot(300-500) pods to Kubernetes, and one of them dies, so the whole workflow is failed. But the rest pods are still in Kubernetes and waiting to be executed. Why does Flyte not kill the rest pods if one of them is failed? To be able to rerun this workflow, all these pods should be manually deleted from the Kubernetes cluster.

Hi Community, I tried to integrate self-hosted Gitlab as an external OAuth2 Authorization Server but stuck at

CSRF_TOKEN_VALIDATION_FAILED

, any clues on how to debug the problem? The flyte is deployed on self-hosted k8s cluster with an internal domain name and HTTP access (not HTTPS) using template generated by

flyte-core

helm chart (

helm template flyte-core -f values.yaml

). The OIDC seems to work (After a fresh deployment, when opening flyte console, it will redirect to the gitlab authorization page, and after approval it can redirect back to flyte console page with username show on the top-right corner), but when I try to register workflow according to the tutorial [1] with command

flytectl register files --project flytesnacks --domain development --archive flyte-package.tgz --version v1

, the gitlab authorization page is prompted successfully but when redirect back to

<http://flyte.example.com/callback|flyte.example.com/callback>

page, it complains about

401

error. And the log of

flyteadmin

shows something like

{"json":{},"level":"error","msg":"Invalid CSRF token cookie [CSRF_TOKEN_VALIDATION_FAILED] CSRF token does not match state 2r4rcd3npg, 3237e1083ec0ae2bd20acbe8a5817d18475faaee5a060d2184ab7ffddd151290 vs OXpoczQyanRxcW43c3hnZ3RjbnBnZjZrNnptMnA2dDY","ts":"2023-01-29T06:43:34Z"}

There are several small questions that might be relevant: 1. In the auth doc [2], there is a comment for

redirectUri: <http://localhost:53593/callback>

saying that

This should not change

, but I suppose it is for sandbox deployment so I changed it to the domain name something like

<http://flyte.example.com/callback>

. Otherwise, it will complains about invalid redirect uri when authorize on gitlab page. 2. For the

scopes

, according to the gitlab doc [3], I only set it to

read_user

. 3. Any approach to show more logs for

flytectl

CLI? I tried to set the log level to

0

, but it does not show anything. 4. Any approach to show more logs for

flyteadmin

service? I found there is a

flyteadmin.extraArgs

in the template but does not know how to inject

--logger.level 0

. The relevant

values.yaml

for auth is shown as following (with some sensitive info masked):

configmap:
  adminServer:
    auth:
      appAuth:
        authServerType: External
        externalAuthServer:
          metadataUrl: .well-known/openid-configuration
        thirdPartyConfig:
          flyteClient:
            clientId: <client_id generated by gitlab>
            redirectUri: <http://flyte.example.com/callback>
            scopes:
              - read_user
      authorizedUris:
        - <http://flyte.example.com>
        - <http://flyteadmin:80>
        - <http://flyteadmin.flyte.svc.cluster.local:80>
      userAuth:
        openId:
          baseUrl: <https://git.example.com>
          clientId: <client_id generated by gitlab>
          scopes:
            - openid
    server:
      security:
        useAuth: true
flyteadmin:
  secrets:
    oidc_client_secret: <client_secret generated by gitlab>
secrets:
  adminOauthClientCredentials:
    clientId: <client_id generated by gitlab>
    clientSecret: <client_secret generated by gitlab>

[1] https://docs.flyte.org/projects/cookbook/en/latest/auto/larger_apps/larger_apps_deploy.html#build-deploy-your-application-to-the-cluster [2] https://docs.flyte.org/en/latest/deployment/cluster_config/auth_setup.html [3] https://docs.gitlab.com/ee/integration/oauth_provider.html#view-all-authorized-applications

Hello! I’m trying to convert a Python dict to a LiteralMap and found the DictTransformer class. It states:

Transformer that transforms a univariate dictionary Dict[str, T] to a Literal Map

But it only seems to be able to transform it into a Literal. So how do I go from a Literal to a LiteralMap?

Hi, is it possible to send Platform Events to something like external Kafka cluster or is AWS SQS/SNS the only available option right now?

Hi Flyte community, Does using the pod template setting for task registration completely override the pod template that Flyte defaults to or does it add to it? I'd like to add the following setting so my raw container Flyte task pods can refer to the their corresponding settings (based on stage/app):

envFrom:
      - configMapRef:
          name: special-config
      - secretRef:
          name: special-config

https://raw.githubusercontent.com/kubernetes/website/main/content/en/examples/pods/pod-configmap-envFrom.yaml