https://flyte.org
Join Slack
is it possible to run a workflow in a different namespace other than `project-domain`?
# flyte-support
s
is it possible to run a workflow in a different namespace other than 
project-domain
?
found docs about namespace_mapping, are there docs/guides on how to change and apply those configurations in general?
m
maybe reference LP/Workflow is useful here https://docs.flyte.org/projects/flytekit/en/latest/generated/flytekit.reference_workflow.html#flytekit-reference-workflow if you are calling it from different project without config changes
🤔 1
f
When you launch a workflow, the execution domain is different from the original domain and project
You do not need to use reference workflows, just simply launch an execution in a different project and domain
s
is it possible to launch executions in a name space not following the 
project-domain
convention, eg a namespace called 
experiments
?
f
Ohh are you talking about k8s namespaces
If so yes
s
yes k8 namespaces
f
K8s namespaces are independent from project domain, default is to create project-domain namespaces
You will Have to configure
The backend for this
s
namespace_mapping and template?
f
Yup I think so - AFK, friday evening 😬
s
no worries, let's continue when you have time have a good weekend!
🙇 1
f
Thank you, ya kids time now
s
@freezing-airport-6809 when you get a chance, could you share docs/guides on how to change the default behavior (creating project-domain namespaces)?
f
Cc @thankful-minister-83577
t
is this what you’re looking for @shy-accountant-549
s
I believe so
I used flyte-binary helm chart to deploy, do I just need to update this cm?
Copy code
apiVersion: v1
data:
  000-core.yaml: |
    admin:
      endpoint: localhost:8089
      insecure: true
    catalog-cache:
      endpoint: localhost:8081
      insecure: true
      type: datacatalog
    cluster_resources:
      standaloneDeployment: false
      templatePath: /etc/flyte/cluster-resource-templates
    logger:
      show-source: true
      level: 1
    propeller:
      create-flyteworkflow-crd: true
    webhook:
      certDir: /var/run/flyte/certs
      localCert: true
      secretName: flyte-backend-flyte-binary-webhook-secret
      serviceName: flyte-backend-flyte-binary-webhook
      servicePort: 443
  001-plugins.yaml: |
    tasks:
      task-plugins:
        enabled-plugins:
          - container
          - sidecar
          - K8S-ARRAY
        default-for-task-types:
          - container: container
          - container_array: K8S-ARRAY
    plugins:
      logs:
        kubernetes-enabled: false
        cloudwatch-enabled: false
        stackdriver-enabled: false
      k8s-array:
        logs:
          config:
            kubernetes-enabled: false
            cloudwatch-enabled: false
            stackdriver-enabled: false
  002-database.yaml: |
    database:
      postgres:
        username: postgres
        passwordPath: /var/run/secrets/flyte/db-pass
        host: <RDS_HOST_DNS>
        port: 5432
        dbname: app
        options: "sslmode=disable"
  003-storage.yaml: |
    propeller:
      rawoutput-prefix: s3://<USER_DATA_BUCKET_NAME>/data
    storage:
      type: stow
      stow:
        kind: s3
        config:
          region: <AWS_REGION>
          disable_ssl: false
          v2_signing: false
          auth_type: iam
      container: <BUCKET_NAME>
  010-inline-config.yaml: |
    plugins:
      k8s:
        default-env-vars:
        - AWS_METADATA_SERVICE_TIMEOUT: 5
        - AWS_METADATA_SERVICE_NUM_ATTEMPTS: 20
        inject-finalizer: true
    storage:
      cache:
        max_size_mbs: 100
        target_gc_percent: 100
    task_resources:
      defaults:
        cpu: 200m
        memory: 1Gi
        storage: 1Gi
      limits:
        cpu: 32
        memory: 128Gi
        storage: 512Gi
kind: ConfigMap
metadata:
  labels:
    <http://app.kubernetes.io/instance|app.kubernetes.io/instance>: flyte-backend
    <http://app.kubernetes.io/managed-by|app.kubernetes.io/managed-by>: Helm
    <http://app.kubernetes.io/name|app.kubernetes.io/name>: flyte-binary
    <http://app.kubernetes.io/version|app.kubernetes.io/version>: 1.16.0
    <http://helm.sh/chart|helm.sh/chart>: flyte-binary-v1.4.3
  name: flyte-backend-flyte-binary-config
  namespace: flyte
or is it not possible with the flyte-binary chart? @thankful-minister-83577
t
it is possible.
put it under the inline section?
just at the top level under the inline section should do it.
s
like this?
Copy code
010-inline-config.yaml: |
    namespace_mapping:
      template: custom_namespace 
    plugins:
      k8s:
        default-env-vars:
        - AWS_METADATA_SERVICE_TIMEOUT: 5
        - AWS_METADATA_SERVICE_NUM_ATTEMPTS: 20
        inject-finalizer: true
    storage:
      cache:
        max_size_mbs: 100
        target_gc_percent: 100
    task_resources:
      defaults:
        cpu: 200m
        memory: 1Gi
        storage: 1Gi
      limits:
        cpu: 32
        memory: 128Gi
        storage: 512Gi
kind: ConfigMap
t
yeah…
but you’re changing the helm chart right?
not like k edit cm
s
for all the tasks running in a namespace called 
custom_namespace
?
yeah I am patching with kustomize
t
okay
well changing in helm is best
otherwise the next time you apply the helm chart the changes go away
what do you mean?
for all the tasks running in a namespace called 
custom_namespace
?
s
I mean if I want to make all the tasks running in a k8 namespace called 
custom_namespace
, no matter what flyte projects or domains these tasks belong. is it the right change?
Copy code
010-inline-config.yaml: |
    namespace_mapping:
      template: custom_namespace 

kind: ConfigMap
t
yes that is correct
s
when you say changing in helm, do you mean adding it to 
values.yaml
for helm install to use?
another question about using inline config: Can all the config options in the documentation be updated/overriden with inline-config.yaml?
t
yes
just make sure you do it correctly - a merge happens in the backend. so if you overwrite a list, you need to re-specify all the elements of the list that you want.
lists are replaced, maps are merged.
👍 1
s
this is super helpful gratitude thank you
f
Cc @average-finland-92144 can we capture this
👍🏽 1
338 Views
Open in Slack
PreviousNext
Powered by

Flyte

Flyte enables production-grade orchestration for machine learning workflows and data processing created to accelerate local workflows to production.

Powered by