https://flyte.org logo
Powered by
#announcements
Title
# announcements
n

Nicholas LoFaso

01/25/2022, 7:53 PM
Hi I would like to set an environment variable for every Pod that gets scheduled to k8s. What is the preferred way to do this? Specifically I’m trying to set the following for datadog metrics
Copy code
env:
    - name: DD_AGENT_HOST
      valueFrom:
          fieldRef:
              fieldPath: status.hostIP
I have tried creating a Pod task and specifying the 
env
using 
V1EnvVar
, but Flyte doesn’t seem to translate to this field to the Pod. The request/limits are set on the Pod as I expect.
Copy code
@task(
    task_config=Pod(
        pod_spec=V1PodSpec(
            containers=[
                V1Container(
                    name="primary",
                    resources=V1ResourceRequirements(
                        requests={"cpu": ".25", "memory": "250Mi"},
                        limits={"cpu": ".5", "memory": "500Mi"},
                    ),
                    env=[V1EnvVar(name="MY_VAL", value="TEST")],
                )
            ],
        ),
        primary_container_name="primary",
    )
)
am I missing something?
y

Yee

01/25/2022, 8:02 PM
you can also do
Copy code
kind: Service
apiVersion: v1
metadata:
  name: flytestatsd
  namespace: datadog
spec:
  selector:
    app: "your-datadog"
  ports:
  - protocol: UDP
    port: 8125
    targetPort: 8125
  topologyKeys:
    - "<http://kubernetes.io/hostname|kubernetes.io/hostname>"
    - "*"
and
Copy code
k8s:
  plugins:
    k8s:
      default-env-vars:
        - FLYTE_STATSD_HOST: "flytestatsd.datadog.svc.cluster.local"
        - FLYTE_STATSD_DISABLE_TAGS: "True"
but that latter block is how you do it
n

Nicholas LoFaso

01/25/2022, 8:05 PM
Hi Yee thanks! this seems like exactly what I’m looking for
what is the purpose of 
FLYTE_STATSD_DISABLE_TAGS
?
y

Yee

01/25/2022, 8:07 PM
i think we did it specifically for datadog
other systems might be better at handling the cardinality
n

Nicholas LoFaso

01/25/2022, 8:12 PM
good to know thanks. Will try this shortly
@Yee I’m seeing my custom metrics show up in DataDog now thank you. Should I also be seeing flyte custom metrics?
y

Yee

01/26/2022, 4:40 PM
do you want them? we do it for propeller and admin
let’s see
Copy code
flytepropeller:
  podAnnotations:
    <http://ad.datadoghq.com/flytepropeller.check_names|ad.datadoghq.com/flytepropeller.check_names>: |
      ["openmetrics"]
    <http://ad.datadoghq.com/flytepropeller.init_configs|ad.datadoghq.com/flytepropeller.init_configs>: |
      [{}]
    <http://ad.datadoghq.com/flytepropeller.instances|ad.datadoghq.com/flytepropeller.instances>: |
      [
        {
          "prometheus_url": "<http://%%host%>%:10254/metrics",
          "namespace": "propeller",
          "metrics": [ "*" ],
          "max_returned_metrics": 10000
        }
      ]
something like that will do it.
we have this also
Copy code
plugins:
      logs:
        cloudwatch-enabled: true
        cloudwatch-log-group: '/eks/opta-development/cluster'
        cloudwatch-region: us-east-2
        kubernetes-enabled: false
        templates:
        - displayName: Datadog
          templateUris:
          - |
            <https://app.datadoghq.com/logs?event&from_ts={{> .podUnixStartTime }}000&live=false&query=pod_name%3A{{ .podName }}&to_ts={{ .podUnixFinishTime }}000
it’s pretty hacky but you can get logs assuming it’s scraping logs.
22 Views
Powered by