<#4183 Implement Spark pod template overrides> Pul...
# flyte-github
a
#4183 Implement Spark pod template overrides Pull request opened by andrewwdye Tracking issue #4105 Describe your changes Create driver, executor, and spark application spec from optional pod template. Uses ToK8sPodSpec as a shared solution for merging defaults with container or pod template target overrides. Driver and executor specs will share the same pod template values. See note below on this choice. Why use the same pod template for driver and executor? As part of #4179 we had a discussion about whether or not to use the task-level pod template, or introduce role-specific pod templates for driver and executor. For now this change adopts the task-level pod template pattern used by other k8s plugins. We may end up moving this and others to role-specific templates in the future. Test plan Unit tests I refactored, extended, and renamed the existing container based Spark unit test (
TestBuildResourceContainer
). I added a new
TestBuildResourcePodTemplate
test to cover most of the interface. Sanity check diff of driver and executor pods locally 1. Setup and build single binary 2. Add
spark
to enabled-plugins to the config following this guide 3. Add spark role, role binding, and service account template files from this guide to
$HOME/.flyte/cluster-resource-templates/
4. Register and run a spark workflow following this guide 5. Compare driver and executor pods between master and this branch Verify behavior of specifying
@task(pod_template=...)
locally
Modified the
my_spark
example to use a pod template
Copy code
@task(
    ...
    pod_template=PodTemplate(
        primary_container_name="primary",
        labels={"lKeyA": "lValA", "lKeyB": "lValB"},
        annotations={"aKeyA": "aValA", "aKeyB": "aValB"},
        pod_spec=V1PodSpec(
            containers=[
                V1Container(
                    name="primary",
                    image=custom_image,
                    command="echo",
                    args=["wow"],
                ),
            ],
            tolerations=[
                V1Toleration(
                    key="x/custom",
                    operator="Equal",
                    value="foo",
                    effect="NoSchedule",
                ),
            ],
        ),
    ),
)
And verified that the resulting pod had tolerations set
Copy code
❯ kubectl get pod -n flytesnacks-development a68zfbjq8r74mjb9q5tq-n0-0-driver -o yaml | grep "tolerations:" -A 12
  tolerations:
  - effect: NoExecute
    key: node.kubernetes.io/not-ready
    operator: Exists
    tolerationSeconds: 300
  - effect: NoExecute
    key: node.kubernetes.io/unreachable
    operator: Exists
    tolerationSeconds: 300
  - effect: NoSchedule
    key: x/custom
    operator: Equal
    value: foo
Check all the applicable boxes ☐ I updated the documentation accordingly. ☑︎ All new and existing tests passed. ☑︎ All commits are signed-off. flyteorg/flyte GitHub Actions: Docker Build Images (flytepropeller) / build_docker GitHub Actions: Docker Build Images (flytecopilot) / build_docker GitHub Actions: Docker Build Images (datacatalog) / build_docker GitHub Actions: Docker Build Images (flyteadmin) / build_docker GitHub Actions: generate_kustomize GitHub Actions: publish-development-helm-chart (flyteagent) GitHub Actions: docs GitHub Actions: publish-development-helm-chart (flyte-core) GitHub Actions: compile GitHub Actions: unpack-envvars GitHub Actions: publish-development-helm-chart (flyte-binary) GitHub Actions: Check for spelling errors GitHub Actions: generate_helm ✅ 2 other checks have passed 2/15 successful checks