Bernhard Stadlbauer
07/12/2022, 6:21 AMDan Rammer (hamersaw)
07/12/2022, 9:10 PMDefaultPodTemplateName
option. Basically FlytePropeller looks for a PodTemplate in the namespace where the Pod is being created (e.x. flytesnacks-deployment
for the project-domain be default), if that doesn't exist then it looks for a PodTemplate in the namespace that FlytePropeller is deployed in (typically flyte
) and uses that PodSpec as the base for configuring Pods. So PodTemplate values will be overridden by the flytek8s configuration options, which will be overridden by specific task configuration.Bernhard Stadlbauer
07/13/2022, 7:56 AMapiVersion: v1
kind: PodTemplate
metadata:
name: flyte-default-template
namespace: {{ .Release.Namespace }}
template:
metadata:
spec:
volumes:
- hostPath:
path: /var/run/datadog/
name: apmsocketpath
containers:
- name: noop
image: <http://docker.io/rwgrim/docker-noop|docker.io/rwgrim/docker-noop> # (<http://docker.io/rwgrim/docker-noop>)
volumeMounts:
- name: apmsocketpath
mountPath: /var/run/datadog
but as the noop
container gets completely overwritten (if I'm not mistaken, the related code is here) this won't work.
Are we missing something here?
One thing we could think of is to merge container specs instead of overwriting them?Dan Rammer (hamersaw)
07/13/2022, 4:00 PMBernhard Stadlbauer
07/13/2022, 8:08 PMDan Rammer (hamersaw)
07/13/2022, 9:39 PMAilin Yu
07/18/2022, 7:01 PMDan Rammer (hamersaw)
07/18/2022, 8:22 PMAilin Yu
07/18/2022, 9:34 PMTaskExecMetadata.GetAnnotations()
but that feels like maybe a bit of an incorrect way to use a getter functionDan Rammer (hamersaw)
07/19/2022, 9:16 PMAilin Yu
07/20/2022, 1:09 AMpodTemplate.Template.Annotations[PrimaryContainerKey] = taskCtx.TaskExecutionMetadata().GetTaskExecutionID().GetGeneratedName()
Since the taskCtx
here is the same thing that gets passed into buildPodSpec
and eventually into ToK8sContainer
to set the containerName, I thought it should be the same thing. Then BuildPodWithSpec can get the annotation out of the podTemplate and try to match that to a container name.
I otherwise realized I was missing a couple of details on locally testing, which I posted about up in the main channel!Dan Rammer (hamersaw)
07/26/2022, 11:48 AMgetPrimaryContainerName
to the podBuilder
interface here . This would, of course need to be implemented for the container (use the task name as you suggested) and sidecar (refactor the current code) plugins. Then something like below in the BuildResource function:
podSpec, err := builder.buildPodSpec(ctx, task, taskCtx)
if err != nil {
return nil, err
}
podSpec.ServiceAccountName = flytek8s.GetServiceAccountNameFromTaskExecutionMetadata(taskCtx.TaskExecutionMetadata())
primaryContainerName, err := builder.getPrimaryContainerName(ctx, task, taskCtx)
if err != nil {
return nil, err
}
podTemplate := flytek8s.DefaultPodTemplateStore.LoadOrDefault(taskCtx.TaskExecutionMetadata().GetNamespace())
pod, err := flytek8s.BuildPodWithSpec(podTemplate, podSpec, primaryContainerName)
if err != nil {
return nil, err
}
pod.Annotations[PrimaryContainerName] = primaryContainerName
Then we can pass the primaryContainerName
as a string to the BuildPodWithSpec
function and don't have to rely on annotations, but we still ensure the annotation is set. This will take a little bit of refactoring, but it should be fairly straight forward. We can certainly meetup as well to iron out any issues in the dev environment and sort through this.Ailin Yu
07/26/2022, 8:03 PMpodBuilder
a bit more to understand how its used currently, with regards to refactoring. Otherwise I’m just wrestling with my dev setup right now to try to get a test task to try out!