<#6529 [BUG] PodTemplate Resources state pollution...
# flyte-github
c
#6529 [BUG] PodTemplate Resources state pollution between workflow runs Issue created by punkerpunker ### Flyte & Flytekit version
main
branch ### Describe the bug ### Summary The PodTemplateStore cache returns direct references to cached objects instead of copies, causing resource configurations from one workflow to leak into subsequent workflows. This results in workflows without inline resources unexpectedly inheriting resource specifications from previously executed workflows. ### Description When multiple workflows execute using the same pod template, the second and subsequent workflows may inherit resource configurations (CPU, memory, GPU, etc.) from earlier workflows, even when no inline resources are specified. This happens because the PodTemplateStore.LoadOrDefault() method returns a pointer to the cached pod template object, which gets mutated during resource processing. ### Root Cause In flyteplugins/go/tasks/pluginmachinery/flytek8s/pod_template_store.go, the LoadOrDefault() method returns a direct reference to the cached object:
Copy code
func (p *PodTemplateStore) LoadOrDefault(namespace string, podTemplateName string) *v1.PodTemplate {
    // ...
    if podTemplate, ok := podTemplates.Load(namespace); ok {
        return podTemplate.(*v1.PodTemplate)  // ← BUG: Returns pointer to cached object
    }
    // ...
}
### Expected behavior flytepropeller doesn't preserve resources different between runs ### Additional context to reproduce 1. Execute a workflow (Workflow A) that specifies inline resources (e.g., high CPU/memory requirements) 2. Execute a different workflow (Workflow B) using the same pod template but without inline resources 3. Observe that Workflow B unexpectedly uses the resource specifications from Workflow A 4. Restart flytepropeller to clear the cache 5. Execute Workflow B again - it now works correctly until another workflow with inline resources executes ### Screenshots No response ### Are you sure this issue hasn't been raised already? • Yes ### Have you read the Code of Conduct? • Yes flyteorg/flyte