Hi all! Can anyone tell me the difference between a sidecar task and python task? I can’t seem to find much doc about it. Thanks! My issue is that I don’t know how they are configured

if you specify pod config (k8s pod spec) for the task, the task type will become sidecar. https://docs.flyte.org/projects/cookbook/en/latest/auto/integrations/kubernetes/pod/pod.html#add-additional-properties-to-the-task-container

I see. And the sidecar task seems to only use the

latest

image. Is there a way to specify the image tag the sidecar task container?

you could do that by

pod_spec=V1PodSpec(
            containers=[
                V1Container(
                    name="primary",
                    image=<image_name>

got it, thanks

did you push the image to the registry? it seems like propeller can’t pull the image

I’m pretty sure I did, the other python tasks using the same image in the same workflow succeeded

could you also share the propeller logs

How do I check the propeller logs? I’m not managing the infrastructure, so I’m not familiar with the setup

kubectl logs <flytepropeller-pod-name> -n flyte. or go to kubernetes dashboard to see the logs

thanks! these are relevant logs

Failed to record taskEvent, error [EventAlreadyInTerminalStateError: conflicting events; destination: ABORTED, caused by [rpc error: code = FailedPrecondition desc = invalid phase change from FAILED to ABORTED for task execution {resource_type:TASK project:"melody" domain:"staging" name:"atomwise.atomnet.flyte.tasks.train" version:"root-a187ff5188044973986b250c791f6b8e-cVO2gVPDzJ-3f9oE-wCwCA=="  node_id:"n11-1-dn2" execution_id:<project:"melody" domain:"staging" name:"root-i85xkgsuskuxsl1quizkjw" >  0 {} [] 0}]]. Trying to record state: ABORTED. Ignoring this error!

Workflow namespace[project-staging]/name[root-i85xkgsuskuxsl1quizkjw] has already been terminated.

Trying to abort a node in state [Failed]

change from FAILED to ABORTED

how did you know it’s grpc issue from this log? do you have any other logs in propeller

i just guessed because

rpc error: