Hi all! Can anyone tell me the difference between...
# ask-the-community
x
Hi all! Can anyone tell me the difference between a sidecar task and python task? I can’t seem to find much doc about it. Thanks! My issue is that I don’t know how they are configured
k
x
I see. And the sidecar task seems to only use the
latest
image. Is there a way to specify the image tag the sidecar task container?
k
you could do that by
Copy code
pod_spec=V1PodSpec(
            containers=[
                V1Container(
                    name="primary",
                    image=<image_name>
x
got it, thanks
@Kevin Su have you seen this issue? It seems to only happen to sidecar tasks
k
did you push the image to the registry? it seems like propeller can’t pull the image
x
I’m pretty sure I did, the other python tasks using the same image in the same workflow succeeded
k
could you also share the propeller logs
x
How do I check the propeller logs? I’m not managing the infrastructure, so I’m not familiar with the setup
k
kubectl logs <flytepropeller-pod-name> -n flyte. or go to kubernetes dashboard to see the logs
x
thanks! these are relevant logs
Copy code
Failed to record taskEvent, error [EventAlreadyInTerminalStateError: conflicting events; destination: ABORTED, caused by [rpc error: code = FailedPrecondition desc = invalid phase change from FAILED to ABORTED for task execution {resource_type:TASK project:"melody" domain:"staging" name:"atomwise.atomnet.flyte.tasks.train" version:"root-a187ff5188044973986b250c791f6b8e-cVO2gVPDzJ-3f9oE-wCwCA=="  node_id:"n11-1-dn2" execution_id:<project:"melody" domain:"staging" name:"root-i85xkgsuskuxsl1quizkjw" >  0 {} [] 0}]]. Trying to record state: ABORTED. Ignoring this error!

Workflow namespace[project-staging]/name[root-i85xkgsuskuxsl1quizkjw] has already been terminated.

Trying to abort a node in state [Failed]
It seems a grpc issue. Is there a timeout setting we can increase?
k
change from FAILED to ABORTED
how did you know it’s grpc issue from this log? do you have any other logs in propeller
x
i just guessed because
rpc error:
144 Views