Hey all wave I have encountered an issue where a single exec

Question

Hey all wave I have encountered an issue where a single execution apparently has invalid utf 8 data and the console fails to 1 show the workflow details on the ui and 2 fails to summarize all executions in the workflow entirely Two screenshots attached in thread Is there a way to simply delete that workflow execution and all its metadata with flyteclt in order to hopefully restore the general workflow page

Len Strnad · Answer

Workflow page fails to summarize all executions because of one execution

Len Strnad · Answer

I can see other executions but only after going back from a previous valid execution

Ketan (kumare3) · Answer

This is a bug we should handle

Ketan (kumare3) · Answer

I actually think it is the http get that s failing

Ketan (kumare3) · Answer

Can you share what is the bad data

Ketan (kumare3) · Answer

We should catch it at creation time

Len Strnad · Answer

Any suggestions of how to discover this bad data I am not the one who deployed Flyte and am just getting familiar with all the components fyi but I do have permissions

Ketan (kumare3) · Answer

Try flytectl

Ketan (kumare3) · Answer

Seems like versions

Len Strnad · Answer

`flytectl get workflow` with version

Len Strnad · Answer

I am looking at ```flytectl get execution admin endpoint $FLYTE HOST p project d development a4gbfch8hmrg67gv2pql output yaml``` and the only culprit I see might be the error message field

Len Strnad · Answer

Is it valid to take that error message and verbatim check if that is utf 8

Ketan (kumare3) · Answer

Ohh man this is bad we should be fixing this cc < Katrina Rogan> < Frank Flitton> can either of you help

Len Strnad · Answer

The error message seems to encode to utf 8 when I copy and paste and try to encode in python shrug cc ing < Bernhard Stadlbauer> since he might be able to dig deeper when he is back Monday

Len Strnad · Answer

The task that failed due to retried exhausted and OOMKilled fwiw

Frank Flitton · Answer

The 500 error code implies it may be in the api layer to me Can you share a screenshot of what the network tab in the chrome inspector looks like after a refresh That will help us narrow things down

Len Strnad · Answer

Of the console This is an error I see

Len Strnad · Answer

It looks like we get a 500 from this request URL ``````

Len Strnad · Answer

Hope that helps fwiw a colleague checked the table for the meta data and we couldn t find any invalid utf 8 data

Bernhard Stadlbauer · Answer

Did a quick check and couldn t find anything more than < Len Strnad> found ```flytectl get execution admin endpoint $FLYTE HOST p project d development a4gbfch8hmrg67gv2pql details``` also fails Had a look at the `node executions` flyteadmin table for that particular execution but most of the columns are of type `bytea` Happy to run a query against the table in case it helps

Len Strnad · Answer

Ok we archived the specific execution and all is well Not ideal but a decent workaround for now

Katrina Rogan · Answer

the latest flyteadmin actually has a fix for the utf 8 encoding which version are you on

Katrina Rogan · Answer

which should be picked up in the flyte 1 6 2 patch release

Len Strnad · Answer

helm chart v 1 6 1

Katrina Rogan · Answer

ah that might be it could you update and let us know if you still run into this

Len Strnad · Answer

Will do

Len Strnad · Answer

Thanks for all the help on this pray

Bernhard Stadlbauer · Answer

< Katrina Rogan> Updating to the latest flyteadmin has fixed things for us thank you