Hey gcp friends We were seeing elevated RESOURCE EXCHAUSTED

Question

Hey gcp friends We were seeing elevated RESOURCE EXCHAUSTED quota reached when writing various files from flytepropeller to GCS I rewrote the logic in the stow library to <https github com flyteorg stow pull 6|combine 2 write requests in 1> and got some very nice results average latency dropped to 200ms from 2 8sec fig 1 some previously silenced errors started appearing in the reconciliation loop due to gcs errors fig3 Great performance improvement overall as RPCs and Quotas reached fig2 were drastically reduced as well Soon to be released on flytepropeller as well

Ketan (kumare3) · Answer

Cc < jeev> < Arief Rahmansyah> < Nicholas LoFaso> < austin> and others This is a massive improvement < Samhita Alla> my vote for contributor of the month

Ketan (kumare3) · Answer

Cc < Fabio Grätz> also

Ketan (kumare3) · Answer

This will significantly improve perceived performance

Nicholas LoFaso · Answer

that s awesome thanks for investigating improving

Fabio Grätz · Answer

Wow nice