Babis Kiosidis

07/20/2022, 1:15 PM
Hey gcp friends! We were seeing elevated RESOURCE_EXCHAUSTED (quota reached) when writing various files from flytepropeller to GCS. I rewrote the logic in the stow library to combine 2 write requests in 1 and got some very nice results. • average latency dropped to 200ms from 2-8sec (fig 1) • some (previously) silenced errors started appearing in the reconciliation loop due to gcs errors (fig3) Great performance improvement overall as RPCs and Quotas reached (fig2) were drastically reduced as well. Soon to be released on flytepropeller as well
🔥 8
❤️ 6

Ketan (kumare3)

07/20/2022, 1:19 PM
Cc @jeev / @Arief Rahmansyah / @Nicholas LoFaso / @austin and others This is a massive improvement- @Samhita Alla my vote for contributor of the month
👍 4
Cc @Fabio Grätz also
This will significantly improve perceived performance

Nicholas LoFaso

07/20/2022, 1:25 PM
that’s awesome thanks for investigating/improving!

Fabio Grätz

07/20/2022, 3:01 PM
Wow nice