Is there a hard limit on allowed for cache lookup We are run

Question

Is there a hard limit on allowed for cache lookup We are running into this error ```array size gt max allowed for cache lookup requested 15434 writer allowed 10000 reader allowed 10000 ```

Louis DiNatale · Answer

The answer was it was a simple config change found here <https docs flyte org en latest deployment cluster config flytepropeller config html catalogcache catalog config>

Louis DiNatale · Answer

a concern to think about is memory on the propellor

Ketan (kumare3) · Answer

Yes I would also recommend running sharded propeller