Please could someone explain which cache the docs ...
# ask-the-community
Please could someone explain which cache the docs are referring too here
Another area of slowdown could be the size of the input-output cache that FlytePropeller maintains in-memory. This can be configured, while configuring the storage for FlytePropeller. Rule of thumb, for FlytePropeller with x memory limit, allocate x/2 to the cache
I believe this is the configuration option. Basically, propeller needs to read / write task input and output data to the blobstore frequently. It uses a write and read through cache for these operations to be more efficient. For example, when writing the output of a cached task it will immediately read it as an input to the next task (given a downstream task). There should be prometheus metrics exported on cache misses / hits to determine if this configuration is working effectively.
Thanks 🙂