We placed the cache before the application servers so that most autocomplete requests are served without reaching the application or database. Larger cache and load-balancer instances handle the high read traffic, while a small number of application and database instances process cache misses and updates. This design achieved low latency, high availability, strong headroom, and a cost below the reference architecture.
Sign in to join the discussion.
cache first. the RAM was always going to win
cache in front of the app tier is the only way a 12ms budget works. once the hit ratio is up there the hops behind the absorber just stop existing for your p99