Modeling Token Buckets in PlusCal and TLA+
Murat (noreply@blogger.com)
Retry storms are infamous in distributed systems. It is easy to run into them. Inevitably, a downstream service experiences a hiccup, so your clients automatically retry their failed requests. Those retries add more load to the struggling service, causing more failures, which trigger more retries. Before you know it, the tiny unavailability cascades into a full-blown self-inflicted denial of service. Token Bucket is a popular technique that helps with gracefully avoiding retry storms. Here is ho