"Build software like a tank." I am not sure where I read this, but I think about it a lot, especially when writing HTTP clients. Tanks are incredible machines - they are designed to move rapidly and protect their inhabitants in any kind of terrain, against enemy gunfire, or worse.
HTTP clients often run in unfriendly territory, because they usually involve a network connection between machines. Connections can fail, packets can be dropped, the other party may respond very slowly, or with a new unknown error message, or they might even change the API from under you. All of this means writing an HTTP client like a tank is difficult. Here are some examples of things that a desirable HTTP client would do for you, that are never there by default.
If a request fails to reach the remote server, we would like to retry it no matter what. We don't want to wait around for the server forever though, so we want to set a timeout on the connection attempt.
If we send the request but the remote server doesn't respond in a timely manner, we want to retry it, but only on requests where it is safe to send the request again - so called idempotent requests.
If the server returns an unexpected response, we want to always retry if the server didn't do any processing - a 429, 502 or a 503 response usually indicate this - as well as all idempotent requests.
Generally we want to sleep between retries to allow the remote connection/server to recover, so to speak. To help prevent thundering herd problems, we usually sleep with an exponential back off.
Here's an example of how you might code this:
def resilient_request(method, uri, retries): while True: try: resp = requests.request(method, uri) if resp.status < 300: break if resp.status in [429, 502, 503]: retries -= 1 if retries <= 0: raise time.sleep(2 ** (3 - retries)) continue if resp.status >= 500 and method in ['GET', 'PUT', 'DELETE']: retries -= 1 if retries <= 0: raise time.sleep(2 ** (3 - retries)) continue except (ConnectionError, ConnectTimeoutError): retries -= 1 if retries <= 0: raise time.sleep(2 ** (3 - retries)) except TimeoutError: if method in ['GET', 'PUT', 'DELETE']: retries -= 1 if retries <= 0: raise time.sleep(2 ** (3 - retries)) continue
Holy cyclomatic complexity, Batman! This suddenly got complex, and
the control flow here is not simple to follow, reason about, or test. Better
hope we caught everything, or we might end up in an infinite loop, or try to
resp when it has not been set. There are some parts of the above code
that we can break into sub-methods, but you can't make the code too much more
compact than it is there, since most of it is control flow. It's also a pain to
write this type of code and verify its correctness; most people just try once,
as this comment from the
pip library illustrates. This is a shame and
the reliability of services on the Internet suffers.
A better way
Andrey Petrov and I have been putting in a lot of work make it really, really easy for you to write resilient requests in Python. We pushed the complexity of the above code down into the urllib3 library, closer to the request that goes over the wire. Instead of the above, you'll be able to write this:
def retry_callable(method, response): """ Determine whether to retry this return ((response.status >= 400 and method in IDEMPOTENT_METHODS) or response.status in (429, 503)) retry = urllib3.util.Retry(read=3, backoff_factor=2, retry_callable=retry_callable) http = urllib3.PoolManager() resp = http.request(method, uri, retries=retry)
You can pass a callable to the retries object to determine the retry behavior
you'd like to see. Alternatively you can use the convenience
codes_whitelist helpers to specify which methods to retry.
retry = urllib3.util.Retry(read=3, backoff_factor=2, codes_whitelist=set([429, 500])) http = urllib3.PoolManager() resp = http.request(method, uri, retries=retry)
And you will get out the same results as the 30 lines above. urllib3 will do all of the hard work for you to catch the conditions mentioned above, with sane (read: non-intrusive) defaults.
This is coming soon to urllib3 (and with it, to Python Requests and pip); we're looking for a bit more review on the pull request before we merge it. We hope this makes it easier for you to write high performance HTTP clients in Python, and appreciate your feedback!
Thanks to Andrey Petrov for reading a draft of this post.
Liked what you read? I am available for hire.