This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: HTTPSConnection request hangs
Type: behavior Stage: resolved
Components: Library (Lib) Versions: Python 3.1, Python 3.2, Python 2.7
process
Status: closed Resolution: out of date
Dependencies: Superseder:
Assigned To: Nosy List: BreamoreBoy, ajaksu2, giampaolo.rodola, grolich, neologix, orsenthil
Priority: normal Keywords:

Created on 2006-07-31 14:11 by grolich, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Messages (5)
msg60955 - (view) Author: Noam Taich (grolich) Date: 2006-07-31 14:11
with python 2.3 and above there is a problem with
HTTPSConnection that comes to life in the following code:

conn = HTTPSConnection(server,port)
conn.connect()
conn.request('POST', '/', data, headers)

This works fine, unless the remote host closes
unexpectedly (unexpectedly for the client. say, reboot
-fn or reset button)

sometimes (not always) when that happens, python gets
stuck on the conn.request command, it never returns or
raises an eyebrow (or an exception...) even after the
remote host is back online. 

problem only happens once every many such occurences

the remote host is running apache (which is really used
a proxy to a python http server on the same remote
machine).
msg84492 - (view) Author: Daniel Diniz (ajaksu2) * (Python triager) Date: 2009-03-30 04:22
This needs a test case reproducing the 'unexpected close'.
msg114810 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2010-08-24 19:56
Is this still a problem with later versions of Python?
msg116967 - (view) Author: Charles-François Natali (neologix) * (Python committer) Date: 2010-09-20 20:06
Skimming through the code, the only place where we can reasonably block is inside HTTPConnection._send_output(), when the data is actually sent to the socket.
What probably happens is the following:
- connect() call succeeded, we have an established TCP connection
- before request() is called - or in the middle of the sending - the host at the other host goes down in brutal way, so that we don't receive a FIN/RST: the TCP stack has no clue that the remote host went down
- we keep sending data through the socket, until the socket write buffer fills up, and then, since httplib uses blocking sockets by default, we block on the send() call

Now, depending on the TCP stack setting, after a given number of retries the stack will decide the other host went down, and return an error. But this can take a long time (under Linux, it's set by net.ipv4.tcp_retries2 sysctl by default to 30 minutes).
The only thing that surprises me here is that when the host is back online, we should get a RST, but it depends on the OS, the timeouts, maybe there's a statefull firewall, etc.

So I'd say it's not a httplib issue here, it's just a standard behaviour of a TCP connection when an host goes down.

Note that the solution is simply to use non-blocking socket, and it's already implemented. Heck, it's even documented actually:

class httplib.HTTPConnection(host[, port[, strict[, timeout[, source_address]]]])
If the optional timeout parameter is given, blocking operations (like connection attempts) will timeout after that many seconds (if it is not given, the global default timeout setting is used).

So for me it's not an issue, but given the lack of infos, maybe I got it completely wrong ;-)
msg118999 - (view) Author: Senthil Kumaran (orsenthil) * (Python committer) Date: 2010-10-18 10:08
Not an issue anymore as socket timeout at client is supposed to happen with the connection is hung due to unresponsive host.
The original bug was raised py2.3. Closing it as out of date.
History
Date User Action Args
2022-04-11 14:56:19adminsetgithub: 43752
2010-10-18 10:08:54orsenthilsetstatus: open -> closed

nosy: + orsenthil
messages: + msg118999

resolution: out of date
stage: test needed -> resolved
2010-09-20 20:06:42neologixsetnosy: + neologix
messages: + msg116967
2010-08-24 20:02:21giampaolo.rodolasetnosy: + giampaolo.rodola
2010-08-24 19:56:00BreamoreBoysetnosy: + BreamoreBoy

messages: + msg114810
versions: + Python 3.1, Python 2.7, Python 3.2, - Python 2.6, Python 3.0
2009-03-30 04:22:32ajaksu2setversions: + Python 2.6, Python 3.0
nosy: + ajaksu2

messages: + msg84492

type: behavior
stage: test needed
2006-07-31 14:11:12grolichcreate