LiveServerTestCase non-deterministic Remote end closed connection without response

[Rares Vernica]

Hello,

I have a test script starting a live server using LiveServerTestCase. The
test script issues requests to the server using requests.Session. At random
points in the script the Django server decides to close the connection
without a response. Here are some more details.

My test script is not testing a Django application, but a non-Django
component that is supposed to interact with a Django Rest Framework. So, my
non-Django component uses requests.Session to connect and interact with
Django. Hence my test script uses requests.Session as well.

This works well on Python 2.7 and Django 1.11.15, but I'm migrating to
Python 3.6 and Django 2.1.1. I see my test script making progress through
the test case, and at a random point it triggers the following exception:


----------------------------------------------------------------------
Traceback (most recent call last):
File "...local/lib/python3.6/site-packages/urllib3/connectionpool.py",
line 600, in urlopen
chunked=chunked)
File "...local/lib/python3.6/site-packages/urllib3/connectionpool.py",
line 384, in _make_request
six.raise_from(e, None)
File "<string>", line 2, in raise_from
File "...local/lib/python3.6/site-packages/urllib3/connectionpool.py",
line 380, in _make_request
httplib_response = conn.getresponse()
File "/usr/lib64/python3.6/http/client.py", line 1331, in getresponse
response.begin()
File "/usr/lib64/python3.6/http/client.py", line 297, in begin
version, status, reason = self._read_status()
File "/usr/lib64/python3.6/http/client.py", line 266, in _read_status
raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without
response

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "...local/lib/python3.6/site-packages/requests/adapters.py", line
445, in send
timeout=timeout
File "...local/lib/python3.6/site-packages/urllib3/connectionpool.py",
line 638, in urlopen
_stacktrace=sys.exc_info()[2])
File "...local/lib/python3.6/site-packages/urllib3/util/retry.py", line
367, in increment
raise six.reraise(type(error), error, _stacktrace)
File "...local/lib/python3.6/site-packages/urllib3/packages/six.py", line
685, in reraise
raise value.with_traceback(tb)
File "...local/lib/python3.6/site-packages/urllib3/connectionpool.py",
line 600, in urlopen
chunked=chunked)
File "...local/lib/python3.6/site-packages/urllib3/connectionpool.py",
line 384, in _make_request
six.raise_from(e, None)
File "<string>", line 2, in raise_from
File "...local/lib/python3.6/site-packages/urllib3/connectionpool.py",
line 380, in _make_request
httplib_response = conn.getresponse()
File "/usr/lib64/python3.6/http/client.py", line 1331, in getresponse
response.begin()
File "/usr/lib64/python3.6/http/client.py", line 297, in begin
version, status, reason = self._read_status()
File "/usr/lib64/python3.6/http/client.py", line 266, in _read_status
raise RemoteDisconnected("Remote end closed connection without"
urllib3.exceptions.ProtocolError: ('Connection aborted.',
RemoteDisconnected('Remote end closed connection without response',))

...
File "...local/lib/python3.6/site-packages/requests/sessions.py", line
559, in post
return self.request('POST', url, data=data, json=json, **kwargs)
File "...local/lib/python3.6/site-packages/requests/sessions.py", line
512, in request
resp = self.send(prep, **send_kwargs)
File "...local/lib/python3.6/site-packages/requests/sessions.py", line
622, in send
r = adapter.send(request, **kwargs)
File "...local/lib/python3.6/site-packages/requests/adapters.py", line
495, in send
raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.',
RemoteDisconnected('Remote end closed connection without response',))


Repeated runs of the test script will progress more or less through the
test cases. It is not deterministic.

On start-up the test script creates a requests.Session and authenticates to
Django. Then it will issue GET/POST requests against a Django Rest
Framework API. I wonder why are the connection closed. I'm using requests
2.19.1.

Does this sound familiar? Is this related to a change in Django 2 that I'm
missing?

Thanks!
Rares

[a***@gaa9.com]

Any updates on this issue? I'm having the same issue right now.

Post by Rares Vernica
Hello,
I have a test script starting a live server using LiveServerTestCase. The
test script issues requests to the server using requests.Session. At
random points in the script the Django server decides to close the
connection without a response. Here are some more details.
My test script is not testing a Django application, but a non-Django
component that is supposed to interact with a Django Rest Framework. So, my
non-Django component uses requests.Session to connect and interact with
Django. Hence my test script uses requests.Session as well.
This works well on Python 2.7 and Django 1.11.15, but I'm migrating to
Python 3.6 and Django 2.1.1. I see my test script making progress through
----------------------------------------------------------------------
File "...local/lib/python3.6/site-packages/urllib3/connectionpool.py",
line 600, in urlopen
chunked=chunked)
File "...local/lib/python3.6/site-packages/urllib3/connectionpool.py",
line 384, in _make_request
six.raise_from(e, None)
File "<string>", line 2, in raise_from
File "...local/lib/python3.6/site-packages/urllib3/connectionpool.py",
line 380, in _make_request
httplib_response = conn.getresponse()
File "/usr/lib64/python3.6/http/client.py", line 1331, in getresponse
response.begin()
File "/usr/lib64/python3.6/http/client.py", line 297, in begin
version, status, reason = self._read_status()
File "/usr/lib64/python3.6/http/client.py", line 266, in _read_status
raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without
response
File "...local/lib/python3.6/site-packages/requests/adapters.py", line
445, in send
timeout=timeout
File "...local/lib/python3.6/site-packages/urllib3/connectionpool.py",
line 638, in urlopen
_stacktrace=sys.exc_info()[2])
File "...local/lib/python3.6/site-packages/urllib3/util/retry.py", line
367, in increment
raise six.reraise(type(error), error, _stacktrace)
File "...local/lib/python3.6/site-packages/urllib3/packages/six.py",
line 685, in reraise
raise value.with_traceback(tb)
File "...local/lib/python3.6/site-packages/urllib3/connectionpool.py",
line 600, in urlopen
chunked=chunked)
File "...local/lib/python3.6/site-packages/urllib3/connectionpool.py",
line 384, in _make_request
six.raise_from(e, None)
File "<string>", line 2, in raise_from
File "...local/lib/python3.6/site-packages/urllib3/connectionpool.py",
line 380, in _make_request
httplib_response = conn.getresponse()
File "/usr/lib64/python3.6/http/client.py", line 1331, in getresponse
response.begin()
File "/usr/lib64/python3.6/http/client.py", line 297, in begin
version, status, reason = self._read_status()
File "/usr/lib64/python3.6/http/client.py", line 266, in _read_status
raise RemoteDisconnected("Remote end closed connection without"
urllib3.exceptions.ProtocolError: ('Connection aborted.',
RemoteDisconnected('Remote end closed connection without response',))
...
File "...local/lib/python3.6/site-packages/requests/sessions.py", line
559, in post
return self.request('POST', url, data=data, json=json, **kwargs)
File "...local/lib/python3.6/site-packages/requests/sessions.py", line
512, in request
resp = self.send(prep, **send_kwargs)
File "...local/lib/python3.6/site-packages/requests/sessions.py", line
622, in send
r = adapter.send(request, **kwargs)
File "...local/lib/python3.6/site-packages/requests/adapters.py", line
495, in send
raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.',
RemoteDisconnected('Remote end closed connection without response',))
Repeated runs of the test script will progress more or less through the
test cases. It is not deterministic.
On start-up the test script creates a requests.Session and authenticates
to Django. Then it will issue GET/POST requests against a Django Rest
Framework API. I wonder why are the connection closed. I'm using requests
2.19.1.
Does this sound familiar? Is this related to a change in Django 2 that I'm
missing?
Thanks!
Rares

[AdrianB]

I'm also seeing this with a slightly simpler scenario that also involves a
requests.get inside a LiveServerTestCase. Success/failure is random with
failures reporting the same ConnectionError message. Any ideas?