Discussion:
Strange Errors with FastCGI when uploading files
Dirk van Oosterbosch, IR labs
2007-06-26 15:36:34 UTC
Permalink
Hi All,

I deployed an app with uploading functionality to a server running
FastCGI.
When I had it tested by my users, the site became completely
unresponsive, returning nothing but Error 500's.

In the log I find a lot of these:
Unhandled exception in thread started by <bound method
ThreadPool._worker of <flup.server.threadpool.ThreadPool object at
0x825e10>>
Traceback (most recent call last):
File "/usr/local/lib/python2.5/site-packages/flup-0.5-py2.5.egg/
flup/server/threadpool.py", line 103, in _worker
job.run()
File "/usr/local/lib/python2.5/site-packages/flup-0.5-py2.5.egg/
flup/server/fcgi_base.py", line 645, in run
except (select.error, socket.error), e:
AttributeError: 'NoneType' object has no attribute 'error'

Does anybody know, what could cause this or (better yet) how to fix it?
best
dirk



-----------------------------
Dirk van Oosterbosch
de Wittenstraat 225
1052 AT Amsterdam
the Netherlands

http://labs.ixopusada.com
-----------------------------



--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to django-***@googlegroups.com
To unsubscribe from this group, send email to django-users-***@googlegroups.com
For more options, visit this group at http://groups.google.com/group/django-users?hl=en
-~----------~----~----~----~------~----~------~--~---
Dirk van Oosterbosch, IR labs
2007-06-26 23:18:38 UTC
Permalink
Replying myself to add some information and ask some extra questions.

I just had contact with the maintainer of my shared host. He warned
that memory exceeding the (shared usage) limit could cause strange
errors. And that memory usage should be dealt with carefully.
He adviced me to save the uploaded files directly to disk instead of
storing them in memory first. Can I do this at all with Django?

I am now using (in the view)
if request.method == 'POST':
if 'uploadedfile' in request.FILES:
new_data = request.POST.copy()
new_data.update(request.FILES)
form = UploadForm(new_data)
if form.is_valid():
success = form.save(request.user)

in the UploadForm class
def clean_uploadedfile(self):
file_data = self.clean_data['uploadedfile']
....
return file_data

in save()
if self.clean_data['uploadedfile']:
uploadedfile_data = self.clean_data['uploadedfile']
file_path = "path/to/saved/file/%s" % (uploadedfile_data['filename'])
save_content_to_file(file_path, uploadedfile_data['content'])

and only finally in save_content_to_file()
def save_content_to_file(filename, content):
try:
fileObj = open(filename, "wb")
try:
fileObj.write(content)
finally:
fileObj.close()


So I am guessing I'm consuming quite some memory to deal with this
incoming request data.
How could I improve my code, so it will use its memory a bit more
economically?

And is there a way to have the files in the requested post data
streamed into files on disk directly?

Thanks,
dirk
Post by Dirk van Oosterbosch, IR labs
Hi All,
I deployed an app with uploading functionality to a server running
FastCGI.
When I had it tested by my users, the site became completely
unresponsive, returning nothing but Error 500's.
Unhandled exception in thread started by <bound method
ThreadPool._worker of <flup.server.threadpool.ThreadPool object at
0x825e10>>
File "/usr/local/lib/python2.5/site-packages/flup-0.5-py2.5.egg/
flup/server/threadpool.py", line 103, in _worker
job.run()
File "/usr/local/lib/python2.5/site-packages/flup-0.5-py2.5.egg/
flup/server/fcgi_base.py", line 645, in run
AttributeError: 'NoneType' object has no attribute 'error'
Does anybody know, what could cause this or (better yet) how to fix it?
best
dirk
-----------------------------
Dirk van Oosterbosch
de Wittenstraat 225
1052 AT Amsterdam
the Netherlands

http://labs.ixopusada.com
-----------------------------



--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to django-***@googlegroups.com
To unsubscribe from this group, send email to django-users-***@googlegroups.com
For more options, visit this group at http://groups.google.com/group/django-users?hl=en
-~----------~----~----~----~------~----~------~--~---
Dirk van Oosterbosch, IR labs
2007-07-01 14:14:11 UTC
Permalink
Hi All,

I am struggling with this problem for a while now. I have a form
(newforms) to allow users to upload images and movie files. All the
code runs fine on my test-development machine, but when I deploy it
on a server with FastCGI, it slowly breaks.

This is the code I am using:
http://dpaste.com/hold/13336/

The problem happens specifically with uploading files. FastCGI is
spawning several python processes, which are not automatically
cleaned up, after the upload script finished successful. After a
couple of uploads (3) by a user, this makes the server filled up with
stalled processes completely, so it becomes unavailable and only
returns error 500.

I find processes like these
ps -aux
USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND
dirk 2814 0.0 0.4 59952 4084 ?? SJ Fri05PM 0:11.29 python
mysite.fcgi (python2.5)
dirk 2822 0.0 0.6 59936 5756 ?? SJ Fri05PM 0:15.23 python
mysite.fcgi (python2.5)
dirk 78887 0.0 0.5 74848 5360 ?? SJ Sat03PM 0:06.93 python
mysite.fcgi (python2.5)


Would any of you, by looking at the code have an idea of what could
cause the spawning of these processes or how they could be controlled
from within my code?
Or maybe you know of some missing directive I might have to add to
Apache / FastCGI to control these renegade processes?

Thanks,
dirk

-----------------------------
Dirk van Oosterbosch
de Wittenstraat 225
1052 AT Amsterdam
the Netherlands

http://labs.ixopusada.com
-----------------------------



--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to django-***@googlegroups.com
To unsubscribe from this group, send email to django-users-***@googlegroups.com
For more options, visit this group at http://groups.google.com/group/django-users?hl=en
-~----------~----~----~----~------~----~------~--~---
Dirk van Oosterbosch, IR labs
2007-07-01 19:24:10 UTC
Permalink
Post by Dirk van Oosterbosch, IR labs
http://dpaste.com/hold/13336/
I narrowed down the moment the extra processes are spawned by FastCGI
to be here:

form = UploadForm({'name1': slots[0]['name'], 'name2': slots[1]
['name'], 'name3': slots[2]['name']})
Post by Dirk van Oosterbosch, IR labs
here
if not (('poster' in request.FILES) or ('movie' in request.FILES)):
form = UploadForm(request.POST)

Between line 174 and 175 of the code posted on dbpaste.

The log says:
[Sun Jul 01 20:58:23 2007] [warn] FastCGI: (dynamic) server "/domains/
ouwepikouwepijp.nl/www/mysite.fcgi" (uid 2195, gid 2063) restarted
(pid 59128)

That new process (pid 59128) then seems not be active at all, just
sitting there consuming memory and filling up my shared resources.
My guess is that while the upload data is coming in via the POST,
FastCGI sees the orginal process takes longer than normal to finish
or maybe even looks stalled (because the upload is not completely
received). And because of that FastCGI is restarting, creating a new
thread.
With the ultimate unfortunate consequence of returning nothing but
error 500.

Does anybody has an idea of how to tell FastCGI not to do that?

best
dirk
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to django-***@googlegroups.com
To unsubscribe from this group, send email to django-users-***@googlegroups.com
For more options, visit this group at http://groups.google.com/group/django-users?hl=en
-~----------~----~----~----~------~----~------~--~---
Dirk van Oosterbosch, IR labs
2007-07-04 15:03:28 UTC
Permalink
Post by Dirk van Oosterbosch, IR labs
I narrowed down the moment the extra processes are spawned by
The following happens upon receivement of large POST's:
FastCGI starts up the (Django) site script which passes the request
to the view code. Then the script gets untill between the two excerpt
lines above. But because it is a large POST and is not completely
received over the wire yet, it hangs / waits (whatever you wanna call
it) there. This makes FastCGI think that the whole handling of the
request is stalled, upon which FastCGI spawns a new process every 3
seconds (ultimately crashing the server).

If found this old thread on the FastCGI list, which seems to
accurately explains the problem:
http://www.fastcgi.com/archives/fastcgi-developers/2004-January/
003206.html

However, the proposed solution, to put this on the top of the view code:
tmp_response = HttpResponse()
tmp_response.write("Content-type: text/html")
tmp_response.flush()

does not seem to work.
Is text/html the correct Content-type to use with Django?

I would be very surprised if uploading files using Django and FastCGI
does work for anyone at all, with the present code.
Is nobody having troubles with large POST's?


Two more questions:
2. I'm trying to debug this uploading script using print statements
code. These print statement however show up in Apache's error.log
quite late, much later than the warnings and notifications of
FastCGI. Is there a way to flush these print statements so I can find
them in the error.log sooner?

3. What is the exact behavior when a request containing a POST comes
in? Shouldn't it be buffering the data while the data comes in and be
telling FastCGI it is handling the request? What is exactly happening
between those two lines in my code above?


Best regards,
dirk




--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to django-***@googlegroups.com
To unsubscribe from this group, send email to django-users-***@googlegroups.com
For more options, visit this group at http://groups.google.com/group/django-users?hl=en
-~----------~----~----~----~------~----~------~--~---
Dirk van Oosterbosch, IR labs
2007-07-04 21:49:55 UTC
Permalink
Aha, of course you guys also have troubles with uploading large
POST's. I found the ticket: #2070
Say, which of those patches could I run over version 0.96?
And can I still use the same code as I have here http://dpaste.com/
hold/13578/ ?

dirk
Post by Dirk van Oosterbosch, IR labs
FastCGI starts up the (Django) site script which passes the request
to the view code. Then the script gets untill between the two
excerpt lines above. But because it is a large POST and is not
completely received over the wire yet, it hangs / waits (whatever
you wanna call it) there. This makes FastCGI think that the whole
handling of the request is stalled, upon which FastCGI spawns a new
process every 3 seconds (ultimately crashing the server).
If found this old thread on the FastCGI list, which seems to
http://www.fastcgi.com/archives/fastcgi-developers/2004-January/
003206.html
[...]
I would be very surprised if uploading files using Django and
FastCGI does work for anyone at all, with the present code.
Is nobody having troubles with large POST's?
2. I'm trying to debug this uploading script using print statements
code. These print statement however show up in Apache's error.log
quite late, much later than the warnings and notifications of
FastCGI. Is there a way to flush these print statements so I can
find them in the error.log sooner?
Found that out. Have this line in yoursite.fcgi
sys.stdout = sys.stderr
Post by Dirk van Oosterbosch, IR labs
3. What is the exact behavior when a request containing a POST
comes in? Shouldn't it be buffering the data while the data comes
in and be telling FastCGI it is handling the request? What is
exactly happening between those two lines in my code above?
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to django-***@googlegroups.com
To unsubscribe from this group, send email to django-users-***@googlegroups.com
For more options, visit this group at http://groups.google.com/group/django-users?hl=en
-~----------~----~----~----~------~----~------~--~---

Continue reading on narkive:
Loading...