Re: [GMG-Devel] debugging a large file upload, EOFError

[Christopher Allan Webber]

Yes, indeed, there are a number of problems with the way we handle large
files throughout our codebase.  There's still a number of places that
use the equivalent of file().write(file.read()) which is well documented
as "a horrible way to do things" since it reads the whole file into
memory before writing it (!!).  We've fixed many of them, but not all of
them.

There are some tickets to address this.  I really need a break from all
the administrative stuff I've been caught up in the last few weeks.
Maybe I should focus on this issue all next week... it would be a
welcome break, and I'm sure very welcome to have resolved by the next
release.

(If someone is interested in stepping up to help on code review /
implementation also, it's welcome!)

 - Chris

ayleph writes:

> My lighttpd config uses /tmp to hold files being uploaded. Check your
> nginx config to see where it stores files in transfer. Check the
> available disk space at that location.
>
> Another idea I have is that perhaps you're exhausting system memory
> trying to load that file in for processing. But I don't understand the
> processing system well enough to know if that's a potential issue.
>
> On November 21, 2014 11:27:03 AM PST, Alon Levy <address@hidden> wrote:
>>Hi,
>>
>> I see there are some tickets about large file suport (647, 834, 419)
>>but I'm not sure if I'm hitting a limitation of gmg or of my webserver
>>or the fcgi or firefox perhaps, so I thought I'd give details of my
>>setup and ask for help:
>>
>> Submitting a 1.4GB video file from firefox 33.1 to mediagoblin running
>>in the same network (wifi though), firefox shows a message about
>>timeout
>>(sadly I closed the window) and in mediagoblin I get an EOFError
>>exception, details below.
>>
>>I'm using nginx with fastcgi. I think I got a 499: (just looking now at
>>the logs)
>>
>>10.81.2.1 - - [21/Nov/2014:20:17:27 +0200] "POST /submit/ HTTP/1.1" 499
>>0 "http://mediagoblin.tami.org.il/submit/"; "Mozilla/5.0 (X11; Linux
>>x86_64; rv:33.0) Gecko/20100101 Firefox/33.0" "10.81.2.231"
>>
>> nginx.conf has 'client_max_body_size 2560m;'
>>
>>
>> Does anyone have a working setup for two gigabyte uploads? If so could
>>you describe it? Any ideas why I cannot upload large files? I don't
>>know
>>what the cutoff is, I guess that would be a next step. I hope someone
>>has a better suggestion,
>>
>>Alon
>>p.s. complete stack trace from mediagoblin.log:
>>
>>Error - <type 'exceptions.EOFError'>:
>>URL: http://mediagoblin.tami.org.il/submit/
>>File
>>'/usr/lib/python2.7/site-packages/paste/exceptions/errormiddleware.py',
>>line 144 in __call__
>>  app_iter = self.application(environ, sr_checker)
>>File '/home/mediagoblin/mediagoblin/mediagoblin/app.py', line 268 in
>>__call__
>>  return self.call_backend(environ, start_response)
>>File
>>'/home/mediagoblin/mediagoblin/lib/python2.7/site-packages/Werkzeug-0.8.3-py2.7.egg/werkzeug/wsgi.py',
>>line 411 in __call__
>>  return self.app(environ, start_response)
>>File '/home/mediagoblin/mediagoblin/mediagoblin/app.py', line 188 in
>>call_backend
>>  request.locale = translate.get_locale_from_request(request)
>>File '/home/mediagoblin/mediagoblin/mediagoblin/tools/translate.py',
>>line 97 in get_locale_from_request
>>  request_args = (request.args, request.form)[request.method=='POST']
>>File
>>'/home/mediagoblin/mediagoblin/lib/python2.7/site-packages/Werkzeug-0.8.3-py2.7.egg/werkzeug/utils.py',
>>line 77 in __get__
>>  value = self.func(obj)
>>File
>>'/home/mediagoblin/mediagoblin/lib/python2.7/site-packages/Werkzeug-0.8.3-py2.7.egg/werkzeug/wrappers.py',
>>line 383 in form
>>  self._load_form_data()
>>File
>>'/home/mediagoblin/mediagoblin/lib/python2.7/site-packages/Werkzeug-0.8.3-py2.7.egg/werkzeug/wrappers.py',
>>line 318 in _load_form_data
>>  data = parser.parse_from_environ(self.environ)
>>File
>>'/home/mediagoblin/mediagoblin/lib/python2.7/site-packages/Werkzeug-0.8.3-py2.7.egg/werkzeug/formparser.py',
>>line 164 in parse_from_environ
>>  return self.parse(stream, mimetype, content_length, options)
>>File
>>'/home/mediagoblin/mediagoblin/lib/python2.7/site-packages/Werkzeug-0.8.3-py2.7.egg/werkzeug/formparser.py',
>>line 188 in parse
>>  content_length, options)
>>File
>>'/home/mediagoblin/mediagoblin/lib/python2.7/site-packages/Werkzeug-0.8.3-py2.7.egg/werkzeug/formparser.py',
>>line 100 in wrapper
>>  stream.exhaust()
>>File
>>'/home/mediagoblin/mediagoblin/lib/python2.7/site-packages/Werkzeug-0.8.3-py2.7.egg/werkzeug/wsgi.py',
>>line 787 in exhaust
>>  self.read(chunk)
>>File
>>'/home/mediagoblin/mediagoblin/lib/python2.7/site-packages/Werkzeug-0.8.3-py2.7.egg/werkzeug/wsgi.py',
>>line 801 in read
>>  read = self._read(to_read)
>>File
>>'/home/mediagoblin/mediagoblin/lib/python2.7/site-packages/flup-1.0.3.dev_20110405-py2.7.egg/flup/server/fcgi_base.py',
>>line 161 in read
>>File
>>'/home/mediagoblin/mediagoblin/lib/python2.7/site-packages/flup-1.0.3.dev_20110405-py2.7.egg/flup/server/fcgi_base.py',
>>line 147 in _waitForData
>>File
>>'/home/mediagoblin/mediagoblin/lib/python2.7/site-packages/flup-1.0.3.dev_20110405-py2.7.egg/flup/server/fcgi_base.py',
>>line 696 in process_input
>>File
>>'/home/mediagoblin/mediagoblin/lib/python2.7/site-packages/flup-1.0.3.dev_20110405-py2.7.egg/flup/server/fcgi_base.py',
>>line 474 in read
>>EOFError:
>>
>>
>>CGI Variables
>>-------------
>>  CONTENT_LENGTH: '1500501143'
>>  CONTENT_TYPE: 'multipart/form-data;
>>boundary=---------------------------4982243468044615281985221407'
>>  DOCUMENT_ROOT: '/usr/share/nginx/html'
>>  DOCUMENT_URI: '/submit/'
>>  GATEWAY_INTERFACE: 'CGI/1.1'
>>  HTTP_ACCEPT:
>>'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8'
>>  HTTP_ACCEPT_ENCODING: 'gzip, deflate'
>>  HTTP_ACCEPT_LANGUAGE: 'en-US,en;q=0.5'
>>  HTTP_CONTENT_LENGTH: '1500501143'
>>  HTTP_CONTENT_TYPE: 'multipart/form-data;
>>boundary=---------------------------4982243468044615281985221407'
>>  HTTP_COOKIE: 'mediagoblin_csrftoken=2178534525516131666;
>>MGSession=eyJzdGF5X2xvZ2dlZF9pbiI6dHJ1ZSwibWVzc2FnZXMiOltdLCJ1c2VyX2lkIjoiMSJ9.B04tgg.s1z0ifQ23U1HlKy6Z-6D1Kb7Mjk;
>>mediagoblin_csrftoken=2178534525516131666'
>>  HTTP_HOST: 'mediagoblin.tami.org.il'
>>  HTTP_REFERER: 'http://mediagoblin.tami.org.il/submit/'
>>  HTTP_USER_AGENT: 'Mozilla/5.0 (X11; Linux x86_64; rv:33.0)
>>Gecko/20100101 Firefox/33.0'
>>  HTTP_X_FORWARDED_FOR: '10.81.2.231'
>>  PATH_INFO: '/submit/'
>>  REDIRECT_STATUS: '200'
>>  REMOTE_ADDR: '10.81.2.1'
>>  REMOTE_PORT: '54761'
>>  REQUEST_METHOD: 'POST'
>>  REQUEST_URI: '/submit/'
>>  SERVER_ADDR: '10.81.2.247'
>>  SERVER_NAME: 'localhost'
>>  SERVER_PORT: '80'
>>  SERVER_PROTOCOL: 'HTTP/1.1'
>>  SERVER_SOFTWARE: 'nginx/1.4.7'
>>
>>
>>WSGI Variables
>>--------------
>>  application: <mediagoblin.app.MediaGoblinApp object at 0x21a0450>
>>  paste.throw_errors: True
>>  werkzeug.request: <Request 'http://mediagoblin.tami.org.il/submit/'
>>[POST]>
>>  wsgi process: 'Multiprocess'
>>
>>_______________________________________________
>>devel mailing list
>>address@hidden
>>http://lists.mediagoblin.org/listinfo/devel