Big File upload

[glyg]

Hi and thanks for your code.

The widget hangs when I try to upload a big file (126 Mb) - even with a local notebook server. How can this be circumvented?

[peteut]

Hmm, there should be no size limitation from FileReader. Sorry, I have no explanation for that behaviour yet.
Could you please provide a pip freeze | grep ipywidgets?
For now you could try/use the builtin upload feature of notebook.

[ialek36]

In my case files larger than ~10Mb do not get uploaded. I used the code sample for README.md, with small modifications. The _cb method doesn't get called at all. It works for smaller files.

(tensorflow) bash-3.2$ pip freeze | grep ipywidgets
ipywidgets==7.3.1

import io
from IPython.display import display
import fileupload

def _upload():

    _upload_widget = fileupload.FileUploadWidget()

    def _cb(change):
        print("hello")
        filename = change['owner'].filename
        print('Uploaded `{}` ({:.2f} kB)'.format(
            filename, len(change['owner'].data) / 2 **10))

    _upload_widget.observe(_cb, names='data')
    display(_upload_widget)

_upload()

[harmvanderheijden]

I'm seeing exactly the same issue as ialek36. Running on Python3, Windows 10, ipywidgets 7.0.0, using the README.md code sample (modified to remove the ioStringIO line, seems not compatible with Python 3). Uploading works fine for smaller files (~100 kB, 4 MB), but hangs (without ever calling _cb) for files > ~10 MB. Please let me know if I can help debug.

[ialek36]

I was actually able to track this down to tornado package. It has websocket.py module which in turn has a setting _default_max_message_size = 10 * 1024 * 1024. That controlled the file size upload for me.

[michcio1234]

Thanks @ialek36 for this trace.
So to actually modify this setting one has to either:

• run jupyter with following option:
  jupyter notebook --NotebookApp.tornado_settings='{"websocket_max_message_size": 62914560}'
  (the magic number above is 60 * 1024 * 1024, i.e. 60 MiB)

• or to create jupyter config file as described here and uncomment and set c.NotebookApp.tornado_settings option.