Templated-docs crashes server with exit codes 139, 77

[dukeimg]

• Templated-docs version: 0.2.9
• Python version: 2.7.12
• Operating System: Ubuntu 16.04

Description

I faced a critical bug which kills Django server with exit codes 139, 77.

I have an API which handles .docx render. I will show one of views, as an example, they are pretty similar:

@permission_classes((permissions.IsAdminUser,))
class BookDocxViewSet(mixins.RetrieveModelMixin, viewsets.GenericViewSet):
    def retrieve(self, request, *args, **kwargs):
        queryset = Pupils.objects.get(id=kwargs['pk'])
        serializer = StudentSerializer(queryset)
        context = dict(serializer.data)
        doc = fill_template('crm/docs/book.ott', context, output_format='docx')
        p = u'docs/books/%s/%s_%s_%s.doc' % (datetime.now().date(), context[u'surname'], context[u'name'], datetime.now().date())
        with open(doc, 'rb') as f:
            content = f.read()
            path = default_storage.save(p, ContentFile(content))
            f.close()
        return response.Response(u'/media/' + path)

When I call it the first time, it creates a .docx file, saves it to my default_storage and then returns me a download link. But when I try to do it again, or do it with another method (which works with another template and context), my server just crashes without any logs. The last thing I see is either

Process finished with exit code 77 if I call it with a little delay (more then one second)
Process finished with exit code 139 (interrupted by signal 11: SIGSEGV) if call my method for the second time right away (in less than one second)
I tried to use debuger -- it said that my server crashes on this line:

doc = fill_template('crm/docs/book.ott', context, output_format='docx')

I bet what happens is:

1. When I call my method the first time templated_docs starts LibreOffice backend, and then does not stop it
2. When I call my method the second time templated_docs tries to start LibreOffice backend again, but it is already busy.

According to this documentation, the process that used pylockt should be killed after. But we cannot kill Django server.

What I Did

I made some changes to __init__.py to add multiprocessing. Pull request is attached.

Paste the command(s) you ran and the output.
If there was a crash, please include the traceback here.

[dudanogueira]

I have the same code base running either on nginx and apache. With @dukeimg commit it works perfectly on nginx. No more crashes (I was also struggling with this bug)

However, on the apache2 codebase, it doesn't work at all. Probably some config issue.

@dukeimg , were you using nginx on deployment too? Thanks for the commit!!

[dukeimg]

@dudanogueira Yes. We do use nginx.

[alexmorozov]

Hey Viktor, thanks for a patch! I've just proposed the pylokit's author to incorporate your change in his/her library, let's see if that works out. Here's the ref: xrmx/pylokit#7

[xrmx]

@dukeimg @dudanogueira which version of libreoffice do you have? Do you have any backtrace or log?

[dudanogueira]

hi @xrmx I am actually using libreoffice 1:5.2.0rc4-0ubuntu1trusty1 and it's working fine with @dukeimg patch with nginx. The strange thing is that it doesn't give much a clue about the problem on the logs. On nginx I got:

2016/08/10 08:09:42 [error] 1623#0: *7 upstream prematurely closed connection while reading response header from upstream, Client. XXXXXXXXX

that's why @dukeimg was bright on his assertion about the origin of the bug. It was like a head shot in the dark :)

[alexmorozov]

@dudanogueira ngnix logs won't tell you much, you'd better check out uwsgi/gunicorn/runner-you-use logs.

[dukeimg]

@dudanogueira unfoutunately, logs won't tell you anything. That's why I spend 2 hours with debugger, looking how those libraries were executed line by line :P

[alexmorozov]

Hope that's fixed in b0a6669. Feel free to reopen if you see it's not.