cPickle.PicklingError in multiprocess mode under python2.7.9

[glli]

05/27/17 master version
command: python -m bypy --processes 4 download

MainProcess: Abort
Traceback (most recent call last):
File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
"main", fname, loader, pkg_name)
File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File "/home/glli/bin/bypy/main.py", line 13, in
main()
File "/home/glli/bin/bypy/bypy.py", line 3566, in main
result = getattr(by, args.command[0])(*uargs)
File "/home/glli/bin/bypy/bypy.py", line 2182, in download
return self.downdir(remotepath, localpath)
File "/home/glli/bin/bypy/bypy.py", line 2170, in downdir
return self.__downdir(rpath, lpath)
File "/home/glli/bin/bypy/bypy.py", line 2155, in __downdir
return self.__downdir_multi(rpath, lpath)
File "/home/glli/bin/bypy/bypy.py", line 2151, in __downdir_multi
"Directory Download")
File "/home/glli/bin/bypy/bypy.py", line 554, in __multi_process
results = ar.get(const.TenYearInSeconds)
File "/usr/local/lib/python2.7/dist-packages/multiprocess/pool.py", line 567, in get
raise self._value
cPickle.PicklingError: Can't pickle <type 'instancemethod'>: attribute lookup builtin.instancemethod failed

[houtianze]

Duplicate of #358

[houtianze]

Install the latest version of bypy (1.5.4) (pip install -U bypy), and run the same command again, it should give you the instructions fixing the error.

In short, Python multiprocess library is not properly installed, you need to install a C compiler (e.g. gcc) first, then re-install Python mulitprocess.
On Debian derivaties:
# apt-get install gcc
# pip uninstall -y multiprocess
# pip install -v multiprocess
If above commands don't give you any error/warning, --processes option should work now.

[glli]

Cool, I resolved the issue. Actually I still need to install python-dev for python2.x because it needs Python.h to compile _multiprocess.so. Thanks!

[houtianze]

Thanks for pointing out that python-dev is also needed, updated the prompt message.