how to use nlp.pipe

[xcwill]

Thanks for this useful tool, but I want to use multi-processors to accelerate the NER and ETL. I found spacy provide an NLP.pipe method, it's not compatible with your pipeline. My question is how to use NLP.pipe in your pipeline.

[MartinoMensio]

Hi @xcwill ,
I will need to look into the implementation of nlp.pipe. At the moment I know it exists but I'm not familiar with implementing it.
I will keep you updated as soon as I have more info.

Best,
Martino

[MartinoMensio]

Hi @xcwill ,
I am working on a multiprocessing enhancement, that should support the nlp.pipe method.

Could you try to install the multiprocessing branch and let me know if this would work in your case?

pip install git+https://github.com/egerber/spaCy-entity-linker@multiprocessing

The only limitation is that you cannot use anymore the .get_span() or read the .span property of the EntityElement class. I'm trying to find a workaround for it.

This issue is linked strictly with #23.

Best,
Martino