Serializing user data in Doc objects

[mupavan]

I am using the spacy for doing some research on entity typing. Since, the doc object doesn't support addition of any new first class fields into it, I am using the suggested way of adding any new annotations/predictions into a custom datastructure in the user_data field of the doc object. The problem is that the default doc serializer - doc.to_bytes() doesn't seem to have support for user_data. I wanted to know if there is any suggested default way of doing it and if there are any plans to support user_data into doc serialization in the future.

[honnibal]

This is a known problem, and yes we want to address it. In version 2 the Doc object will support the pickle protocol.

The best solution at the moment would be to represent your docs as a tuple (user_data, byte_string), which you could then pickle. You would then be able to write a deserialisation helper that took the vocab object and the tuple and reconstructed the Doc.

[ines]

Closing this and making #1045 the master issue. Work in progress for spaCy v2.0!

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.