Problem while parsing bcif files written by py-mmcif

[josemduarte]

This week, the RCSB PDB switched BCIF encoding from Mol* package to py-mmcif. Biotite fails at parsing these BCIF files when accessing any empty category, e.g. it often happens for label_alt_id.

This issue was also reported here: rcsb/py-mmcif#44 .

The recipe to reproduce it is get file from http://models.rcsb.org/2uzi.bcif and then:

from biotite.structure.io.pdbx import BinaryCIFFile, get_structure

x = BinaryCIFFile.read('2uzi.bcif')
get_structure(x, model=1, use_author_fields=False)

The issue seems to be in decoding a StringArrayEncoding. The comment in that issue analyses the problem in some more depth.

I am not sure if py-mmcif is fully compliant with the BCIF spec, but the fact is that other decoders have no issue parsing these files: py-mmcif, Mol*, ciftools-java. So somehow they are lenient to this condition. So perhaps biotite should also be lenient.

Looking a little bit into the code, this is what ciftools-java (which follows quite closely the original Mol* implementation) does to process the offsets array:

https://github.com/rcsb/ciftools-java/blob/5d5fd56dcf4675ca7695e5977b7f9f1501dcbb4e/src/main/java/org/rcsb/cif/binary/encoding/StringArrayEncoding.java#L78-L82

i.e. in the case of an offset of length 1 it still creates an empty string for the first position.

Could something similar be done in biotite?

[padix-key]

Thanks for reporting 👍 ! #833 will resolve this issue, by properly interpreting the -1 indices in the StringArrayEncoding as empty strings.

[josemduarte]

Wonderful, thank you! I can confirm that both Mol*-encoded and py-mmcif -encoded files are read correctly now.

Would you have an ETA for next release?

[padix-key]

I plan to release it this week.

[BradyAJohnston]

Any chance we could get a new release sooner rather than later? This has nuked a whole bunch of functionality and tests in Molecular Nodes until the new version.

[padix-key]

End of last week we had a problem with some flaky requests in the tests for biotite.database.afdb, but this seems to be solved now. I just triggered the new release: https://github.com/biotite-dev/biotite/releases/tag/v1.5.0

[BradyAJohnston]

Thanks @padix-key - appreciate it!

[BradyAJohnston]

Thanks for the hot fix @padix-key but in the final workflow run no assets were uploaded to pypi because it wasn't on the final tagged release.

[padix-key]

Yes, with the hotfix merged, I'll retrigger the release

[BradyAJohnston]

I hate to be the bearer of bad news but the docs failed to build again @padix-key

[padix-key]

That happens sometimes due to some potential database errors during running gallery examples. That's why the GitHub action caches the result from the previous run to speed up a rerun of the job 😄 .

Biotite 1.5 is on PyPI now

[BradyAJohnston]

Thanks so much @padix-key ! Sorry to be annoying about it but the PDB secretly changing their encoding has been a thorn in my side for months now.

I first started noticing it with intermittent test failures weeks and weeks ago. It was never consistent so I assume requested structures were just being delivered from different servers so I was inconsistently getting the newer version / older version. I could never reproduce reliably so I couldn't troubleshoot it and raise issues.

Very annoyed with them around this and their lack of detail around the .bcif spec. Appreciate the hard work on your end though!