Initial memory map of the database is contiguous

[finnschiermer]

To retain backward compatibility with databases created before the chunked memory mapping was introduced (0.93) we always map any existing database as one contiguous address range - initially. Now that all other uses of large contiguous mappings have been eliminated, this will become a visible limit. When can we break backward compatibility? Or should we build a file format upgrade where we handle the problem? Or something different?

[finnschiermer]

I think we should choose the "something different" option:

Instead of breaking backward compatibility or upgrade the file, we can extend the memory mapping scheme to handle Arrays that cross a section boundary. This allows us to eliminate the special handling of the initial mapping of the file. It is not known at the time a mapping is established if it holds an array crossing its boundaries. This can only be detected when such an Array is accessed. We'll likely need to build a dedicated mapping for each such array.

[AndyDentFree]

Will _ handle Arrays that cross a section boundary_ approach also lend itself to huge blob sectioning?

Is it likely to provide a fix for our Windows issues?

[finnschiermer]

Will _ handle Arrays that cross a section boundary_ approach also lend itself to huge blob sectioning?

I haven't fully analysed it, but yes I think that should be the ambition.

Is it likely to provide a fix for our Windows issues?

No, it is unrelated.