_revs_limit not applied after calling _compact

[tudordumitriu]

After calling _revs_limit with 20 as value and after that calling _compact to a db _revs_diff seem to be still returning a 400+ list of revs.

Context

Everything started from checking the network traffic and we noticed _revs_diff are sending a ever-growing list revs.
That happens naturally because we are updating often some entities - eg a list of items being modified by different users (same entity is replicated in all user dbs).
Also because this document is shared by multiple user dbs conflicts happen and we are solving them and deleting loosing conflicts.
But this seems to be generating a growing list of revs sent back and forth by the replication protocol.

After looking at PouchDB quite a bit, we started looking at CouchDB and tried limiting the revs to 20 and after that calling _compact.
But nothing actually happens.

This is a very concerning issues for us because these lists will be heavily updated (accumulating lots of revs), but we care only about latest rev + conflicts so don't need the entire revs history, and mostrly we don't need it carried back and forth for every _revs_diff, _bulk_docs, and _changes call (especially since our app will be used on mobile devices).

We have posted and issues on PouchDB, but don't think is related to it. (apache/pouchdb#7491)

Your Environment

• Version used: 2.2.0
• Browser Name and version: Chrome, Mobile Hybrid App Crosswalk / WKWebview
• Operating System and version (desktop or mobile): Windows 10
• PouchDB 7.0.0

[wohali]

There is no 2.2.2. You must mean 2.2.0..

Have you checked for conflicts in your documents? CouchDB will never reduce the number of revs below the max # of conflicts on any given document.

[tudordumitriu]

Hi @wohali and thanks for the quick answer.
Sry, have corrected to 2.2.0.

I can strongly confirm there are NO conflicts at all, our auxiliary services are keeping track of them and are solving them constantly.
One of these docs that has for example 1000+ revisions, at at time had even 10 conflicts, but data was merged and loosing conflicts deleted, and now there are NO conflicts at all in DB.

So can you confirm that _revs_limit should impact the _revs_diff response as well?
Also, can you please confirm we did the right things?

1. _revs_limit to 50 (is that an advisable value?) on db
2. _compact db

Thank you

[wohali]

Sorry, someone else will have to help you.

[kocolosk]

Hi @tudordumitriu, I think I understand what is happening here.

When you resolve a conflict in a CouchDB document the conflicted path continues to exist in the revision tree. CouchDB simply adds a deleted:true leaf on the end of the path. If you request one of these documents with ?meta=true I suspect you'll see a large block of deleted_conflicts entries.

The _revs_limit setting will cap the length of each individual path in the revision tree, and will then merge those shortened paths together into one or more trees. So, the number of entries in the _revs_info can be as high as the product of _revs_limit times the number of total paths -- deleted or otherwise.

I would like to improve the behavior of the database so that it can intelligently "prune" the branches of the revision tree that track resolved conflicts once it's confident that all replication peers have observed the latest updates to the branch. It's a subtle change though with some important side effects to consider. We've also contemplated introducing user-defined conflict resolution strategies which could be more efficient than the current (maximally robust) strategy.

CouchDB 1.x supported the notion of "purging" revisions entirely, which would certainly accomplish your goal of getting the revision metadata slimmed down. CouchDB 2.0 lost this capability because of some unique challenges supporting it in a cluster. We did recently re-introduce the capability in #1370 ... just after we released 2.2.0. It'll be in 2.3.0 once that's released.

Net: there's not a lot you can do right now to fix up documents that had a lot of conflicts in the past. Shortening the _revs_limit does help -- but not as much as you think, and if it gets shortened too much you run the risk of generating additional spurious conflicts.

[tudordumitriu]

Thank you @kocolosk
Then we are looking forward for 2.3.0!