NFS-safety via flufl.lock?

[jwodder]

Diskcache is almost perfect for our uses, aside from the fact that it uses SQLite and is thus not NFS-safe. It there a way to add support for locking with flufl.lock so that diskcache can be safely used on NFS systems?

[grantjenks]

Afraid not. The use of SQLite is central to all the metadata stored. It’s not just used for locking.

[jwodder]

@grantjenks I didn't mean getting rid of SQLite. I was envisioning something like wrapping all accesses to SQLite in a flufl.lock in order to (hopefully) make the library NFS-safe.

[grantjenks]

Oh, that’s an interesting idea. Maybe. There are kind of two different writes in Disk Cache: writes during initialization and writes in a transaction.

During initialization, Disk Cache uses a retry strategy to set pragmas, create tables/triggers, and store settings. Many of these operations don’t work inside transactions. Take a look at the core.Cache._sql_retry method and its references for the details.

All other activities, like setting or deleting items, use a transaction. The transaction logic is implemented as a context manager in core.Cache._transact. There’s some special scenarios to support forking and writing files but it’s otherwise a pair of BEGIN IMMEDIATE and COMMIT statements to SQLite.

I don’t think I want to add flufl.lock by default since it’ll add extra overhead but maybe you can experiment with support by simply overriding a couple of methods.