[v8r0] Implement a recursive listDirectory for the DFC

[chaen]

closes #3911

I'd have hoped for a simple flag in the listDirectory method, but it unfortunately can't be done (too much modification of too many methods that are used by too many components).

So I created a new getDirectoryDump method. For a given directory, it returns all the files and sub folders contained at any depth.

For now I've applied it only to the dirac-dms-user-lfn commands. In the case of the LHCb managers, the speedup is substantial.

For example, on the test instance, I can dump 250k files and 100k sub directories in less than 10 seconds. The original command never returned....

time dirac-dms-user-lfns -b /lhcb/user/c/cburr/
Will search for files in /lhcb/user/c/cburr
/lhcb/user/c/cburr: 251681 files, 98570 sub-directories
251681 matched files have been put in lhcb-user-c-cburr.lfns

real    0m9.893s
user    0m4.280s
sys     0m0.248s

BEGINRELEASENOTES

*DMS
NEW: Introduce getDirectoryDump method in the DFC
CHANGE: dirac-dms-user-lfns uses getDirectoryDump method
CHANGE: DataManager getReplicasFromDirectory and __cleanDirectory use getDirectoryDump

ENDRELEASENOTES

[chrisburr]

I'm surprised that this has recursive=True?

[chaen]

It's maybe not needed indeed, but I'd rather not change it here in the risk of breaking something, and I'd like this to be merged before Monday's release 👼

[DIRACGridBot]

Sweep summary

Sweep ran in https://github.com/DIRACGrid/DIRAC/actions/runs/3961922796

Successful:

• integration