Use case:
Currently we are implementing various caching strategies, for example caching in findGenerations, to avoid calling ParseFileName too often. It would be good instead to cache ParseFileName which may improve many areas of code where we have to parse a lot of TSM files on the fly. For example during our compaction process given the following CPU profile, you can see a lot of the work is parsing Uints from strings:
Proposal:
We already have our Filestore and Engine structs enabled for generic parse file name functions. Here is where we pass in the default one while spinning up a new compactor object:
|
return &Compactor{ |
|
formatFileName: DefaultFormatFileName, |
|
parseFileName: DefaultParseFileName, |
|
} |
. We would just need to implement a new parser function, something like:
ParseFileNameCachedFunc and extend the already existing code to use it.
This will likely be non-trivial and would require a lot of testing since it's a core piece of functionality.
Use case:
Currently we are implementing various caching strategies, for example caching in
findGenerations, to avoid callingParseFileNametoo often. It would be good instead to cacheParseFileNamewhich may improve many areas of code where we have to parse a lot of TSM files on the fly. For example during our compaction process given the following CPU profile, you can see a lot of the work is parsing Uints from strings:Proposal:
We already have our Filestore and Engine structs enabled for generic parse file name functions. Here is where we pass in the default one while spinning up a new compactor object:
influxdb/tsdb/engine/tsm1/compact.go
Lines 824 to 827 in 22bec4f
ParseFileNameCachedFuncand extend the already existing code to use it.This will likely be non-trivial and would require a lot of testing since it's a core piece of functionality.