Skip to content

Conversation

@klauspost
Copy link
Owner

@klauspost klauspost commented Feb 26, 2021

Remove repeat checks in better mode since they are no often than not stealing better matches.

Instead check if matches are repeats when emitting. Similar performance, decompression prefers this.

Before/After:

nyc-taxi:
Compressing... 3325605752 -> 954776589 [28.71%]; 492ms, 6445.5MB/s
Compressing... 3325605752 -> 935022950 [28.12%]; 489ms, 6485.8MB/s
Decompressing. 954776589 -> 3325605752 [348.31%]; 2.434s, 1302.9MB/s
Decompressing. 935022950 -> 3325605752 [355.67%]; 2.297s, 1381.0MB/s

enwik9:
Compressing... 1000000000 -> 426854233 [42.69%]; 213ms, 4477.3MB/s
Compressing... 1000000000 -> 426242923 [42.62%]; 219ms, 4354.7MB/s
Decompressing. 426242923 -> 1000000000 [234.61%]; 996ms, 957.0MB/s
Decompressing. 426854233 -> 1000000000 [234.27%]; 1.009s, 945.2MB/s

Remove repeat checks in better mode since they are no often than not stealing better matches.

Instead check if matches are repeats when emitting.
@klauspost klauspost merged commit 005d22e into master Feb 26, 2021
@klauspost klauspost deleted the better-dont-search-repeat branch February 26, 2021 15:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants