cut: fix -s flag for newline delimiter and optimize memory allocation#11143
cut: fix -s flag for newline delimiter and optimize memory allocation#11143cakebaker merged 1 commit intouutils:mainfrom
Conversation
|
GNU testsuite comparison: |
f82da8d to
cb8cddd
Compare
Merging this PR will improve performance by 45.19%
Performance Changes
Comparing Footnotes
|
|
GNU testsuite comparison: |
10358f4 to
34eee51
Compare
|
GNU testsuite comparison: |
34eee51 to
1a2b3d4
Compare
|
GNU testsuite comparison: |
95bad10 to
76b271e
Compare
|
GNU testsuite comparison: |
76b271e to
a5a3666
Compare
|
GNU testsuite comparison: |
4821938 to
69c5cfe
Compare
|
GNU testsuite comparison: |
0f4f795 to
fc991a1
Compare
|
GNU testsuite comparison: |
|
Hi @cakebaker, the tests passed, but the benchmark failed due to an infrastructure issue. Could you please re-run that job? Thanks! |
|
@sylvestre I noticed Attempt №3 was cancelled. Since I don't have permissions to trigger the CI/CD jobs myself, could you let me know if there’s a specific fix I need to make, or if you could re-run the checks when the environment is ready? Thanks! |
2c55e4a to
261a1fd
Compare
|
GNU testsuite comparison: |
d9727d9 to
f70f671
Compare
|
GNU testsuite comparison: |
|
@cakebaker should be ready for a review/merge |
1719399 to
f7a0636
Compare
|
GNU testsuite comparison: |
a21f3c1 to
9213a87
Compare
|
GNU testsuite comparison: |
deea6e9 to
2cabf8e
Compare
|
GNU testsuite comparison: |
- Fixed the -s flag incorrectly suppressing output when the delimiter is a newline. - Improved performance in cut_fields_newline_char_delim. - Updated tests to match GNU cut behavior for newline delimiters.
2cabf8e to
fe4e36b
Compare
|
GNU testsuite comparison: |
|
Thanks for your PR! |
Fixes & Improvements
-slogic fix: Add the missingonly_delimitedcheck to properly suppress non-delimited lines.split().collect()with amemchr-powered loop. This shifts memory complexity from O(Total File Size) to O(Max Field Size) - as "OOM-safe" as the specification allows.BufReader::consume()to avoid heap copies.segments.get()lookups with a single-passrange_idxpointer that synchronizes "Skip" and "Keep" paths in one linear sweep.Benchmarks
10,000,000 records (
seq 1 10000000 > bench_input.txt), base M1 Pro chip.Case 1: Filtered Selection with Early Exit (
-s -d $'\n' -f 2,1024,4096)gcut./cut_old./cut_newResult: ~224x faster than
cut_old, ~170x faster than GNUcut.Case 2: Full File Read / Base Throughput (
-s -d $'\n' -f 1-10000000)gcut./cut_old./cut_newResult: ~3x faster than
cut_old, ~4x faster than GNUcut.References