Skip to content

Conversation

@JackieTien97
Copy link
Contributor

  1. Correct the retained size calculation for BinaryColumn and BinaryColumnBuilder
  2. remove some useless todo
  3. throw new UnsupportedOperationException for isNull method in TimeColumn

Do the performance test, build 1 billion binary using TsBlockBuilder, we only see 1-second performance impact(macOS M2 pro, OpenJDK-21).
Before:
6779
6420
6394
6387
6527

After:
7589
7524
8063
7592
7696

You can replay that test, using the following codes

public static void main(String[] args) {
  TsBlockBuilder builder = new TsBlockBuilder(Collections.singletonList(TSDataType.BLOB));
  TimeColumnBuilder timeColumnBuilder = builder.getTimeColumnBuilder();
  ColumnBuilder columnBuilder = builder.getColumnBuilder(0);
  long startTime = System.nanoTime();
  for (int i = 0; i < 1_000_000_000; i++) {
    timeColumnBuilder.writeLong(i);
    Binary binary = new Binary(BytesUtils.intToBytes(i));
    columnBuilder.writeBinary(binary);
    builder.declarePosition();
    if (builder.isFull()) {
      builder.build();
      builder.reset();
      timeColumnBuilder = builder.getTimeColumnBuilder();
      columnBuilder = builder.getColumnBuilder(0);
    }
  }
  if (!builder.isEmpty()) {
    builder.build();
    builder.reset();
  }
  System.out.println("cost: " + (System.nanoTime() - startTime) / 1_000_000);
}

@codecov-commenter
Copy link

codecov-commenter commented Jan 21, 2025

Codecov Report

Attention: Patch coverage is 51.11111% with 22 lines in your changes missing coverage. Please review.

Project coverage is 64.82%. Comparing base (02727c5) to head (5bc78e5).
Report is 3 commits behind head on develop.

Files with missing lines Patch % Lines
.../tsfile/read/common/block/column/BinaryColumn.java 19.04% 17 Missing ⚠️
...ile/read/common/block/column/DictionaryColumn.java 0.00% 1 Missing ⚠️
...he/tsfile/read/common/block/column/NullColumn.java 0.00% 1 Missing ⚠️
...ad/common/block/column/RunLengthEncodedColumn.java 0.00% 1 Missing ⚠️
...he/tsfile/read/common/block/column/TimeColumn.java 50.00% 1 Missing ⚠️
.../tsfile/read/common/block/column/TsBlockSerde.java 66.66% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop     #384      +/-   ##
===========================================
+ Coverage    64.66%   64.82%   +0.16%     
===========================================
  Files          541      543       +2     
  Lines        32371    33148     +777     
  Branches      4318     4416      +98     
===========================================
+ Hits         20932    21489     +557     
- Misses       10942    11117     +175     
- Partials       497      542      +45     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@JackieTien97 JackieTien97 merged commit 9f8ab12 into develop Feb 7, 2025
11 checks passed
@JackieTien97 JackieTien97 deleted the TuneBinaryColumnMemory branch February 7, 2025 01:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants