Skip to content

Reduce heap size needed by PrintUtility#3627

Open
lbschanno wants to merge 3 commits into
integrationfrom
task/printUtility
Open

Reduce heap size needed by PrintUtility#3627
lbschanno wants to merge 3 commits into
integrationfrom
task/printUtility

Conversation

@lbschanno

Copy link
Copy Markdown
Collaborator

The PrintUtility class forces us to allocate 2g heaps for unit tests due to attempting to decoded non-HyperLogLogPlus cardinality values as such. The HyperLogLogPlus.legacyDecode() method will at times allocate a very large byte array larger than 1g.

Add the class PrintUtility.TableWriter to handle writing entries of a table to an output. This class can be configured with custom key, timestamp, and Value formatters.

Add constant instances of TableWriter to PrintUtility that handle writing entries for specific tables, such as shard, metadata, facet, etc. This allows for targeted formatting to avoid accidentally decoding a non-HyperLogLogPlus cardinality value as such.

Update all usages of PrintUtility to call the new methods, and reduce the allocated heap size for unit tests to 1g.

Fixes #3248

The PrintUtility class forces us to allocate 2g heaps for unit tests due
to attempting to decoded non-HyperLogLogPlus cardinality values as such.
The HyperLogLogPlus.legacyDecode() method will at times allocate a very
large byte array larger than 1g.

Add the class PrintUtility.TableWriter to handle writing entries of a
table to an output. This class can be configured with custom key,
timestamp, and Value formatters.

Add constant instances of TableWriter to PrintUtility that handle
writing entries for specific tables, such as shard, metadata, facet,
etc. This allows for targeted formatting to avoid accidentally decoding
a non-HyperLogLogPlus cardinality value as such.

Update all usages of PrintUtility to call the new methods, and reduce
the allocated heap size for unit tests to 1g.

Fixes #3248
apmoriarty
apmoriarty previously approved these changes Jun 23, 2026
Comment thread warehouse/query-core/src/test/java/datawave/helpers/Output.java Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

PrintUtility forces us to allocate 2g heaps for unit tests

3 participants