diff --git a/PLAN.md b/PLAN.md deleted file mode 100644 index c4291ee41e..0000000000 --- a/PLAN.md +++ /dev/null @@ -1,20 +0,0 @@ -# jts-agentic-workflow - -## Source - -- **Project**: jts-agentic-workflow -- **Branch**: `docs-v2-jts-agentic-workflow` -- **Created**: 2026-02-10 - -## Objective - - - -## Tasks - -- [ ] Task 1 -- [ ] Task 2 - -## Notes - - diff --git a/content/influxdb3/enterprise/performance-preview/_index.md b/content/influxdb3/enterprise/performance-preview/_index.md new file mode 100644 index 0000000000..fc788582e4 --- /dev/null +++ b/content/influxdb3/enterprise/performance-preview/_index.md @@ -0,0 +1,331 @@ +--- +title: Performance upgrade preview +seotitle: Performance upgrade preview for InfluxDB 3 Enterprise +description: > + Preview performance upgrades in InfluxDB 3 Enterprise with improved + single-series query performance, consistent resource usage, wide-and-sparse + table support, column families, and bulk data export. +menu: + influxdb3_enterprise: + name: Performance upgrade preview +weight: 12 +influxdb3/enterprise/tags: [storage, performance, beta, preview] +related: + - /influxdb3/enterprise/get-started/setup/ + - /influxdb3/enterprise/performance-preview/configure/ + - /influxdb3/enterprise/performance-preview/monitor/ + - /influxdb3/enterprise/admin/performance-tuning/ +--- + +> [!Warning] +> #### Performance preview beta +> The performance upgrade preview is available to {{% product-name %}} Trial +> and Commercial users as a beta. These features are subject to breaking changes +> and **should not be used for production workloads**. +> +> To share feedback on this preview, see [Support and feedback options](#bug-reports-and-feedback). +> Your feedback on stability +> and performance at scale helps shape the future of InfluxDB 3. + +## What is the performance upgrade preview? + +{{% product-name %}} includes a preview of major upgrades to the +storage layer that improve how data is written, stored, compressed, compacted, +and queried. +These upgrades touch every layer of the storage path—from a new on-disk file +format to how fields are organized into column families and how compaction +manages resources. + +## Why these upgrades + +The existing InfluxDB 3 storage layer uses [Apache Parquet](https://parquet.apache.org/) +and is optimized for analytical workloads. +Customers running high-cardinality, wide-schema, and query-intensive workloads +need better single-series query performance, more predictable resource usage, +and the schema flexibility that made InfluxDB v1 and v2 popular. +These upgrades extend the storage layer to support those workloads while +maintaining full compatibility with InfluxDB 3's data model and query languages. + +Key improvements include: + +- **Faster single-series queries**: Single-digit millisecond response times + for highly selective time-series queries. +- **Consistent resource usage**: Bounded CPU and memory during persistence + and compaction, eliminating spikes during heavy ingestion or compaction bursts. +- **Wide-and-sparse table support**: Schemas with up to millions of columns + and dynamic schema evolution without expensive rewrites. +- **Column families**: Group related fields for efficient compression and I/O, + so queries only read the data they need. +- **Bulk data export**: Export compacted data as Parquet files for use with + external tools. +- **Automatic Parquet upgrade**: Seamlessly migrate existing data with + hybrid query mode during the transition. + +## Enable the preview + +Include the `--use-pacha-tree` flag in your +[`influxdb3 serve` startup command](/influxdb3/enterprise/get-started/setup/): + +{{< code-callout "--use-pacha-tree" >}} +```bash +influxdb3 serve \ + --node-id host01 \ + --cluster-id cluster01 \ + --object-store file \ + --data-dir ~/.influxdb3 \ + --use-pacha-tree +``` +{{< /code-callout >}} + +You can also enable the preview with an environment variable: + +```bash +export INFLUXDB3_ENTERPRISE_USE_PACHA_TREE=true +influxdb3 serve ... +``` + +The `--use-pacha-tree` flag exposes additional configuration options prefixed +with `--pt-`. +See [Configure the preview](/influxdb3/enterprise/performance-preview/configure/) +for tuning options, or +[Monitor the preview](/influxdb3/enterprise/performance-preview/monitor/) +for system tables and telemetry. + +> [!Warning] +> #### Existing clusters with Parquet data +> +> On clusters with existing Parquet data, enabling `--use-pacha-tree` +> **automatically converts Parquet files to `.pt` format** on startup, which +> consumes additional CPU and memory while the migration runs. +> Queries continue to work normally during this period. +> See [Upgrade from Parquet](#upgrade-from-parquet) for details. +> +> For the beta, we recommend enabling the preview with a fresh cluster in a +> staging or test environment first. + +## What's changed + +These upgrades touch every layer of the storage path—from the on-disk file +format to how data is compressed, organized, and compacted. + +### New file format + +Data is stored in a new columnar file format (`.pt` files) optimized for +time-series workloads. +All data within a file is sorted by column family key, +[series key](/influxdb3/enterprise/reference/glossary/#series-key), and +timestamp, which enables efficient compaction, querying, and filtering. + +The format uses type-specific compression algorithms that adapt to data +characteristics—delta-delta RLE for timestamps, Gorilla encoding for floats, +dictionary encoding for low-cardinality strings, and more—typically +achieving 5-20x compression ratios. + +### Column families + +Column families let you group related fields together so that queries only read +the data they need. +Fields in the same family are stored together on disk. +For wide tables with hundreds of fields, this dramatically reduces I/O. + +When writing [line protocol](/influxdb3/enterprise/reference/line-protocol/), use the `::` (double-colon) delimiter in field +names to assign fields to a family. +The portion before `::` is the family name; everything after is the field name. + +```txt +metrics,host=sA cpu::usage_user=55.2,cpu::usage_sys=12.1,cpu::usage_idle=32.7 1000000000 +metrics,host=sA mem::free=2048i,mem::used=6144i,mem::cached=1024i 1000000000 +metrics,host=sA disk::read_bytes=50000i,disk::write_bytes=32000i 1000000000 +``` + +This creates three column families: + +| Family | Fields | +|:-------|:-------| +| `cpu` | `usage_user`, `usage_sys`, `usage_idle` | +| `mem` | `free`, `used`, `cached` | +| `disk` | `read_bytes`, `write_bytes` | + +When a query references only `mem::free`, the storage layer reads only the +`mem` family block and skips `cpu` and `disk` data entirely. + +> [!Note] +> Only the first `::` is significant. +> A field name like `a::b::c` creates family `a` with field `b::c`. + +Fields written without `::` are assigned to auto-generated families (named +`__0`, `__1`, etc.), each holding up to 100 fields. +Explicit family names are an excellent way to optimize performance with known +workloads, but they're not required to achieve good results. + +### Bounded compaction + +Incoming writes are buffered in the WAL, flushed to snapshots, and then merged +into [Gen0 files](/influxdb3/enterprise/performance-preview/configure/#gen0). +The upgraded storage layer organizes compacted data into 24-hour UTC windows +and progresses Gen0 files through four [compaction levels (L1 through L4)](/influxdb3/enterprise/performance-preview/configure/#l1-l4-level-tuning). +Compaction runs continuously in the background with a byte-based memory budget +(default: 50% of system RAM), so it never causes resource spikes. + +Old files are cleaned up after a cooldown period, ensuring query replicas have +time to see new checkpoints before old data is removed. +Failures are automatically retried, and the system is designed to be +self-healing for transient issues. + +## Upgrade from Parquet + +Existing clusters with Parquet data can upgrade with zero manual migration. +The upgrade is fully automatic and occurs on initial startup. + +When you restart a cluster with `--use-pacha-tree`, the system: + +1. Detects existing Parquet data and enters hybrid mode. +2. Clears the legacy WAL on ingest nodes and streams Parquet files through a + conversion pipeline. +3. Integrates converted files into the new storage format through compaction. +4. Automatically transitions once all data is migrated. + +During hybrid mode, queries merge results from both the legacy and upgraded +storage layers. +If there is a conflict (same series key and timestamp), the upgraded data takes +precedence. + +### Monitor upgrade progress + +Use system tables to track upgrade status: + +```sql +-- Per-node upgrade status +SELECT * FROM system.upgrade_parquet_node + +-- Per-file migration progress +SELECT * FROM system.upgrade_parquet +``` + +### Configure upgrade behavior + +| Option | Description | Default | +|:-------|:------------|:--------| +| `--pt-disable-hybrid-query` | Disable hybrid query mode. Queries return only data from the upgraded storage layer, even during migration. | `false` | +| `--pt-upgrade-poll-interval` | Polling interval for upgrade status monitoring. | `5s` | + +## Downgrade to Parquet + +If you need to revert from the performance preview back to standard Parquet +storage, use the `influxdb3 downgrade-to-parquet` command. +This command updates the catalog and deletes all PachaTree-specific files from +object storage. + +> [!Note] +> #### Downgrade impacts +> +> The downgrade deletes all `.pt` files, including data written +> after the upgrade. +> **Only data that existed before the upgrade (original Parquet files) is preserved.** +> You can re-enable the preview later by restarting with `--use-pacha-tree`. + +### Before you downgrade + +1. **Stop all nodes** in the cluster before running the downgrade command. + The command checks for running nodes and refuses to proceed if any are active. + + ```bash + influxdb3 stop node --node-id + ``` + +2. **Verify table compatibility.** + The downgrade validates that all tables can be represented in Parquet format. + Tables that exceed the Parquet column limit or contain columns without legacy + Parquet column IDs block the downgrade. + +### Preview the downgrade + +Use the `--dry-run` flag to list files that would be deleted without making +any changes: + +```bash +influxdb3 downgrade-to-parquet \ + --cluster-id cluster01 \ + --object-store file \ + --data-dir ~/.influxdb3 \ + --dry-run +``` + +### Run the downgrade + +```bash +influxdb3 downgrade-to-parquet \ + --cluster-id cluster01 \ + --object-store file \ + --data-dir ~/.influxdb3 +``` + +After the downgrade completes, restart nodes without the `--use-pacha-tree` flag +to resume standard Parquet storage mode. + +For all available options, see +[Downgrade options](/influxdb3/enterprise/performance-preview/configure/#downgrade-options). + +## Export to Parquet + +You can export compacted data as Parquet files for use with external tools. + +> [!Note] +> Data must be compacted before it can be exported. +> Uncompacted data is not available for export at this time. + +### Export workflow + +```bash +# Step 1: List available databases +influxdb3 export databases + +# Step 2: List tables in a database +influxdb3 export tables -d mydb + +# Step 3: List compacted 24-hour windows for a table +influxdb3 export windows -d mydb -t cpu + +# Step 4: Export data as Parquet files +influxdb3 export data -d mydb -t cpu -o ./export_output +``` + +To export specific time windows only: + +```bash +influxdb3 export data -d mydb -t cpu -w 2026-01-15,2026-01-16 -o ./export_output +``` + +## Who should try the preview + +Consider enabling the preview in your staging or development environment if +you have workloads with: + +- High cardinality or wide tables +- Frequent backfill across time ranges +- Query-heavy access patterns requiring low latency +- Sparse schemas with dynamic column creation +- Resource constraints where bounded memory and CPU usage matter + +> [!Important] +> #### Important: New file format +> +> These upgrades use a new columnar file format (`.pt` files). +> When you enable the preview, new data is written in the new format. +> Hybrid query mode (enabled by default) allows querying across both legacy +> Parquet data and new `.pt` data seamlessly. +> +> For the beta, we recommend starting with a fresh setup for +> testing and evaluation rather than converting existing data. + +## Bug reports and feedback + +To share feedback on the performance upgrade preview: + +- Contact [InfluxData support](https://support.influxdata.com) +- Reach out to your InfluxData account team + +Your feedback on stability and performance at scale helps shape the future of +InfluxDB 3. + +{{< children hlevel="h2" readmore=true hr=true >}} diff --git a/content/influxdb3/enterprise/performance-preview/configure.md b/content/influxdb3/enterprise/performance-preview/configure.md new file mode 100644 index 0000000000..1b6a599c0c --- /dev/null +++ b/content/influxdb3/enterprise/performance-preview/configure.md @@ -0,0 +1,467 @@ +--- +title: Configure the performance upgrade preview +seotitle: Performance upgrade preview configuration reference for InfluxDB 3 Enterprise +description: > + Complete reference for all configuration options available with the InfluxDB 3 Enterprise + performance upgrades, including WAL, snapshot, compaction, caching, and replication settings. +menu: + influxdb3_enterprise: + name: Configuration reference + parent: Performance upgrade preview +weight: 202 +influxdb3/enterprise/tags: [storage, configuration, beta, preview, reference] +related: + - /influxdb3/enterprise/performance-preview/ + - /influxdb3/enterprise/performance-preview/monitor/ + - /influxdb3/enterprise/admin/performance-tuning/ + - /influxdb3/enterprise/reference/config-options/ +--- + +> [!Warning] +> #### Performance preview beta +> The performance upgrade preview is available to {{% product-name %}} Trial +> and Commercial users as a beta. These features are subject to breaking changes +> and **should not be used for production workloads**. + +This page provides a complete reference for all configuration options available +with the performance upgrade preview. +All `--pt-*` performance upgrade options require the `--use-pacha-tree` flag. + +If an option is omitted, the preview either derives a value from the existing +`influxdb3 serve` configuration or falls back to an engine-specific default +that balances resource usage and throughput. + +> [!Important] +> Set `--num-io-threads` to the number of cores on the machine when using the +> performance upgrade preview. + +- [General](#general) +- [WAL](#wal) +- [Snapshot](#snapshot) +- [Gen0](#gen0) +- [File cache](#file-cache) +- [Replication (query nodes)](#replication-query-nodes) +- [Compactor](#compactor) +- [L1-L4 level tuning](#l1-l4-level-tuning) +- [Example configurations](#example-configurations) +- [Downgrade options](#downgrade-options) + +## General + +| Option | Description | Default | +|:-------|:------------|:--------| +| `--use-pacha-tree` | Enable the performance upgrade preview. Required for any other `--pt-` option to have effect. | Disabled | +| `--pt-engine-path-prefix` | Optional path prefix for all engine data (WAL and compaction generations). Max 32 characters. Must start and end with alphanumeric; inner characters allow `[a-zA-Z0-9._-]`. Shorter paths improve partitioning in object stores. | No prefix | +| `--pt-max-columns` | Maximum total columns across the entire instance. Must be at least 2. | `10,000,000` (10M) | +| `--pt-enable-retention` | Enable retention enforcement. | `true` | +| `--pt-disable-hybrid-query` | Disable hybrid query mode. When the preview is enabled with existing Parquet data, queries normally merge results across both Parquet and `.pt` files. Set this flag to query only `.pt` data. | `false` | +| `--pt-enable-auto-dvc` | Enable automatic distinct value caching for `SHOW TAG VALUES` queries and the `tag_values()` SQL function. | Disabled | +| `--pt-upgrade-poll-interval` | Polling interval for Parquet-to-PachaTree upgrade status monitoring. See [Upgrade from Parquet](/influxdb3/enterprise/performance-preview/#upgrade-from-parquet). | `5s` | + +### Engine path prefix + +Use a short prefix to improve partitioning in object stores: + +```bash +influxdb3 serve \ + # ... + --use-pacha-tree \ + --pt-engine-path-prefix mydata +``` + +### Hybrid query mode + +When you enable the preview on an instance with existing Parquet data, +hybrid query mode merges results across both Parquet and `.pt` files. +Disable hybrid mode to query only `.pt` data: + +```bash +influxdb3 serve \ + # ... + --use-pacha-tree \ + --pt-disable-hybrid-query +``` + +## WAL + +Configure Write-Ahead Log (WAL) behavior for durability and performance. + +| Option | Description | Default | +|:-------|:------------|:--------| +| `--pt-wal-flush-interval` | Flush interval for the WAL. | Inherits `--wal-flush-interval` (1s) | +| `--pt-wal-flush-concurrency` | WAL flush concurrency. | `max(io_threads - 2, 2)` | +| `--pt-wal-max-buffer-size` | Maximum in-memory WAL buffer before a flush is triggered regardless of the flush interval. Increase this if WAL files are flushed before the interval elapses. | `15MB` | +| `--pt-wal-snapshots-to-keep` | Number of snapshot manifests worth of WAL history to retain. Must be greater than 0. | `5` | + +### WAL buffer size + +The WAL buffer accumulates incoming writes before flushing to object storage. +Larger buffers reduce flush frequency and produce larger WAL files, but increase +memory usage: + +```bash +influxdb3 serve \ + # ... + --use-pacha-tree \ + --pt-wal-max-buffer-size 30MB +``` + +### Flush interval and concurrency + +Control how frequently the WAL flushes and how many workers run flushes in +parallel: + +```bash +influxdb3 serve \ + # ... + --use-pacha-tree \ + --pt-wal-flush-interval 2s \ + --pt-wal-flush-concurrency 8 +``` + +## Snapshot + +Configure snapshot buffer behavior, which controls how WAL files are merged +into Gen0 files. + +| Option | Description | Default | +|:-------|:------------|:--------| +| `--pt-snapshot-size` | Maximum size of the active snapshot bucket before it is rotated for snapshotting. | `250MB` | +| `--pt-snapshot-duration` | Time-based snapshot rotation trigger. Controls how often the ingester creates snapshots. Also used on query nodes as the bucket rotation interval for the replica buffer. | `10s` | +| `--pt-max-concurrent-snapshots` | Maximum number of concurrent snapshot operations before applying backpressure to writers. | `5` | +| `--pt-merge-threshold-size` | Maximum unmerged file size before triggering a merge operation. | `--pt-snapshot-size` / 4 (62.5MB) | + +### Snapshot size and duration + +Control when snapshot rotation triggers: + +```bash +influxdb3 serve \ + # ... + --use-pacha-tree \ + --pt-snapshot-size 500MB \ + --pt-snapshot-duration 15s +``` + +### Merge threshold + +Set the size threshold that triggers background merge operations. +Lower values result in more frequent merges: + +```bash +influxdb3 serve \ + # ... + --use-pacha-tree \ + --pt-merge-threshold-size 125MB +``` + +## Gen0 + +Control the size of Gen0 files produced during merge operations. + +| Option | Description | Default | +|:-------|:------------|:--------| +| `--pt-gen0-max-rows-per-file` | Upper bound on rows per Gen0 file emitted during merge. | `10000000` (10M) | +| `--pt-gen0-max-bytes-per-file` | Upper bound on bytes per Gen0 file emitted during merge. | `100MB` | + +### Gen0 file size limits + +Control the size of Gen0 files for query and compaction performance: + +```bash +influxdb3 serve \ + # ... + --use-pacha-tree \ + --pt-gen0-max-rows-per-file 5000000 \ + --pt-gen0-max-bytes-per-file 50MB +``` + +## File cache + +Configure data file caching for query performance. + +| Option | Description | Default | +|:-------|:------------|:--------| +| `--pt-file-cache-size` | Size of the data file cache (bytes or %). Set to `0` on dedicated ingest nodes. | Mirrors `--parquet-mem-cache-size` | +| `--pt-disable-data-file-cache` | Disable data file caching. Set to `true` on dedicated ingest nodes. | `false` (automatically `true` if `--disable-parquet-mem-cache` is set) | +| `--pt-file-cache-recency` | Only cache files newer than this age. Pre-caching on all-in-one and query nodes is based on this value. | Mirrors `--parquet-mem-cache-query-path-duration` (`3d`) | +| `--pt-file-cache-evict-after` | Evict cached files that have not been read within this duration. | `24h` | + +> [!Note] +> #### Dedicated ingest nodes +> On dedicated ingest nodes (`--mode ingest`), disable the data file cache to avoid +> wasting memory on data that ingest nodes never query. +> Set `--pt-file-cache-size 0` or use `--pt-disable-data-file-cache`. +> These options must be explicitly set—they are not applied automatically when +> `--mode ingest` is used. +> See [Disable caching on ingest nodes](#disable-caching-on-ingest-nodes) for an example. + +### File cache size + +Set the maximum size for the data file cache: + +```bash +influxdb3 serve \ + # ... + --use-pacha-tree \ + --pt-file-cache-size 8GB +``` + +### Cache recency filter + +Only cache files containing data within a recent time window: + +```bash +influxdb3 serve \ + # ... + --use-pacha-tree \ + --pt-file-cache-recency 24h +``` + +### Disable caching on ingest nodes + +For dedicated ingest nodes, disable the data file cache to save memory: + +```bash +influxdb3 serve \ + # ... + --use-pacha-tree \ + --mode ingest \ + --pt-disable-data-file-cache +``` + +## Replication (query nodes) + +Configure replication behavior for query nodes in distributed deployments. + +| Option | Description | Default | +|:-------|:------------|:--------| +| `--pt-wal-replication-interval` | Polling interval to check for new WAL files to replicate from ingest nodes. | `250ms` | +| `--pt-wal-replica-recovery-concurrency` | Number of concurrent WAL file fetches during replica recovery or catchup. | `8` | +| `--pt-wal-replica-steady-concurrency` | Number of concurrent WAL file fetches during steady-state replication. | `8` | +| `--pt-wal-replica-queue-size` | Size of the queue between WAL file fetching and replica buffer merging. | `100` | +| `--pt-wal-replica-recovery-tail-skip-limit` | Number of consecutive missing WAL files before stopping replica recovery. | `128` | +| `--pt-replica-gen0-load-concurrency` | Limit on the number of Gen0 files loaded concurrently when the replica starts. | `16` | +| `--pt-replica-max-buffer-size` | Maximum replica buffer size (bytes or %). Used by query nodes to store WAL files replicated from ingest nodes. | 50% of available memory, capped at 16GB | + +### Recovery concurrency + +Control parallelism during query node recovery or catchup: + +```bash +influxdb3 serve \ + # ... + --use-pacha-tree \ + --mode query \ + --pt-wal-replica-recovery-concurrency 16 +``` + +### Steady-state replication + +Configure ongoing replication performance: + +```bash +influxdb3 serve \ + # ... + --use-pacha-tree \ + --mode query \ + --pt-wal-replica-steady-concurrency 4 \ + --pt-wal-replica-queue-size 200 +``` + +### Replica buffer size + +Control the maximum buffer size for replicated data on query nodes: + +```bash +influxdb3 serve \ + # ... + --use-pacha-tree \ + --mode query \ + --pt-replica-max-buffer-size 8GB +``` + +## Compactor + +Configure background compaction behavior. +The compactor organizes data into fixed 24-hour UTC windows and progresses data +through four compaction levels (L1 through L4). + +| Option | Description | Default | +|:-------|:------------|:--------| +| `--pt-partition-count` | Target number of partitions per compaction window. | `1` | +| `--pt-compactor-input-size-budget` | Maximum total input bytes across all active compaction jobs. Acts as an admission control budget for the compactor scheduler. | 50% of system memory at startup | +| `--pt-final-compaction-age` | Age threshold for final compaction. When all L1-L3 run sets in a window are older than this, a final compaction merges everything into L4. | `72h` | +| `--pt-compactor-cleanup-cooldown` | Cooldown after checkpoint publish before replaced files can be cleaned up. | `10min` | + +### Compaction budget + +Control total memory allocated to active compaction jobs: + +```bash +influxdb3 serve \ + # ... + --use-pacha-tree \ + --pt-compactor-input-size-budget 8GB +``` + +### Final compaction age + +Control when windows receive their final compaction into L4: + +```bash +influxdb3 serve \ + # ... + --use-pacha-tree \ + --pt-final-compaction-age 48h +``` + +## L1-L4 level tuning + +These options control per-level compaction parameters. +Data enters L1 from snapshot batch compaction and promotes through levels +based on run set count triggers. + +| Level | Role | Default tail target | Default file size | Promotion trigger | +|:------|:-----|:--------------------|:------------------|:------------------| +| **L1** | Ingest landing zone | 600MB | 25MB | 3 run sets | +| **L2** | First promotion tier | 1.2GB | 40MB | 3 run sets | +| **L3** | Second promotion tier | 2.5GB | 75MB | 4 run sets | +| **L4** | Terminal (fully compacted) | 50GB | 125MB | N/A | + +### L1 options + +| Option | Description | Default | +|:-------|:------------|:--------| +| `--pt-l1-tail-target-bytes` | L1 tail run set target size. | `600MB` | +| `--pt-l1-target-file-bytes` | L1 target file size. | `25MB` | +| `--pt-l1-promotion-count` | Number of L1 run sets that triggers promotion to L2. | `3` | + +### L2 options + +| Option | Description | Default | +|:-------|:------------|:--------| +| `--pt-l2-tail-target-bytes` | L2 tail run set target size. | `1.2GB` | +| `--pt-l2-target-file-bytes` | L2 target file size. | `40MB` | +| `--pt-l2-promotion-count` | Number of L2 run sets that triggers promotion to L3. | `3` | + +### L3 options + +| Option | Description | Default | +|:-------|:------------|:--------| +| `--pt-l3-tail-target-bytes` | L3 tail run set target size. | `2.5GB` | +| `--pt-l3-target-file-bytes` | L3 target file size. | `75MB` | +| `--pt-l3-promotion-count` | Number of L3 run sets that triggers promotion to L4. | `4` | + +### L4 options + +| Option | Description | Default | +|:-------|:------------|:--------| +| `--pt-l4-tail-target-bytes` | L4 tail run set target size. | `50GB` | +| `--pt-l4-target-file-bytes` | L4 target file size. | `125MB` | + +## Example configurations + +### Development (minimal resources) + +```bash +influxdb3 serve \ + --node-id dev01 \ + --cluster-id dev-cluster \ + --object-store file \ + --data-dir ~/.influxdb3 \ + --use-pacha-tree \ + --num-io-threads 2 \ + --pt-file-cache-size 512MB \ + --pt-wal-max-buffer-size 5MB \ + --pt-snapshot-size 100MB +``` + +### Production all-in-one (8 cores, 32 GB RAM) + +```bash +influxdb3 serve \ + --node-id prod01 \ + --cluster-id prod-cluster \ + --object-store s3 \ + --bucket S3_BUCKET \ + --aws-access-key-id AWS_ACCESS_KEY_ID \ + --aws-secret-access-key AWS_SECRET_ACCESS_KEY \ + --use-pacha-tree \ + --num-io-threads 8 \ + --pt-file-cache-size 8GB \ + --pt-wal-max-buffer-size 30MB \ + --pt-snapshot-size 500MB \ + --pt-wal-flush-concurrency 4 +``` + +### High-throughput ingest node + +```bash +influxdb3 serve \ + --node-id ingest01 \ + --cluster-id prod-cluster \ + --object-store s3 \ + --bucket S3_BUCKET \ + --aws-access-key-id AWS_ACCESS_KEY_ID \ + --aws-secret-access-key AWS_SECRET_ACCESS_KEY \ + --use-pacha-tree \ + --mode ingest \ + --num-io-threads 16 \ + --pt-wal-max-buffer-size 50MB \ + --pt-wal-flush-interval 2s \ + --pt-wal-flush-concurrency 8 \ + --pt-snapshot-size 1GB \ + --pt-disable-data-file-cache +``` + +### Query-optimized node + +```bash +influxdb3 serve \ + --node-id query01 \ + --cluster-id prod-cluster \ + --object-store s3 \ + --bucket S3_BUCKET \ + --aws-access-key-id AWS_ACCESS_KEY_ID \ + --aws-secret-access-key AWS_SECRET_ACCESS_KEY \ + --use-pacha-tree \ + --mode query \ + --num-io-threads 16 \ + --pt-file-cache-size 16GB \ + --pt-file-cache-recency 24h \ + --pt-replica-max-buffer-size 8GB +``` + +### Dedicated compactor + +```bash +influxdb3 serve \ + --node-id compact01 \ + --cluster-id prod-cluster \ + --object-store s3 \ + --bucket S3_BUCKET \ + --aws-access-key-id AWS_ACCESS_KEY_ID \ + --aws-secret-access-key AWS_SECRET_ACCESS_KEY \ + --use-pacha-tree \ + --mode compact \ + --num-io-threads 8 \ + --pt-compactor-input-size-budget 12GB +``` + +## Downgrade options + +The `influxdb3 downgrade-to-parquet` command reverts a cluster from the +performance preview back to standard Parquet storage. +For the downgrade procedure, see +[Downgrade to Parquet](/influxdb3/enterprise/performance-preview/#downgrade-to-parquet). + +| Option | Description | +|:-------|:------------| +| `--cluster-id` | _(Required)_ Cluster identifier. | +| `--object-store` | _(Required)_ Object storage type (`file`, `s3`, `gcs`, `azure`). | +| `--data-dir` | Location of data files for a local (`file`) object store. | +| `--bucket` | Object store bucket name (for `s3`, `gcs`, `azure`). | +| `--dry-run` | Preview mode--list files that would be deleted without making changes. | +| `--yes` | Skip the confirmation prompt. | +| `--ignore-running` | Proceed even if nodes appear to be running. **Warning:** may cause data inconsistency if nodes are actively writing. | diff --git a/content/influxdb3/enterprise/performance-preview/monitor.md b/content/influxdb3/enterprise/performance-preview/monitor.md new file mode 100644 index 0000000000..958c2e745f --- /dev/null +++ b/content/influxdb3/enterprise/performance-preview/monitor.md @@ -0,0 +1,354 @@ +--- +title: Monitor the performance upgrade preview +seotitle: Monitor the performance upgrade preview in InfluxDB 3 Enterprise +description: > + Use system tables and query telemetry to monitor file status, query execution, + and overall performance when using InfluxDB 3 Enterprise performance upgrades. +menu: + influxdb3_enterprise: + name: Monitor + parent: Performance upgrade preview +weight: 203 +influxdb3/enterprise/tags: [storage, monitoring, beta, preview, system tables] +related: + - /influxdb3/enterprise/performance-preview/ + - /influxdb3/enterprise/performance-preview/configure/ + - /influxdb3/enterprise/admin/query-system-data/ +--- + +> [!Warning] +> #### Performance preview beta +> The performance upgrade preview is available to {{% product-name %}} Trial +> and Commercial users as a beta. These features are subject to breaking changes +> and **should not be used for production workloads**. + +{{% product-name %}} provides system tables and a query telemetry endpoint to +monitor file status, query execution, and overall performance when using the +performance upgrade preview. + +## System tables + +The upgraded storage engine exposes internal state through system tables that +you can query with SQL. + +### system.pt_ingest_wal + +View WAL files and their partitions: + +```sql +SELECT * FROM system.pt_ingest_wal; +``` + +Example output: + +| wal_file_id | partition_id | database_id | table_id | min_time | max_time | row_count | size_bytes | +|:------------|:-------------|:------------|:---------|:---------|:---------|:----------|:-----------| +| wal_001 | p_1 | db_1 | t_1 | 2024-01-01T00:00:00Z | 2024-01-01T00:10:00Z | 50000 | 2456789 | +| wal_002 | p_1 | db_1 | t_1 | 2024-01-01T00:10:00Z | 2024-01-01T00:20:00Z | 48000 | 2345678 | + +Use this table to monitor: + +- **WAL accumulation**: Track the number and size of unmerged WAL files +- **Partition distribution**: See how data is distributed across partitions +- **Time coverage**: Verify data time ranges + +#### Monitor WAL backlog + +Check for WAL accumulation that may indicate merging is falling behind: + +```sql +SELECT + COUNT(*) as wal_file_count, + SUM(size_bytes) / 1024 / 1024 as total_size_mb, + MIN(min_time) as oldest_data, + MAX(max_time) as newest_data +FROM system.pt_ingest_wal; +``` + +### system.pt_ingest_files + +View Gen0 files with metadata: + +```sql +SELECT * FROM system.pt_ingest_files; +``` + +Example output: + +| file_id | generation | database_id | table_id | min_time | max_time | row_count | size_bytes | +|:--------|:-----------|:------------|:---------|:---------|:---------|:----------|:-----------| +| gen0_001 | 0 | db_1 | t_1 | 2024-01-01T00:00:00Z | 2024-01-01T01:00:00Z | 500000 | 45678901 | +| gen0_002 | 0 | db_1 | t_1 | 2024-01-01T01:00:00Z | 2024-01-01T02:00:00Z | 480000 | 43567890 | + +Use this table to monitor: + +- **File counts per generation**: Track compaction progress +- **File sizes**: Verify files are within configured limits +- **Time ranges**: Identify Gen0 files that span multiple compaction windows + +#### Monitor file distribution + +Check file distribution and compaction status: + +```sql +SELECT + generation, + COUNT(*) as file_count, + SUM(row_count) as total_rows, + SUM(size_bytes) / 1024 / 1024 as total_size_mb, + AVG(size_bytes) / 1024 / 1024 as avg_file_size_mb +FROM system.pt_ingest_files +GROUP BY generation +ORDER BY generation; +``` + +## Parquet upgrade status + +If you [upgraded from Parquet](/influxdb3/enterprise/performance-preview/#upgrade-from-parquet), +use these system tables to monitor migration progress. + +### system.upgrade_parquet_node + +View per-node upgrade status: + +```sql +SELECT * FROM system.upgrade_parquet_node; +``` + +Monitor this table to confirm each node reaches `completed` status. +During the upgrade, nodes progress through detection, conversion, and +finalization stages. + +### system.upgrade_parquet + +View per-file migration progress: + +```sql +SELECT * FROM system.upgrade_parquet; +``` + +Use this table to track individual file conversions during the migration. +The status updates on a polling interval (default 5 seconds, configurable with +`--pt-upgrade-poll-interval`). + +## Query telemetry + +The query telemetry endpoint provides detailed execution statistics for +analyzing query performance. + +### Enable query telemetry + +Query the telemetry endpoint after executing a query: + +```bash +curl -X GET "http://localhost:8181/api/v3/query_sql_telemetry" \ + -H "Authorization: Bearer AUTH_TOKEN" +``` + +Replace `AUTH_TOKEN` with your authentication token. + +### Telemetry response + +The response includes: + +| Field | Description | +|:------|:------------| +| `query_id` | Unique identifier for the query | +| `execution_time_us` | Total execution time in microseconds | +| `chunks` | Per-chunk statistics | +| `cache_stats` | Cache hit rates by type | +| `file_stats` | File-level read statistics | + +### Example telemetry output + +```json +{ + "query_id": "q_12345", + "execution_time_us": 4523, + "chunks": [ + { + "chunk_id": "c_1", + "files_scanned": 3, + "blocks_processed": 12, + "rows_read": 24000, + "rows_returned": 150, + "bytes_read": 1234567 + } + ], + "cache_stats": { + "gen0_hits": 5, + "gen0_misses": 1, + "compacted_hits": 8, + "compacted_misses": 2 + } +} +``` + +## Performance analysis + +### Query performance metrics + +Track these key metrics for query performance: + +| Metric | Good | Warning | Action | +|:-------|:-----|:--------|:-------| +| Cache hit rate | >80% | <60% | Increase `--pt-file-cache-size` or `--pt-file-cache-recency` | +| Rows read vs returned ratio | <100:1 | >1000:1 | Add more selective predicates | + +### Ingest performance metrics + +Monitor these metrics for write performance: + +| Metric | Healthy | Warning | Action | +|:-------|:--------|:--------|:-------| +| WAL file count | <50 | >100 | Increase `--pt-wal-flush-concurrency` | +| Gen0 file count | <100 | >200 | Increase `--pt-compactor-input-size-budget` | + +### Monitor with SQL + +Create a performance summary query: + +```sql +-- File generation summary +SELECT + 'Gen0 files' as metric, + COUNT(*) as count, + SUM(size_bytes) / 1024 / 1024 as size_mb +FROM system.pt_ingest_files +WHERE generation = 0 + +UNION ALL + +SELECT + 'Compacted files' as metric, + COUNT(*) as count, + SUM(size_bytes) / 1024 / 1024 as size_mb +FROM system.pt_ingest_files +WHERE generation > 0 + +UNION ALL + +SELECT + 'WAL files' as metric, + COUNT(*) as count, + SUM(size_bytes) / 1024 / 1024 as size_mb +FROM system.pt_ingest_wal; +``` + +## Troubleshooting + +### High WAL file count + +**Symptom**: `system.pt_ingest_wal` shows many accumulated files. + +**Possible causes**: + +- Merge operations falling behind write rate +- Insufficient flush concurrency +- Object storage latency + +**Solutions**: + +1. Increase flush concurrency: + ```bash + --pt-wal-flush-concurrency 8 + ``` + +2. Increase WAL flush interval to create larger, fewer files: + ```bash + --pt-wal-flush-interval 5s + ``` + +3. Increase the WAL buffer size so each flush produces a larger file: + ```bash + --pt-wal-max-buffer-size 30MB + ``` + +4. Check object storage performance and connectivity. + +### High cache miss rate + +**Symptom**: `cache_stats` shows >40% miss rate. + +**Possible causes**: + +- Cache size too small for working set +- Cache recency window too narrow +- Random access patterns across time ranges + +**Solutions**: + +1. Increase cache size: + ```bash + --pt-file-cache-size 16GB + ``` + +2. Extend cache recency window: + ```bash + --pt-file-cache-recency 24h + ``` + +3. Extend eviction timeout: + ```bash + --pt-file-cache-evict-after 48h + ``` + +### Slow compaction + +**Symptom**: Gen0 file count continues to grow. + +**Possible causes**: + +- Compaction budget too low for write volume +- High write rate overwhelming compaction +- Snapshot size too large, creating oversized Gen0 files + +**Solutions**: + +1. Increase the compaction input size budget: + ```bash + --pt-compactor-input-size-budget 12GB + ``` + +2. Reduce snapshot size to create smaller, more frequent Gen0 files: + ```bash + --pt-snapshot-size 125MB + ``` + +3. For distributed deployments, add a dedicated compactor node: + ```bash + influxdb3 serve \ + # ... + --use-pacha-tree \ + --mode compact + ``` + +### Query node lag + +**Symptom**: Query nodes return stale data. + +**Possible causes**: + +- Replication falling behind +- Network latency to object storage +- Insufficient replica concurrency + +**Solutions**: + +For a full list of replication options, see +[Replication (query nodes)](/influxdb3/enterprise/performance-preview/configure/#replication-query-nodes). + +1. Increase replication concurrency: + ```bash + --pt-wal-replica-steady-concurrency 8 + ``` + +2. Reduce the replication polling interval: + ```bash + --pt-wal-replication-interval 100ms + ``` + +3. Increase replica queue size: + ```bash + --pt-wal-replica-queue-size 200 + ```