Skip to content

Add "Extended Clickbench" benchmark for median and approx_median for high cardinality aggregates #12438

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
alamb merged 7 commits into from
Sep 16, 2024
Merged

Add "Extended Clickbench" benchmark for median and approx_median for high cardinality aggregates #12438

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
34b67e6
Add "Extended Clickbench" benchmark for median and approx_median for …
alamb Sep 11, 2024
1824593
Merge remote-tracking branch 'apache/main' into alamb/test_mediate_perf
alamb Sep 12, 2024
5bcd2a1
Update benchmarks/queries/clickbench/extended.sql
alamb Sep 12, 2024
55a0436
git Merge branch 'alamb/test_mediate_perf' of github.com:alamb/datafu…
alamb Sep 12, 2024
5b1522c
fix doc
alamb Sep 12, 2024
6ec7789
Merge remote-tracking branch 'apache/main' into alamb/test_mediate_perf
alamb Sep 13, 2024
e86280b
Update queries to be something more reasonable
alamb Sep 13, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
58 changes: 57 additions & 1 deletion benchmarks/queries/clickbench/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ LIMIT 10;

### Q3: What is the income distribution for users in specific regions

**Question**: "What regions and social networks have the highest variance of parameter price
**Question**: "What regions and social networks have the highest variance of parameter price?"

**Important Query Properties**: STDDEV and VAR aggregation functions, GROUP BY multiple small ints

Expand All @@ -73,6 +73,62 @@ ORDER BY s DESC
LIMIT 10;
```

### Q4: Response start time distribution analysis (median)

**Question**: Find the WatchIDs with the highest median "ResponseStartTiming" without Java enabled

**Important Query Properties**: MEDIAN, functions, high cardinality grouping that skips intermediate aggregation

Note this query is somewhat synthetic as "WatchID" is almost unique (there are a few duplicates)

```sql
SELECT "ClientIP", "WatchID", COUNT(*) c, MIN("ResponseStartTiming") tmin, MEDIAN("ResponseStartTiming") tmed, MAX("ResponseStartTiming") tmax
FROM 'hits.parquet'
WHERE "JavaEnable" = 0 -- filters to 32M of 100M rows
GROUP BY "ClientIP", "WatchID"
HAVING c > 1
ORDER BY tmed DESC
LIMIT 10;
```

Results look like

+-------------+---------------------+---+------+------+------+
| ClientIP | WatchID | c | tmin | tmed | tmax |
+-------------+---------------------+---+------+------+------+
| 1611957945 | 6655575552203051303 | 2 | 0 | 0 | 0 |
| -1402644643 | 8566928176839891583 | 2 | 0 | 0 | 0 |
+-------------+---------------------+---+------+------+------+


### Q5: Response start time distribution analysis (p95)

**Question**: Find the WatchIDs with the highest p95 "ResponseStartTiming" without Java enabled

**Important Query Properties**: APPROX_PERCENTILE_CONT, functions, high cardinality grouping that skips intermediate aggregation

Note this query is somewhat synthetic as "WatchID" is almost unique (there are a few duplicates)

```sql
SELECT "ClientIP", "WatchID", COUNT(*) c, MIN("ResponseStartTiming") tmin, APPROX_PERCENTILE_CONT("ResponseStartTiming", 0.95) tp95, MAX("ResponseStartTiming") tmax
FROM 'hits.parquet'
WHERE "JavaEnable" = 0 -- filters to 32M of 100M rows
GROUP BY "ClientIP", "WatchID"
HAVING c > 1
ORDER BY tp95 DESC
LIMIT 10;
```

Results look like

+-------------+---------------------+---+------+------+------+
| ClientIP | WatchID | c | tmin | tp95 | tmax |
+-------------+---------------------+---+------+------+------+
| 1611957945 | 6655575552203051303 | 2 | 0 | 0 | 0 |
| -1402644643 | 8566928176839891583 | 2 | 0 | 0 | 0 |
+-------------+---------------------+---+------+------+------+


## Data Notes

Here are some interesting statistics about the data used in the queries
Expand Down
2 changes: 2 additions & 0 deletions benchmarks/queries/clickbench/extended.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,5 @@ SELECT COUNT(DISTINCT "SearchPhrase"), COUNT(DISTINCT "MobilePhone"), COUNT(DIST
SELECT COUNT(DISTINCT "HitColor"), COUNT(DISTINCT "BrowserCountry"), COUNT(DISTINCT "BrowserLanguage") FROM hits;
SELECT "BrowserCountry", COUNT(DISTINCT "SocialNetwork"), COUNT(DISTINCT "HitColor"), COUNT(DISTINCT "BrowserLanguage"), COUNT(DISTINCT "SocialAction") FROM hits GROUP BY 1 ORDER BY 2 DESC LIMIT 10;
SELECT "SocialSourceNetworkID", "RegionID", COUNT(*), AVG("Age"), AVG("ParamPrice"), STDDEV("ParamPrice") as s, VAR("ParamPrice") FROM hits GROUP BY "SocialSourceNetworkID", "RegionID" HAVING s IS NOT NULL ORDER BY s DESC LIMIT 10;
SELECT "ClientIP", "WatchID", COUNT(*) c, MIN("ResponseStartTiming") tmin, MEDIAN("ResponseStartTiming") tmed, MAX("ResponseStartTiming") tmax FROM hits WHERE "JavaEnable" = 0 GROUP BY "ClientIP", "WatchID" HAVING c > 1 ORDER BY tmed DESC LIMIT 10;
SELECT "ClientIP", "WatchID", COUNT(*) c, MIN("ResponseStartTiming") tmin, APPROX_PERCENTILE_CONT("ResponseStartTiming", 0.95) tp95, MAX("ResponseStartTiming") tmax FROM 'hits' WHERE "JavaEnable" = 0 GROUP BY "ClientIP", "WatchID" HAVING c > 1 ORDER BY tp95 DESC LIMIT 10;