|
| 1 | +================================================================================================ |
| 2 | +Coalesced RDD , large scale |
| 3 | +================================================================================================ |
| 4 | + |
| 5 | +OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64 |
| 6 | +Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz |
| 7 | +Coalesced RDD: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative |
| 8 | +------------------------------------------------------------------------------------------------------------------------ |
| 9 | +Coalesce Num Partitions: 100 Num Hosts: 1 455 623 151 0.2 4547.4 1.0X |
| 10 | +Coalesce Num Partitions: 100 Num Hosts: 5 340 409 65 0.3 3397.1 1.3X |
| 11 | +Coalesce Num Partitions: 100 Num Hosts: 10 292 363 95 0.3 2923.3 1.6X |
| 12 | +Coalesce Num Partitions: 100 Num Hosts: 20 307 320 11 0.3 3069.8 1.5X |
| 13 | +Coalesce Num Partitions: 100 Num Hosts: 40 333 368 55 0.3 3329.1 1.4X |
| 14 | +Coalesce Num Partitions: 100 Num Hosts: 80 286 338 63 0.3 2862.5 1.6X |
| 15 | +Coalesce Num Partitions: 500 Num Hosts: 1 769 837 59 0.1 7693.5 0.6X |
| 16 | +Coalesce Num Partitions: 500 Num Hosts: 5 427 461 31 0.2 4268.5 1.1X |
| 17 | +Coalesce Num Partitions: 500 Num Hosts: 10 372 389 27 0.3 3722.2 1.2X |
| 18 | +Coalesce Num Partitions: 500 Num Hosts: 20 347 365 31 0.3 3468.5 1.3X |
| 19 | +Coalesce Num Partitions: 500 Num Hosts: 40 335 336 1 0.3 3347.3 1.4X |
| 20 | +Coalesce Num Partitions: 500 Num Hosts: 80 329 360 49 0.3 3294.5 1.4X |
| 21 | +Coalesce Num Partitions: 1000 Num Hosts: 1 1254 1292 47 0.1 12538.6 0.4X |
| 22 | +Coalesce Num Partitions: 1000 Num Hosts: 5 518 553 47 0.2 5177.0 0.9X |
| 23 | +Coalesce Num Partitions: 1000 Num Hosts: 10 394 432 42 0.3 3937.3 1.2X |
| 24 | +Coalesce Num Partitions: 1000 Num Hosts: 20 341 381 44 0.3 3414.4 1.3X |
| 25 | +Coalesce Num Partitions: 1000 Num Hosts: 40 313 358 48 0.3 3134.9 1.5X |
| 26 | +Coalesce Num Partitions: 1000 Num Hosts: 80 335 360 38 0.3 3347.0 1.4X |
| 27 | +Coalesce Num Partitions: 5000 Num Hosts: 1 3937 4066 156 0.0 39375.0 0.1X |
| 28 | +Coalesce Num Partitions: 5000 Num Hosts: 5 1413 1453 40 0.1 14133.4 0.3X |
| 29 | +Coalesce Num Partitions: 5000 Num Hosts: 10 826 861 49 0.1 8255.2 0.6X |
| 30 | +Coalesce Num Partitions: 5000 Num Hosts: 20 542 609 58 0.2 5423.3 0.8X |
| 31 | +Coalesce Num Partitions: 5000 Num Hosts: 40 410 470 64 0.2 4101.0 1.1X |
| 32 | +Coalesce Num Partitions: 5000 Num Hosts: 80 352 427 69 0.3 3515.3 1.3X |
| 33 | +Coalesce Num Partitions: 10000 Num Hosts: 1 7101 7151 54 0.0 71007.4 0.1X |
| 34 | +Coalesce Num Partitions: 10000 Num Hosts: 5 2540 2582 59 0.0 25396.2 0.2X |
| 35 | +Coalesce Num Partitions: 10000 Num Hosts: 10 1378 1432 48 0.1 13781.4 0.3X |
| 36 | +Coalesce Num Partitions: 10000 Num Hosts: 20 829 867 66 0.1 8286.8 0.5X |
| 37 | +Coalesce Num Partitions: 10000 Num Hosts: 40 573 630 49 0.2 5730.2 0.8X |
| 38 | +Coalesce Num Partitions: 10000 Num Hosts: 80 438 449 9 0.2 4382.5 1.0X |
| 39 | + |
| 40 | + |
0 commit comments