Skip to content

Commit 35bd2d8

Browse files
zsxwingjeanlyn
zsxwing
authored and
jeanlyn
committed
[SPARK-6939] [STREAMING] [WEBUI] Add timeline and histogram graphs for streaming statistics
This is the initial work of SPARK-6939. Not yet ready for code review. Here are the screenshots: ![graph1](https://cloud.githubusercontent.com/assets/1000778/7165766/465942e0-e3dc-11e4-9b05-c184b09d75dc.png) ![graph2](https://cloud.githubusercontent.com/assets/1000778/7165779/53f13f34-e3dc-11e4-8714-a4a75b7e09ff.png) TODOs: - [x] Display more information on mouse hover - [x] Align the timeline and distribution graphs - [x] Clean up the codes Author: zsxwing <zsxwing@gmail.com> Closes apache#5533 from zsxwing/SPARK-6939 and squashes the following commits: 9f7cd19 [zsxwing] Merge branch 'master' into SPARK-6939 deacc3f [zsxwing] Remove unused import cd03424 [zsxwing] Fix .rat-excludes 70cc87d [zsxwing] Streaming Scheduling Delay => Scheduling Delay d457277 [zsxwing] Fix UIUtils in BatchPage b3f303e [zsxwing] Add comments for unclear classes and methods ff0bff8 [zsxwing] Make InputDStream.name private[streaming] cc392c5 [zsxwing] Merge branch 'master' into SPARK-6939 e275e23 [zsxwing] Move time related methods to Streaming's UIUtils d5d86f6 [zsxwing] Fix incorrect lastErrorTime 3be4b7a [zsxwing] Use InputInfo b50fa32 [zsxwing] Jump to the batch page when clicking a point in the timeline graphs 203605d [zsxwing] Merge branch 'master' into SPARK-6939 74307cf [zsxwing] Reuse the data for histogram graphs to reduce the page size 2586916 [zsxwing] Merge branch 'master' into SPARK-6939 70d8533 [zsxwing] Remove BatchInfo.numRecords and a few renames 7bbdc0a [zsxwing] Hide the receiver sub table if no receiver a2972e9 [zsxwing] Add some ui tests for StreamingPage fd03ad0 [zsxwing] Add a test to verify no memory leak 4a8f886 [zsxwing] Merge branch 'master' into SPARK-6939 18607a1 [zsxwing] Merge branch 'master' into SPARK-6939 d0b0aec [zsxwing] Clean up the codes a459f49 [zsxwing] Add a dash line to processing time graphs 8e4363c [zsxwing] Prepare for the demo c81a1ee [zsxwing] Change time unit in the graphs automatically 4c0b43f [zsxwing] Update Streaming UI 04c7500 [zsxwing] Make the server and client use the same timezone fed8219 [zsxwing] Move the x axis at the top and show a better tooltip c23ce10 [zsxwing] Make two graphs close d78672a [zsxwing] Make the X axis use the same range 881c907 [zsxwing] Use histogram for distribution 5688702 [zsxwing] Fix the unit test ddf741a [zsxwing] Fix the unit test ad93295 [zsxwing] Remove unnecessary codes a0458f9 [zsxwing] Clean the codes b82ed1e [zsxwing] Update the graphs as per comments dd653a1 [zsxwing] Add timeline and histogram graphs for streaming statistics
1 parent 4f5c132 commit 35bd2d8

17 files changed

Lines changed: 1228 additions & 244 deletions

File tree

‎LICENSE‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -643,6 +643,36 @@ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
643643
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
644644
THE SOFTWARE.
645645

646+
========================================================================
647+
For d3 (core/src/main/resources/org/apache/spark/ui/static/d3.min.js):
648+
========================================================================
649+
650+
Copyright (c) 2010-2015, Michael Bostock
651+
All rights reserved.
652+
653+
Redistribution and use in source and binary forms, with or without
654+
modification, are permitted provided that the following conditions are met:
655+
656+
* Redistributions of source code must retain the above copyright notice, this
657+
list of conditions and the following disclaimer.
658+
659+
* Redistributions in binary form must reproduce the above copyright notice,
660+
this list of conditions and the following disclaimer in the documentation
661+
and/or other materials provided with the distribution.
662+
663+
* The name Michael Bostock may not be used to endorse or promote products
664+
derived from this software without specific prior written permission.
665+
666+
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
667+
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
668+
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
669+
DISCLAIMED. IN NO EVENT SHALL MICHAEL BOSTOCK BE LIABLE FOR ANY DIRECT,
670+
INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
671+
BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
672+
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
673+
OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
674+
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE,
675+
EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
646676

647677
========================================================================
648678
For Scala Interpreter classes (all .scala files in repl/src/main/scala

‎core/src/main/resources/org/apache/spark/ui/static/bootstrap-tooltip.js‎

Lines changed: 104 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,9 @@
11
/* ===========================================================
2-
* bootstrap-tooltip.js v2.2.2
3-
* http://twitter.github.com/bootstrap/javascript.html#tooltips
2+
* bootstrap-tooltip.js v2.3.2
3+
* http://getbootstrap.com/2.3.2/javascript.html#tooltips
44
* Inspired by the original jQuery.tipsy by Jason Frame
55
* ===========================================================
6-
* Copyright 2012 Twitter, Inc.
6+
* Copyright 2013 Twitter, Inc.
77
*
88
* Licensed under the Apache License, Version 2.0 (the "License");
99
* you may not use this file except in compliance with the License.
@@ -38,19 +38,27 @@
3838
, init: function (type, element, options) {
3939
var eventIn
4040
, eventOut
41+
, triggers
42+
, trigger
43+
, i
4144

4245
this.type = type
4346
this.$element = $(element)
4447
this.options = this.getOptions(options)
4548
this.enabled = true
4649

47-
if (this.options.trigger == 'click') {
48-
this.$element.on('click.' + this.type, this.options.selector, $.proxy(this.toggle, this))
49-
} else if (this.options.trigger != 'manual') {
50-
eventIn = this.options.trigger == 'hover' ? 'mouseenter' : 'focus'
51-
eventOut = this.options.trigger == 'hover' ? 'mouseleave' : 'blur'
52-
this.$element.on(eventIn + '.' + this.type, this.options.selector, $.proxy(this.enter, this))
53-
this.$element.on(eventOut + '.' + this.type, this.options.selector, $.proxy(this.leave, this))
50+
triggers = this.options.trigger.split(' ')
51+
52+
for (i = triggers.length; i--;) {
53+
trigger = triggers[i]
54+
if (trigger == 'click') {
55+
this.$element.on('click.' + this.type, this.options.selector, $.proxy(this.toggle, this))
56+
} else if (trigger != 'manual') {
57+
eventIn = trigger == 'hover' ? 'mouseenter' : 'focus'
58+
eventOut = trigger == 'hover' ? 'mouseleave' : 'blur'
59+
this.$element.on(eventIn + '.' + this.type, this.options.selector, $.proxy(this.enter, this))
60+
this.$element.on(eventOut + '.' + this.type, this.options.selector, $.proxy(this.leave, this))
61+
}
5462
}
5563

5664
this.options.selector ?
@@ -59,7 +67,7 @@
5967
}
6068

6169
, getOptions: function (options) {
62-
options = $.extend({}, $.fn[this.type].defaults, options, this.$element.data())
70+
options = $.extend({}, $.fn[this.type].defaults, this.$element.data(), options)
6371

6472
if (options.delay && typeof options.delay == 'number') {
6573
options.delay = {
@@ -72,7 +80,15 @@
7280
}
7381

7482
, enter: function (e) {
75-
var self = $(e.currentTarget)[this.type](this._options).data(this.type)
83+
var defaults = $.fn[this.type].defaults
84+
, options = {}
85+
, self
86+
87+
this._options && $.each(this._options, function (key, value) {
88+
if (defaults[key] != value) options[key] = value
89+
}, this)
90+
91+
self = $(e.currentTarget)[this.type](options).data(this.type)
7692

7793
if (!self.options.delay || !self.options.delay.show) return self.show()
7894

@@ -97,14 +113,16 @@
97113

98114
, show: function () {
99115
var $tip
100-
, inside
101116
, pos
102117
, actualWidth
103118
, actualHeight
104119
, placement
105120
, tp
121+
, e = $.Event('show')
106122

107123
if (this.hasContent() && this.enabled) {
124+
this.$element.trigger(e)
125+
if (e.isDefaultPrevented()) return
108126
$tip = this.tip()
109127
this.setContent()
110128

@@ -116,19 +134,18 @@
116134
this.options.placement.call(this, $tip[0], this.$element[0]) :
117135
this.options.placement
118136

119-
inside = /in/.test(placement)
120-
121137
$tip
122138
.detach()
123139
.css({ top: 0, left: 0, display: 'block' })
124-
.insertAfter(this.$element)
125140

126-
pos = this.getPosition(inside)
141+
this.options.container ? $tip.appendTo(this.options.container) : $tip.insertAfter(this.$element)
142+
143+
pos = this.getPosition()
127144

128145
actualWidth = $tip[0].offsetWidth
129146
actualHeight = $tip[0].offsetHeight
130147

131-
switch (inside ? placement.split(' ')[1] : placement) {
148+
switch (placement) {
132149
case 'bottom':
133150
tp = {top: pos.top + pos.height, left: pos.left + pos.width / 2 - actualWidth / 2}
134151
break
@@ -143,11 +160,56 @@
143160
break
144161
}
145162

146-
$tip
147-
.offset(tp)
148-
.addClass(placement)
149-
.addClass('in')
163+
this.applyPlacement(tp, placement)
164+
this.$element.trigger('shown')
165+
}
166+
}
167+
168+
, applyPlacement: function(offset, placement){
169+
var $tip = this.tip()
170+
, width = $tip[0].offsetWidth
171+
, height = $tip[0].offsetHeight
172+
, actualWidth
173+
, actualHeight
174+
, delta
175+
, replace
176+
177+
$tip
178+
.offset(offset)
179+
.addClass(placement)
180+
.addClass('in')
181+
182+
actualWidth = $tip[0].offsetWidth
183+
actualHeight = $tip[0].offsetHeight
184+
185+
if (placement == 'top' && actualHeight != height) {
186+
offset.top = offset.top + height - actualHeight
187+
replace = true
188+
}
189+
190+
if (placement == 'bottom' || placement == 'top') {
191+
delta = 0
192+
193+
if (offset.left < 0){
194+
delta = offset.left * -2
195+
offset.left = 0
196+
$tip.offset(offset)
197+
actualWidth = $tip[0].offsetWidth
198+
actualHeight = $tip[0].offsetHeight
199+
}
200+
201+
this.replaceArrow(delta - width + actualWidth, actualWidth, 'left')
202+
} else {
203+
this.replaceArrow(actualHeight - height, actualHeight, 'top')
150204
}
205+
206+
if (replace) $tip.offset(offset)
207+
}
208+
209+
, replaceArrow: function(delta, dimension, position){
210+
this
211+
.arrow()
212+
.css(position, delta ? (50 * (1 - delta / dimension) + "%") : '')
151213
}
152214

153215
, setContent: function () {
@@ -161,6 +223,10 @@
161223
, hide: function () {
162224
var that = this
163225
, $tip = this.tip()
226+
, e = $.Event('hide')
227+
228+
this.$element.trigger(e)
229+
if (e.isDefaultPrevented()) return
164230

165231
$tip.removeClass('in')
166232

@@ -179,6 +245,8 @@
179245
removeWithAnimation() :
180246
$tip.detach()
181247

248+
this.$element.trigger('hidden')
249+
182250
return this
183251
}
184252

@@ -193,11 +261,12 @@
193261
return this.getTitle()
194262
}
195263

196-
, getPosition: function (inside) {
197-
return $.extend({}, (inside ? {top: 0, left: 0} : this.$element.offset()), {
198-
width: this.$element[0].offsetWidth
199-
, height: this.$element[0].offsetHeight
200-
})
264+
, getPosition: function () {
265+
var el = this.$element[0]
266+
return $.extend({}, (typeof el.getBoundingClientRect == 'function') ? el.getBoundingClientRect() : {
267+
width: el.offsetWidth
268+
, height: el.offsetHeight
269+
}, this.$element.offset())
201270
}
202271

203272
, getTitle: function () {
@@ -215,6 +284,10 @@
215284
return this.$tip = this.$tip || $(this.options.template)
216285
}
217286

287+
, arrow: function(){
288+
return this.$arrow = this.$arrow || this.tip().find(".tooltip-arrow")
289+
}
290+
218291
, validate: function () {
219292
if (!this.$element[0].parentNode) {
220293
this.hide()
@@ -236,8 +309,8 @@
236309
}
237310

238311
, toggle: function (e) {
239-
var self = $(e.currentTarget)[this.type](this._options).data(this.type)
240-
self[self.tip().hasClass('in') ? 'hide' : 'show']()
312+
var self = e ? $(e.currentTarget)[this.type](this._options).data(this.type) : this
313+
self.tip().hasClass('in') ? self.hide() : self.show()
241314
}
242315

243316
, destroy: function () {
@@ -269,10 +342,11 @@
269342
, placement: 'top'
270343
, selector: false
271344
, template: '<div class="tooltip"><div class="tooltip-arrow"></div><div class="tooltip-inner"></div></div>'
272-
, trigger: 'hover'
345+
, trigger: 'hover focus'
273346
, title: ''
274347
, delay: 0
275348
, html: false
349+
, container: false
276350
}
277351

278352

@@ -285,4 +359,3 @@
285359
}
286360

287361
}(window.jQuery);
288-

‎core/src/main/resources/org/apache/spark/ui/static/streaming-page.css‎

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
/*
2+
* Licensed to the Apache Software Foundation (ASF) under one or more
3+
* contributor license agreements. See the NOTICE file distributed with
4+
* this work for additional information regarding copyright ownership.
5+
* The ASF licenses this file to You under the Apache License, Version 2.0
6+
* (the "License"); you may not use this file except in compliance with
7+
* the License. You may obtain a copy of the License at
8+
*
9+
* http://www.apache.org/licenses/LICENSE-2.0
10+
*
11+
* Unless required by applicable law or agreed to in writing, software
12+
* distributed under the License is distributed on an "AS IS" BASIS,
13+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
* See the License for the specific language governing permissions and
15+
* limitations under the License.
16+
*/
17+
18+
19+
.graph {
20+
font: 10px sans-serif;
21+
}
22+
23+
.axis path, .axis line {
24+
fill: none;
25+
stroke: gray;
26+
shape-rendering: crispEdges;
27+
}
28+
29+
.axis text {
30+
fill: gray;
31+
}
32+
33+
.tooltip-inner {
34+
max-width: 500px !important; // Make sure we only have one line tooltip
35+
}
36+
37+
.line {
38+
fill: none;
39+
stroke: #0088cc;
40+
stroke-width: 1.5px;
41+
}
42+
43+
.bar rect {
44+
fill: #0088cc;
45+
shape-rendering: crispEdges;
46+
}
47+
48+
.bar rect:hover {
49+
fill: #00c2ff;
50+
}
51+
52+
.timeline {
53+
width: 500px;
54+
}
55+
56+
.histogram {
57+
width: auto;
58+
}

0 commit comments

Comments
 (0)