HPCNow · LydiaSevelt · Jan 20, 2016 · Jan 20, 2016 · Jan 22, 2016 · Jan 22, 2016
diff --git a/README b/README
diff --git a/README.md b/README.md
@@ -0,0 +1,127 @@
+ABOUT PHPQstat
+==============================================
+**PHPQstat** is a web interface to qstat and other useful commands of the Grid Engine (GE) batch queue system.
+With this interface, you and your users can monitor your jobs and your queue status conveniently via a web browser.
+
+**AUTHORS**  
+UGE support, HTML5 interface, qstat reduce and remote master options added by Lydia Sevelt ([email protected])  
+Originally written by Jordi Blasco Pallarès ([email protected]).
+
+**REPORTING BUGS**  
+Report bugs to GitHUB issue Tracker https://github.com/LydiaSevelt/PHPQstat/issues
+
+**ADDITIONAL LIBRARIES**  
+The HTML5 interface utilizes the excellent datatables (https://datatables.net) and jquery (https://jquery.com) javascript libraries.  
+
+**TESTED WITH**  
+Son of Grid Engine 8.1.9  
+Univa Grid Engine 8.3.1p6  
+
+**LICENSE**  
+This is free software: you are free to change and redistribute it. GNU General Public License version 3.0 (GPLv3).
+
+**Version**  
+0.3.0 (December 2016)
+
+https://github.com/LydiaSevelt/PHPQstat
+
+SCREENSHOTS
+==============================================
+Screenshots were taken from a test instance of Son of Grid Engine 8.1.9
+
+
+Queue Status page with two test queues.
+![Alt text](https://cloud.githubusercontent.com/assets/4594964/21457190/37e6a6fc-c8fb-11e6-8b6c-f1b04b920e5c.jpg "Queue Status")
+
+
+All Jobs on the cluster, both running and in queue, tables are sortable by field, in the screenshot I am using the search feature to filter the running jobs table.
+![Alt text](https://cloud.githubusercontent.com/assets/4594964/21457203/5405e6b8-c8fb-11e6-9039-4af29a50761a.jpg "Job Status")
+
+
+All Jobs on the cluster with multiple parallel environment jobs with the new display scheme that shows all queues grouped with a single job.  
+![Alt text](https://cloud.githubusercontent.com/assets/4594964/22034925/fa44aec8-dcbb-11e6-9054-4e3c53f83569.jpg "Job Status with parallel environment jobs")
+
+
+Job information page displaying some details about a running job.
+![Alt text](https://cloud.githubusercontent.com/assets/4594964/21457210/5d1d7108-c8fb-11e6-8609-79425139d3f2.jpg "Job Info")
+
+
+Hosts Status with only the one desktop as the single host, so not very impressive looking.
+![Alt text](https://cloud.githubusercontent.com/assets/4594964/21457439/53acf240-c8fd-11e6-9c04-31d40a264593.jpg "Hosts Status")
+
+
+DEPENDENCIES
+==============================================
+Basic setup (on sge_master host):  
+apache, php5, rrdtool and awk.
+
+Remote master setup:  
+webserver host: apache, php5, rrdtool, awk, snmp-utils  
+sge_master host: snmpd, awk  
+
+INSTALL
+==============================================
+1. Copy all files in your web accesible filesystem or download the project using GIT:  
+    git clone https://github.com/LydiaSevelt/PHPQstat
+2. Setup the following paths on phpqstat.conf :  
+    SGE_ROOT=/opt/sge  
+    SGE_CELL=default  
+    RRD_ROOT=/var/www/PHPQstat/rrd  
+    WEB_ROOT=/var/www/PHPQstat  
+3. Edit the line: "source /var/www/PHPQstat/phpqstat.conf" to point to the location of your phpqstat.conf  
+   in your web root in the files :  
+    accounting.sh  
+    qinfo.sh  
+4. Setup the following config variables in config.inc.php :  
+    $qstat_reduce="yes";  
+    $cache_time="3";  
+5. If using Univa Grid Engine set the variable in config.inc.php :  
+    $UGE="yes";  
+6. If using Son of Grid Engine and you do *not* already have a /$SGE_ROOT/$SGE_CELL/common/settings.sh file  
+   then copy the sog.8.1.9.settings.sh file to /$SGE_ROOT/$SGE_CELL/common/settings.sh and modify  
+   the variables to match your config :  
+     export SGE_ROOT="/opt/sge"  
+     export SGE_CELL="default"  
+     export SGE_CLUSTER_NAME="p6444"  
+     export DRMAA_LIBRARY_PATH="/opt/sge/lib//libdrmaa.so"  
+7. If using qstat_reduce set LOAD_WAIT variable in phpqstat.conf for high load average protection :  
+    LOAD_WAIT=10.00  
+8. Add the following line to the proper users crontab, making sure you replace /var/www/PHPQstat with the proper path :  
+    */3 * * * * /var/www/PHPQstat/accounting.sh > /dev/null 2>&1
+
+  SETTING UP A REMOTE MASTER CONFIG
+  ----------------------------------------------
+9. Set REMOTE_MASTER in phpqstat.conf to the hostname of sge_master server :  
+    REMOTE_MASTER=sgemaster.company.com  
+10. Configure snmpd on sge_master host to provide uptime and load information via community public :  
+    ```
+    com2sec notConfigUser  default       public  
+    group   notConfigGroup v1           notConfigUser  
+    view    systemview    included   .1.3.6.1.4.1.2021.10  
+    access  notConfigGroup ""      any       noauth    exact  systemview none none  
+    ```
+
+  OPTIONAL
+  ----------------------------------------------
+11. Replace PHPQstat/img/logo.png with the logo of your company/school to brand the page  
+
+TODO LIST
+==============================================
+* Add install script to take care of some of the tedium automatically
+* Add job accounting page and qstat_reduce functionality to allow users to view stats on completed jobs
+* Admin page - config variables, display options, project/department, etc
+* Add additional job information to job page
+* Completely replace rrdtool graphs with something pretty (grafana?)
+
+CHANGELOG
+==============================================
+* 0.1.0 Project started
+* 0.1.1 Install instructions and job details support
+* 0.1.2 Solved problem on cputime request on pending job
+* 0.1.3 Solved problems with Start time and Submission Time
+* 0.2.0 Real-time accounting feature
+* 0.2.1 Migration to HPCNow GitHUB repo
+* 0.2.2 Added qstat_reduce to cache xml files and only refesh based a time interval with high load average protection
+* 0.2.3 Added ability to run phpqstat on a webserver that is a submit host, eliminating the need to run on the sge_master node
+* 0.3.0 Added new HTML5 interface to fix look and feel as well as add functionality for users, This utilizes the excellent datatables and jquery-ui javascript libraries. Added UGE support option.
+* 0.3.1 Parallel environment jobs are now displayed in a single line with all active queues associated. Many other small bug fixes and improvments as well. Install instructions also updated.
diff --git a/about.php b/about.php
@@ -5,14 +5,22 @@
   <meta name="AUTHOR" content="Jordi Blasco Pallares ">
   <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
   <meta name="KEYWORDS" content="gridengine sge sun hpc supercomputing batch queue linux xml qstat qhost jordi blasco solnu">
-  <link rel="stylesheet" href="phpqstat.css" type="text/css" /> 
+  <link rel="stylesheet" type="text/css" href="jquery-ui.min.css"/>
+  <link rel="stylesheet" type="text/css" href="datatables.min.css"/> 
 </head>
 
 <?php
 $owner  = $_GET['owner'];
-echo "<body><table align=center width=95% border=\"1\" cellpadding=\"0\" cellspacing=\"0\"><tbody>";
-echo "<tr><td><h1>PHPQstat</h1></td></tr>
-      <tr><td CLASS=\"bottom\" align=center><a href='index.php'>Home</a> *  <a href=\"qhost.php?owner=$owner\">Hosts status</a> *  <a href=\"qstat.php?owner=$owner\">Queue status</a> * <a href=\"qstat_user.php?owner=$owner\">Jobs status ($owner)</a> * <a href=\"about.php?owner=$owner\">About PHPQstat</a></td></tr>";
+echo "<body><table align=center width=100% border=\"0\" cellpadding=\"0\" cellspacing=\"0\"><tbody>";
+include("header.php");
+echo "<tr><td align=center>
+<a class='ui-button ui-widget ui-corner-all' href=\"index.php\">Home</a> 
+<a class='ui-button ui-widget ui-corner-all' href=\"qhost.php?owner=$owner\">Hosts status</a>
+<a class='ui-button ui-widget ui-corner-all' href=\"qstat.php?owner=$owner\">Queue status</a>
+<a class='ui-button ui-widget ui-corner-all' href=\"qstat_user.php?owner=$owner\">Jobs status ($owner)</a>
+<a class='ui-button ui-widget ui-corner-all' href=\"about.php?owner=$owner\">About PHPQstat</a>
+</td></tr>";
+
 ?>
 
 
@@ -22,12 +30,17 @@
 <center>
 <table align=center width=50% border="0" cellpadding="0" cellspacing="0">
 <tr><td align=center>
-<b>PHPQstat</b> is a web interface that allows to connect to the usefull commands of the Grid Engine (GE) batch queue system. With this interface, you can monitor your job status and your queues health at real time.<br><br>
-<b>AUTHOR</b> Written by Jordi Blasco Pallarès.<br>
-<b>REPORTING BUGS</b> Report bugs to <a href=http://sourceforge.net/tracker/?group_id=306687 target=bugs>SourceForge Tracker</a><br>
-<b>LICENSE</b> This is free software: you are free to change and redistribute it. GNU General Public License version 3.0 (<a href=http://gnu.org/licenses/gpl.html target=gpl>GPLv3</a>).<br>
-<b>Version : 0.2.0 (February 2012)</b><br><br>
-<a href=http://phpqstat.sourceforge.net target=SF>http://phpqstat.sourceforge.net</a><br>
+<b>PHPQstat</b><br>A web interface to qstat and other useful commands of the Grid Engine (GE) batch queue system.<br>
+With this interface, you and your users can monitor your jobs and your queue status conveniently via a web browser.<br><br>
+<b>AUTHORS</b><br>
+HTML5 interface, UGE support and additional features added by Lydia Sevelt<br>
+Originally written by Jordi Blasco Pallarès.<br><br>
+<b>Additional libraries</b><br>
+HTML5 interface utilizes the excellent <a href=https://datatables.net>datatables</a> and <a href=https://jquery.com>jquery</a> javascript libraries.<br><br>
+<b>LICENSE</b><br>
+This is free software: you are free to change and redistribute it. GNU General Public License version 3.0 (<a href=http://gnu.org/licenses/gpl.html target=gpl>GPLv3</a>).<br><br>
+<b>Version : 0.3.0 (December 2016)</b><br>
+<a href=https://github.com/LydiaSevelt/PHPQstat>https://github.com/LydiaSevelt/PHPQstat</a><br>
 </td></tr>
 </table>
 </center>

diff --git a/accounting.sh b/accounting.sh
@@ -6,7 +6,7 @@ source /var/www/PHPQstat/phpqstat.conf
 #########################################
 
 if ! [ -d $RRD_ROOT ]; then mkdir -p $RRD_ROOT; fi
-#QUEUES=$(qconf -sql | cut -d. -f1)
+QUEUES=$(qconf -sql | cut -d. -f1)
 
 # Inici BBDD
 #################
@@ -56,25 +56,28 @@ DATE=$(date '+%a %b %-d %H\:%M\:%S %Z %Y')
 unset datagrups
 i=0 
 for q in $QUEUES; do
- datagrups="$datagrups DEF:${q}-used=$RRD_ROOT/qacct_${q}.rrd:${q}-used:AVERAGE LINE1:${q}-used#${COLOR[${i}]}:${q} "
+ datagrups="$datagrups DEF:${q}-used=$RRD_ROOT/qacct_${q}.rrd:${q}-used:AVERAGE LINE2:${q}-used#${COLOR[${i}]}:${q} "
  datagrups="$datagrups GPRINT:${q}-used:MIN:%12.0lf%s"
  datagrups="$datagrups GPRINT:${q}-used:MAX:%12.0lf%s"
  datagrups="$datagrups GPRINT:${q}-used:AVERAGE:%12.0lf%s\\l"
  i=$((i+1))
 done
 
 # Queue Waiting
- datagrups="$datagrups DEF:slots-qw=$RRD_ROOT/qacct_qw.rrd:slots-qw:AVERAGE LINE1:slots-qw#${COLOR[${i}]}:slots-qw"
+ datagrups="$datagrups DEF:slots-qw=$RRD_ROOT/qacct_qw.rrd:slots-qw:AVERAGE LINE2:slots-qw#${COLOR[${i}]}:slots-qw"
  datagrups="$datagrups GPRINT:slots-qw:MIN:%12.0lf%s"
  datagrups="$datagrups GPRINT:slots-qw:MAX:%12.0lf%s"
  datagrups="$datagrups GPRINT:slots-qw:AVERAGE:%12.0lf%s\\l"
 
-rrdtool graph $WEB_ROOT/img/hour.png -a PNG -s -1hour -t "HPC Accounting (hourly)" -h 200 -w 600 -v "Used CPU's" COMMENT:"                   Min Used" COMMENT:"   Max Used"  COMMENT:"    Avg Used \\l" $datagrups COMMENT:"Last update\: $DATE" > /dev/null
+HEIGHT=200
+WIDTH=600
+
+rrdtool graph $WEB_ROOT/img/hour.png -a PNG -s -1hour -t "HPC Accounting (hourly)" -h  $HEIGHT -w  $WIDTH -v "Used CPU's" COMMENT:"                   Min Used" COMMENT:"   Max Used"  COMMENT:"    Avg Used \\l" $datagrups COMMENT:"Last update\: $DATE" > /dev/null
 
-rrdtool graph $WEB_ROOT/img/day.png -a PNG -s -1day -t "HPC Accounting (daily)" -h 200 -w 600 -v "Used CPU's" COMMENT:"             Min Used" COMMENT:"   Max Used"  COMMENT:"    Avg Used \\l" $datagrups COMMENT:"Last update\: $DATE" > /dev/null
+rrdtool graph $WEB_ROOT/img/day.png -a PNG -s -1day -t "HPC Accounting (daily)" -h $HEIGHT -w $WIDTH -v "Used CPU's" COMMENT:"             Min Used" COMMENT:"   Max Used"  COMMENT:"    Avg Used \\l" $datagrups COMMENT:"Last update\: $DATE" > /dev/null
 
-rrdtool graph $WEB_ROOT/img/week.png -a PNG -s -1week -t "HCP Accounting (Weekly)" -h 200 -w 600 -v "Used CPU's" COMMENT:"             Min Used" COMMENT:"   Max Used"  COMMENT:"    Avg Used \\l" $datagrups COMMENT:"Last update\: $DATE" > /dev/null
+rrdtool graph $WEB_ROOT/img/week.png -a PNG -s -1week -t "HPC Accounting (Weekly)" -h $HEIGHT -w $WIDTH -v "Used CPU's" COMMENT:"             Min Used" COMMENT:"   Max Used"  COMMENT:"    Avg Used \\l" $datagrups COMMENT:"Last update\: $DATE" > /dev/null
 
-rrdtool graph $WEB_ROOT/img/month.png -a PNG -s -1month -t "HPC Accounting (Monthly)" -h 200 -w 600 -v "Used CPU's" COMMENT:"             Min Used" COMMENT:"   Max Used"  COMMENT:"    Avg Used \\l" $datagrups COMMENT:"Last update\: $DATE" > /dev/null
+rrdtool graph $WEB_ROOT/img/month.png -a PNG -s -1month -t "HPC Accounting (Monthly)" -h $HEIGHT -w $WIDTH -v "Used CPU's" COMMENT:"             Min Used" COMMENT:"   Max Used"  COMMENT:"    Avg Used \\l" $datagrups COMMENT:"Last update\: $DATE" > /dev/null
 
-rrdtool graph $WEB_ROOT/img/year.png -a PNG -s -1year -t "HPC Accounting (Yearly)" -h 200 -w 600 -v "Used CPU's" COMMENT:"       Min Used" COMMENT:"   Max Used"  COMMENT:"    Avg Used \\l" $datagrups  COMMENT:"Last update\: $DATE" > /dev/null
+rrdtool graph $WEB_ROOT/img/year.png -a PNG -s -1year -t "HPC Accounting (Yearly)" -h $HEIGHT -w $WIDTH -v "Used CPU's" COMMENT:"       Min Used" COMMENT:"   Max Used"  COMMENT:"    Avg Used \\l" $datagrups  COMMENT:"Last update\: $DATE" > /dev/null
diff --git a/bottom.php b/bottom.php
@@ -1,8 +1,4 @@
 <?php
 echo "<tr> 
-      <td CLASS=\"bottom\"> 
-	<b>Version : 0.2.0 (February 2012)</b><br>
-        <a href=\"http://phpqstat.sourceforge.net\">http://phpqstat.sourceforge.net</a><br>
-      </td> 
     </tr>"; 
-?>
+?>
diff --git a/config.inc.php b/config.inc.php
@@ -0,0 +1,6 @@
+<?php
+// See phpqstat.conf for descriptions of these variable settings.
+$qstat_reduce="yes";
+$cache_time="3";
+$UGE="no"
+?>