Added a lot of functionality to pull back into the base

.gitignore

@@ -0,0 +1,5 @@
+config.php
+nest-api-master*
+yahoo-api-master*
+.htpasswd
+.htaccess

.gitmodules

@@ -0,0 +1,6 @@
+[submodule "nest-api-master"]
+	path = nest-api-master
+	url = https://github.com/gboudreau/nest-api.git
+[submodule "yahoo-api-master"]
+	path = yahoo-api-master
+	url = https://github.com/mycon/Yahoo-Weather-API-PHP-class.git

README.md

@@ -20,32 +20,44 @@ I also wanted an excuse to play with the [D3](http://d3js.org) (Data-Driven Docu
 * Lower mini-chart is interactive pan-and-zoom of the upper chart
 * Hover over the gray circles to get the exact timestamp and temperature
 
-![nestgraph screenshot](https://github.com/chriseng/nestgraph/raw/master/nestgraph-screenshot.png)
+![nestgraph screenshot](https://github.com/mycon/nestgraph/raw/master/nestgraph-screenshot.png)
 
 ## Dependencies
 
 * LAMP stack
 * Unofficial [nest-api](https://github.com/gboudreau/nest-api) library by Guillaume Boudreau
+* Yahoo Weather API (https://github.com/phront/Yahoo-Weather-API-PHP-class.git yahoo-api-master)
+* D3 Js Library (https://www.d3js.org)
 
 ## Getting Started
 
 Clone this repo into your web root.
 
 ```bash
 cd [your-web-root]
-git clone https://github.com/chriseng/nestgraph.git
+git clone https://github.com/mycon/nestgraph.git
 ```
 
-Grab a copy of nest-api and unzip into the ```nestgraph``` directory you created in the previous step. It should create a subdirectory called ```nest-api-master```.
+Initialize git submodules `nest-api-master` and `yahoo-api-master`:
 
 ```bash
 cd nestgraph
-wget https://github.com/gboudreau/nest-api/archive/master.zip
-unzip master.zip
-rm -f master.zip
+git submodule init
+git submodule update
 ```
-Open ```inc/config.php``` in a text editor and update the ```nest_user``` and ```nest_pass``` variables with your username and password for nest.com.  Update the ```local_tz``` variable to reflect your time zone.
 
+Grab the minified v3 version of d3js from https://d3js.org/d3.v3.min.js and place it into:
+
+```
+[your-web-root]/nestgraph/d3.min.js
+```
+
+Open `inc/config.php` in a text editor and update:
+
+* `nest_user` and `nest_pass` variables with your username and password for nest.com
+* `local_tz` variable to reflect your time zone.
+* `local_woeid` with the code you find for your local address or city from: http://woeid.rosselliot.co.nz/
+
 Run the test script to make sure that the API is able to pull your thermostat data correctly from nest.com.
 
 ```bash
@@ -70,21 +82,25 @@ As root or using a DBA account, run the commands in dbsetup to create the MySQL
 mysql -u root < dbsetup
 ```
 
-Create a cron job to poll the website periodically and update the local database. The thermostat does not phone home on a fixed schedule, but typically it updates in 5 to 30 minute intervals. The script will only insert into the database if there is new data available. Obviously, update the path to ```insert.php``` if it's not in ```/var/www/html/nestgraph```.
+Add your nests to the database.
+
+```bash
+php init_devices.php
+```
+
+Create a cron job to poll the website periodically and update the local database. The thermostat does not phone home on a fixed schedule, but typically it updates in 5 to 30 minute intervals. The script will only insert into the database if there is new data available. Obviously, update the path to ```insert.php``` if it's not in ```/var/www/html/nestgraph```. Additionally, add an entry for the job that polls energy usage and weather data daily.
 
 ```bash
 */5 * * * *     /bin/rm -f /tmp/nest_php_* ; /usr/bin/php /var/www/html/nestgraph/insert.php > /dev/null
+2 9 * * * /bin/rm -f /tmp/nest_php_* ; /usr/bin/php /var/www/html/nestgraph/insertEnergy.php > /dev/null
 ```
 (FYI, the reason we remove the files in ```/tmp``` is because it seems the nest-api library attempts to cache authentication info too aggressively, and after a few days it ends up trying to connect to an AWS server that no longer exists.)
 
-Point web browser to the ```nestgraph``` directory on your webserver!  Admire pretty graphs (actually, they won't be all that pretty until it has collected some data).
+Point web browser to the ```nestgraph/index.php``` directory on your webserver!  Admire pretty graphs (actually, they won't be all that pretty until it has collected some data).
 
 
 ## Known Issues
 
-* Only checks for heating on/off, not cooling (I don't have cooling)
-* Only supports a single Nest thermostat (I only have one)
-* Heating on/off trendline lazily mapped on to the temperature graph
 * Assumes you want temperatures displayed in Fahrenheit
 * Doesn't automatically redraw when you resize the browser window
 * Labels (current/target/heating) don't follow the trend lines when you pan/zoom

collect.php

@@ -3,26 +3,34 @@
 require 'inc/config.php';
 require 'nest-api-master/nest.class.php';
 
-define('USERNAME', $config['nest_user']);
-define('PASSWORD', $config['nest_pass']);
-
 date_default_timezone_set($config['local_tz']);
 
-function get_nest_data() {
-  $nest = new Nest();
-  $info = $nest->getDeviceInfo();
+function get_nest_data($serial_number=null) {
+  global $config;
+  $nest = new Nest($config['nest_user'], $config['nest_pass']);
+  $info = $nest->getDeviceInfo($serial_number);
+
+  if (preg_match("/away/", $info->current_state->mode) || preg_match("/range/", $info->current_state->mode)) {
+      $targetTemp = $info->target->temperature[0];
+      $targetTemp2 = $info->target->temperature[1];
+  }
+  else {
+    $targetTemp = $info->target->temperature;
+  }
+
   $data = array('heating'      => ($info->current_state->heat == 1 ? 1 : 0),
-		'timestamp'    => $info->network->last_connection,
-		'target_temp'  => sprintf("%.02f", (preg_match("/away/", $info->current_state->mode) ? 
-						    $info->target->temperature[0] : $info->target->temperature)),
-		'current_temp' => sprintf("%.02f", $info->current_state->temperature),
-		'humidity'     => $info->current_state->humidity
-		);
+                'cooling'      => ($info->current_state->ac == 1 ? 1 : 0),
+                'fan'          => ($info->current_state->fan == 1 ? 1 : 0),
+                'autoAway'     => ($info->current_state->auto_away == 1 ? 1 : ($info->current_state->auto_away == -1 ? -1 : 0)),
+                'manualAway'   => ($info->current_state->manual_away == 1 ? 1 : 0),
+                'leaf'         => ($info->current_state->leaf == 1 ? 1 : 0),
+                'timestamp'    => $info->network->last_connection,
+                'target_temp'  => sprintf("%.02f", $targetTemp),
+                'target_temp2' => (isset($targetTemp2) ? sprintf("%.02f", $targetTemp2) : null),
+                'current_temp' => sprintf("%.02f", $info->current_state->temperature),
+                'humidity'     => $info->current_state->humidity
+               );
   return $data;
 }
 
-function c_to_f($c) {
-  return ($c * 1.8) + 32;
-}
-
-?>
+?>

collectEnergy.php

@@ -0,0 +1,130 @@
+<?php
+
+require 'inc/config.php';
+require 'nest-api-master/nest.class.php';
+
+
+date_default_timezone_set($config['local_tz']);
+
+function get_nest_data($serial_number=null) {
+  global $config;
+  $nest = new Nest($config['nest_user'], $config['nest_pass']);
+  //$info = $nest->getDeviceInfo($serial_number);
+  $energy = $nest->getEnergyLatest($serial_number);
+  //Change nulls to 0
+  foreach($energy->objects as &$object)
+  {
+    $value = $object->value;
+    foreach ($value->days as &$day)
+    {
+      foreach ($day->events as &$events)
+      {
+        if ( empty($events -> continuation) )
+        {
+          $events -> continuation = 0;
+        }
+        if ( empty($events -> touched_by) )
+        {
+          $events -> touched_by = 0;
+        }
+        if ( empty($events -> touched_when) )
+        {
+          $events -> touched_when = 0;
+        }
+        if ( empty($events -> touched_timezone_offset) )
+        {
+          $events -> touched_timezone_offset = 0;
+        }
+        if ( empty($events -> touched_where) )
+        {
+          $events -> touched_where = 0;
+        }
+
+        if ( empty($events -> heat_temp) )
+        {
+          $events -> heat_temp = 0;
+        }
+        else
+        {
+          $events -> heat_temp = $nest->temperatureInUserScale($events -> heat_temp);
+        }
+
+        if ( empty($events -> cool_temp) )
+        {
+          $events -> cool_temp = 0;
+        }
+        else
+        {
+          $events -> cool_temp = $nest->temperatureInUserScale($events -> cool_temp);
+        }
+
+      }
+      unset($events);
+    }
+    unset($day);
+  }
+  unset($object);
+  return $energy;
+}
+
+function compute_daily_temps(&$db, $time_start, $id)
+{
+    $json = array();
+
+    //$where_stmt = "timestamp BETWEEN \"" . $time_start . "\" AND \"" . $time_end . "\"";
+    $where_stmt = "timestamp BETWEEN " . $time_start . " AND DATE_ADD(" . $time_start . ", INTERVAL 24 HOUR)";
+
+    $sql_query = "SELECT outsideTemperature from data where device_id=? and " . $where_stmt . " order by timestamp";
+    //print $sql_query;
+
+    if ($stmt = $db->res->prepare( $sql_query)) {
+      $stmt->bind_param("i", $id);
+      $stmt->execute();
+      $stmt->bind_result($outsideTemperature);
+      header('Content-type: application/json');
+      $i=0;
+      $temp_min = 1000; //initial value that temp will always be less than
+      $temp_max = -1000;
+      $temp_total = 0;
+      while ($stmt->fetch()) {
+       $temp_total += $outsideTemperature;
+       $temp_min = min($temp_min, $outsideTemperature);
+       $temp_max = max($temp_max, $outsideTemperature);
+       $i++;
+      }
+      $stmt->close();
+      if($i > 0)
+      {
+        $temp_avg = $temp_total/$i;
+        $temp_avg = sprintf("%.02f", $temp_avg);
+        $temp_min = sprintf("%.02f", $temp_min);
+        $temp_max = sprintf("%.02f", $temp_max);
+
+        $json['temperature_avg'] = $temp_avg;
+        $json['temperature_min'] = $temp_min;
+        $json['temperature_max'] = $temp_max;
+
+        //print json_encode($json);
+        $insert_time = str_replace('"', '', $time_start);
+        $insert_time = $insert_time . ' 00:00:00'; //Make a new time string (append all 0s for time after date
+
+        $sql_query = "UPDATE energy_data SET daily_temp_avg=?, daily_temp_min=?, daily_temp_max=? " .
+            "WHERE device_id=? and energyDate=?";
+        //print $sql_query;
+        if ($stmt = $db->res->prepare($sql_query)) 
+        {
+          //print "Time: $insert_time \n";
+          $stmt->bind_param("iiiis", $temp_avg, $temp_min, $temp_max, $id, $insert_time );
+          if (!$stmt->execute()) 
+          {
+            echo "Execute failed: (" . $stmt->errno . ") " . $stmt->error;
+          }
+          $stmt->close();
+        }
+
+        return json_encode($json);
+      }
+    }
+    return "";
+}
+?>

dbsetup

@@ -3,15 +3,77 @@ GRANT ALL PRIVILEGES ON nest.* TO 'nest_admin'@'localhost' IDENTIFIED BY 'choose
 FLUSH PRIVILEGES;
 
 USE nest;
-CREATE TABLE `data` (
+CREATE TABLE `nest`.`devices` (
+   `id` tinyint unsigned NOT NULL AUTO_INCREMENT,
+   `serial` varchar(16) NOT NULL,
+   `name` varchar(256),
+   PRIMARY KEY (`id`),
+   UNIQUE KEY (`serial`)
+)ENGINE=InnoDB DEFAULT CHARSET=latin1;
+
+CREATE TABLE `nest`.`data` (
+  `device_id` tinyint unsigned NOT NULL,
   `timestamp` timestamp NOT NULL,
   `heating` tinyint unsigned NOT NULL,
+  `cooling` tinyint unsigned NOT NULL,
+  `fan` tinyint unsigned NOT NULL,
+  `autoAway` tinyint signed NOT NULL,
+  `manualAway` tinyint unsigned NOT NULL,
+  `leaf` tinyint unsigned NOT NULL,
   `target` numeric(7,3) NOT NULL,
+  `target2` numeric(7,3),
   `current` numeric(7,3) NOT NULL,
   `humidity` tinyint unsigned NOT NULL,
   `updated` timestamp NOT NULL,
-  PRIMARY KEY (`timestamp`),	      
-  UNIQUE KEY `timestamp` (`timestamp`)
-)
-ENGINE=MyISAM DEFAULT CHARSET=latin1;
+  `outsideTemperature` numeric(7,3) NOT NULL,
+  `outsideHumidity` numeric(7,3) NOT NULL,
+  `outsidePressure` numeric(7,3) NOT NULL,
+  PRIMARY KEY (`device_id`,`timestamp`) USING BTREE
+)ENGINE=InnoDB DEFAULT CHARSET=latin1;
+
+CREATE TABLE  `nest`.`cycles_data` (
+  `device_id` tinyint unsigned NOT NULL,
+  `cycleNum` int(10) unsigned NOT NULL,
+  `cycleDate` datetime NOT NULL,
+  `start` int(10) unsigned NOT NULL,
+  `duration` int(10) unsigned NOT NULL,
+  `type` int(10) unsigned NOT NULL,
+  PRIMARY KEY (`device_id`,`cycleNum`,`cycleDate`) USING BTREE
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
+
+CREATE TABLE  `nest`.`energy_data` (
+  `device_id` tinyint unsigned NOT NULL,
+  `energyDate` datetime NOT NULL,
+  `device_timezone_offset` int(11) NOT NULL,
+  `total_heating_time` int(10) unsigned NOT NULL,
+  `total_cooling_time` int(10) unsigned NOT NULL,
+  `total_fan_cooling_time` int(10) unsigned NOT NULL,
+  `total_humidifier_time` int(10) unsigned NOT NULL,
+  `total_dehumidifier_time` int(10) unsigned NOT NULL,
+  `leafs` int(11) NOT NULL,
+  `whodunit` int(11) NOT NULL,
+  `recent_avg_used` int(10) unsigned NOT NULL,
+  `usage_over_avg` int(10) NOT NULL,
+  `daily_temp_avg` numeric(7,3),
+  `daily_temp_min` numeric(7,3),
+  `daily_temp_max` numeric(7,3),
+  PRIMARY KEY (`device_id`,`energyDate`) USING BTREE
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;
 
+CREATE TABLE  `nest`.`events_data` (
+  `device_id` tinyint unsigned NOT NULL,
+  `eventNum` int(10) unsigned NOT NULL,
+  `eventDate` datetime NOT NULL,
+  `start` int(10) unsigned NOT NULL,
+  `end` int(10) unsigned NOT NULL,
+  `type` int(10) unsigned NOT NULL,
+  `touched_by` int(10) unsigned NOT NULL,
+  `touched_when` int(10) unsigned NOT NULL,
+  `touched_timezone_offset` int(11) NOT NULL,
+  `touched_where` int(11) NOT NULL,
+  `heat_temp` int(11) NOT NULL,
+  `cool_temp` int(11) NOT NULL,
+  `continuation` int(11) NOT NULL,
+  `event_touched_by` int(11) NOT NULL,
+  PRIMARY KEY (`device_id`,`eventNum`,`eventDate`) USING BTREE
+) ENGINE=InnoDB DEFAULT CHARSET=latin1;