This Is What We Are Here For

To reinforce the point made in the previous post - we are here to fix it.

The whole point of this project (among other useful things like actually controlling your hardware) is to make everything visible.

You will find out details about the hidden life of your house and your HVAC equipment you never suspected of. We'll be here to help you make sense of the overwhelming stream of data you have no idea of the meaning of, and don't care much about - but the things you do care about (home comfort and energy savings) will be delivered, in a tangible and verifiable form.

Power Can Violate Your Privacy, Too

Remember I've been telling you not to publish your temperature graphs on Internet for years? All right, with DZ you have a choice to publish or not to publish, but not so with your power company.


Here's the article: Experts: Smart grid poses privacy risks

Well, here's my take on it: if you wanna get, you wanna give. Want to save a penny, gotta let them know what you're doing. Privacy is dead - regrettable, but a fact. You're back to village life, where everyone knows everything about everyone. Just on a new level, that's all.

<via /.>

DZ3 Instrumentation Release is out

• Download from SourceForge
• Download from Google Code
• Get source code from Subversion

Don't forget to read release notes.

DZ3 Instrumentation Release: Tagged in Subversion

Source is ready for consumption, available from Subversion repository with the root being https://diy-zoning.svn.sourceforge.net/svnroot/diy-zoning/tags/3.0-INSTRUMENTATION.

Packed archive will be available for download shortly.

Please check the release notes.

UPDATE: released.

DZ3 Instrumentation Release: What's In, What's Not

Any large working system has inevitably evolved from a small working system.

-- Source lost in time

WHAT'S IN

• 1-Wire Sensors. Temperature and humidity for now;
• Shell Sensor. If you can extract information from your hardware via shell script, this is what you use to make it available to DZ;
• Thermostat;
• Zone Controller;
• HVAC Unit abstraction;
• Damper Controller for bang-bang dampers;
• Damper Controller for modulating dampers;
• Complete data logging for every measurement point in between, including hysteresis and PID controllers inside of thermostat;
• JMX instrumentation (quite comprehensive) and control (limited for now).

WHAT'S NOT

Unfortunately, some essential stuff -

• Servo controller integration;
• HVAC Unit driver;
• GUI;
• Scheduler

And some not so essential -

• DAC & Core talking over TCP;
• Obviously, sensor discovery over LAN;
• xAP, xPL loggers (try, they most probably work, but I haven't checked).

Reason for this is that I wanted to clean up issues accumulated over years, and make sure that the system is completely transparent, can be monitored, and bugs can be seen visually (and most of them are). So the net result of this effort, so far, is a working stable system with no actual outputs (other than data loggers). You can watch it, but you can't touch it.

SO HOW IS IT USEFUL?

Well, if you've never had DZ installed because it was too complex to install and configure, the day of Instrumentation Release is your lucky day. What is already done is a turnkey acceptance of all the sensor network with no configuration beyond 1-Wire adapter port name necessary. Hang the sensors on the wall, start DZ3 and watch your house's thermal behavior in gory details.

If you do already have some variant of DZ installed and connected to actuators, then hold off for a short while, the good stuff is coming.

WHAT'S NEXT?

Exactly what is missing from this release:

• Servo controller integration;
• HVAC Unit driver;
• JMX instrumentation allowing to actually control the thermostat setpoints.

GUI and scheduler ports are still on a back burner for now, the priority is a stable and working system, not a nice looking system.

UPDATE: released.

DZ3 Instrumentation Release: Early Access

1-Wire sensor access works in the Subversion code, go for trunk/dz3-master directory tree.

Here's a simple configuration file to have a turnkey logger for all of your existing 1-Wire sensor network, with JMX instrumentation to boot:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans
          http://www.springframework.org/schema/beans/spring-beans-2.5.xsd">
         
   <bean id="device_factory" class="net.sf.dz3.device.sensor.impl.onewire.DeviceFactory" init-method="start">
 <constructor-arg index="0" value="/dev/ttyUSB0"/>    
 <constructor-arg index="1" value="regular"/>    
   </bean>

<!-- Loggers -->
<bean id="rrdtool" class="java.io.File">
 <constructor-arg type="java.lang.String" value="/usr/bin/rrdtool"/>
</bean>
<bean id="rrdbase_onewire" class="java.io.File">
 <constructor-arg type="java.lang.String" value="./rrd-onewire"/>
</bean>
<bean id="rrdlogger_onewire" class="net.sf.jukebox.datastream.logger.impl.rrd.RrdLogger" init-method="start">
 <constructor-arg index="0" type="java.util.Set">
  <set>
   <ref bean="device_factory"/>
  </set>
 </constructor-arg>
 <constructor-arg index="1" type="java.io.File" ref="rrdbase_onewire"/>
 <constructor-arg index="2" type="java.io.File" ref="rrdtool"/>
</bean>

<!-- JMX configuration -->
<bean id="jmx-wrapper" class="net.sf.jukebox.jmx.JmxWrapper">
 <constructor-arg index="0" type="java.util.Set">
  <set>
   <ref bean="device_factory"/>
  </set>
 </constructor-arg>
</bean>
</beans>

Just watch the paths, that's all.

Instrumentation Release itself will be finalized either late tonight, or tomorrow. The difference between now and then is that the path from DeviceFactory to TemperatureSensor is currently broken. The rest of the system (if configured) will work, except for the actual physical control - actuators haven't been ported yet.

UPDATE: release source is tagged in Subversion. The path in question has been created, obviously.

DZ3: JMX Instrumentation and Control

If you update the DZ runner shell script to its current form (just add -Dcom.sun.management.jmxremote to Java command line), you can use JConsole to monitor and control DZ3. This is what it looks like (click to enlarge):



And this is how to make it happen (see previous examples for context):

<!-- JMX configuration -->
<bean id="jmx-wrapper" class="net.sf.jukebox.jmx.JmxWrapper">
  <constructor-arg index="0" type="java.util.Set">
    <set>
      <ref bean="temperature_sensor-6EE055000000"/>
      <ref bean="temperature_sensor-cpu1"/>
      <ref bean="temperature_sensor-cpu2"/>
      <ref bean="temperature_sensor-mobo"/>
      <ref bean="temperature_sensor-sda"/>
      <ref bean="temperature_sensor-sdb"/>
      
      <ref bean="rrdlogger_sensors"/>
      <ref bean="rrdlogger_thermostats"/>
      <ref bean="rrdlogger_dampers"/>
    </set>
  </constructor-arg>
</bean>

Not all entities are JMX enabled at this time. Adding one to the configuration won't do any harm, but you'll get a warning message in the log.

It's worth mentioning that you need to add DZ3 classes to JConsole's classpath in order to get complex entities like DataSample<Double> rendered by JConsole, like this:

${JAVA_HOME}/bin/jconsole \
-J-Djava.class.path=${JAVA_HOME}/lib/jconsole.jar:\
${JAVA_HOME}/lib/tools.jar:$CLASSPATH

where CLASSPATH is the same that is used in DZ3 Runner.

DZ3 Data Logger: Logging *Everything*

During DZ2 development it became clear tat there's no such thing as too much instrumentation data. Hence, DZ3 was architected in such a way that any object that produces DataSample<E extends Number> can have their data logged.

Here's an example how you can log temperature sensor data (this has been added to the bottom of this example):

<!-- Loggers -->
<bean id="rrdbase" class="java.io.File">
       <!-- This is the path in the file system. If in doubt, make it absolute --/>
       <constructor-arg type="java.lang.String" value="./rrd"/>
</bean>
<bean id="rrdtool" class="java.io.File">
       <constructor-arg type="java.lang.String" value="/usr/bin/rrdtool"/>
</bean>
<bean id="rrdlogger" class="net.sf.jukebox.datastream.logger.impl.rrd.RrdLogger" init-method="start">
       <constructor-arg index="0" type="java.util.Set">
               <set>
                       <ref bean="temperature_sensor-6EE055000000"/>
                       <ref bean="temperature_sensor-cpu1"/>
                       <ref bean="temperature_sensor-cpu2"/>
                       <ref bean="temperature_sensor-mobo"/>
                       <ref bean="temperature_sensor-sda"/>
                       <ref bean="temperature_sensor-sdb"/>
               </set>
       </constructor-arg>
       <constructor-arg index="1" type="java.io.File" ref="rrdbase"/>
       <constructor-arg index="2" type="java.io.File" ref="rrdtool"/>
</bean>

And here's the data that it produced during its first 3 hours of operation:



UPDATE (2009/11/08): Added comment to rrdbean constructor argument - thanks to David for clarification.

Measure, So You Can Improve - Take Two

Remember this?



Well, Google hasn't been sleeping on the job, and they now have the first batch of partners. See if your energy company is one of them.

Measure, So You Can Improve

"What you cannot measure, you cannot improve".

Until recently, it was practically impossible to measure energy consumption. Kill-A-Watt was one of the first practical tools available. Still, it was pretty limited for it couldn't be used to continuously log the energy consumption (and it is being addressed), and it couldn't be used to measure the energy consumption for built-in or high consumption devices such as washers, dryers and HVACs.

Well, if you are concerned about your house energy consumption, it seems that your time is coming - Google is busy working on it. They've just announced the PowerMeter.


It will take some time. Currently, Google is building partnerships, and I'm sure we'll see tangible results soon.

UPDATE (2011/6/24): Google Power Meter is no more. Too bad.

Backward incompatible change: DZ will become JDK 1.6+ only

Jukebox, the platform DZ is built on top of, is currently undergoing a code review and enhancement process. One side effect of advanced JMX uplift is that the new Jukebox will be incompatible with JDK 1.5.

Whereas JMX is probably *the* cause to upgrade from JDK 1.5 to 1.6 (latter is the first version where it is really usable), the source code compatibility of JMX features is broken.

The improvement is so significant that I'm willing to restrict DZ to 1.6 as soon as new Jukebox is pulled in.

Speak up if you care.