Early Access: Google Calendar API v3 Integration

Google shut down Calendar API v2 and v1 today, replaced by v3. v3 is not a drop-in replacement for earlier versions, changes are required.

BEFORE

You specify credentials either as a part of configuration (permanently), or via JMX (after server startup).

NOW

First time, DZ starts an external browser with OAuth callback URL, you choose credentials to use, grant the permission to use them, DZ stores the token (not credentials) until it expires. Afterwards, calendar updater reads the token and proceeds without requiring a browser.

CODE CHANGES

Get the current version from Github. Release will be following shortly.

CONFIGURATION CHANGES

You need to remove credentials from the configuration.

BEHAVIOR CHANGES

The only change you should notice is how to make DZ understand that you want to change the account associated with the calendar. Remove ${HOME}/.dz/calendar/StoredCredential file to make DZ request authentication again.

DZ 3.4 "Scheduler-Barebones" Release Is Out

• Download from SourceForge
• Download from Google Code
• Get source code from Subversion

Changes since 3.3.2

Ability to change thermostat settings based on a schedule. Following settings can be changed:

• Setpoint;
• Voting status;
• Off status;
• Dump priority^*

No user interface is provided, the schedule can be set via configuration file only and cannot be changed during runtime - the focus of this release is to provide a stable and usable feature. Correctness of scheduler operation can for now be verified with data loggers - see configuration guide for details.

And this is how to configure the schedule for now:



The next major release will provide scheduling functionality with Google Calendar being the user interface. Details are described in FAQ: How Schedule Periods Work. Functionality provided by this release will be used as a bootstrap, default or fallback implementation.

---

^* This feature is configurable, but not yet activated

DZ3 Configuration Walkthrough

Beware: this is an experiment, and work in progress (I'm writing the presentation as you're reading it). Just following the "release early, release often" policy to maximize the chance of getting useful feedback in time.



UPDATE: Restriction on minimal font size in your browser will affect the presentation view in the small window - maximizing it will fix the issue.

UPDATE: Permanent location for the configuration guide (this one and subsequent): http://www.homeclimatecontrol.com/configuration-guide

Saturation Limit, Configuration Testing and Other Disasters

This is what happens when you forget to specify the saturation limit. P=-1, I=-0.0000002, D=0, no saturation limit is specified. Oops...

Wonder how many hours it'll take to get back in shape - no big deal, since HVAC is not controlled by DZ at this time, tit's just working as a testbed at this location. But a runaway condition like that on real hardware would have cost you a pretty dime, and possibly damaged hardware.

A saturation limit of, say, 3 would've fixed the situation.

Which brings up a question - how do you test configuration changes without jeopardizing very expensive hardware? The answer is trivial - you clone control elements of the configuration, use elements that havbe been tried and found correct to control the hardware, and use untested elements for monitoring only. Yes, there will be differentces between real behavior and tested behavior later due to the fact that control outputs from elements out of feedback loop (actual hardware control) are ignored, but you will at least get a rough idea of how they will perform.

Configuration Change Required

If you're using a Subversion snapshot of the code and keeping it up to date, you'll have to change the configuration for 1-Wire device factory:

• Was: net.sf.dz3.device.sensor.impl.onewire.DeviceFactory
• Now: net.sf.dz3.device.sensor.impl.onewire.OwapiDeviceFactory

DZ3 Configuration: 1-Wire Sensors

Being too excited about the fact that OWAPI code started working again, I forgot to mention how to configure it for use with not just loggers, but to actually do something useful. Here:

<bean id="device_factory"
     class="net.sf.dz3.device.sensor.impl.onewire.DeviceFactory"
     init-method="start">
<constructor-arg index="0" value="/dev/ttyUSB0"/>    
<constructor-arg index="1" value="regular"/>    
</bean>

<bean id="temperature_sensor-6500000055FF1A26"
factory-bean="device_factory"
factory-method="getTemperatureSensor">
<constructor-arg value="6500000055FF1A26"/>
</bean>

The temperature_sensor-6500000055FF1A26 bean can be used in further configuration as usual.

NOTE: If you're inclined to fiddle with configuration before the GUI configurator is out, you might just as well get familiar with Spring Framework, if you aren't already. A powerful tool. All you need for now is the IoC container configuration.

"Passive Mode": Early Access

As of now, Phidget Quad Servo controller is happily shuffling a servo on my desk, controlled by the code that you can get from Subversion trunk. This is what you have to add to the configuration in order to enable it:

    <bean id="damper_factory" class="net.sf.dz3.device.actuator.servomaster.DamperFactory">
 <constructor-arg index="0" value="net.sf.servomaster.device.impl.usb.phidget.QuadServoController"/>
 <constructor-arg index="1" value="00515"/>    
  </bean>

  <bean id="damper-quad_servo0"
   factory-bean="damper_factory"
   factory-method="getDamper">
   <constructor-arg index="0" value="USB Bus Powered Servo"/>
   <constructor-arg index="1" value="0"/>
  </bean>

Other servo controllers will be supported as well. Please answer the poll questions (on your right) to let me know which drivers I have to take care of first.

As of right now, DZ3 is completely capable of supporting the passive mode. I'm not going to release the code just yet, until all prerequisites for Barebones are satisfied.

DZ3: JMX Instrumentation and Control

If you update the DZ runner shell script to its current form (just add -Dcom.sun.management.jmxremote to Java command line), you can use JConsole to monitor and control DZ3. This is what it looks like (click to enlarge):



And this is how to make it happen (see previous examples for context):

<!-- JMX configuration -->
<bean id="jmx-wrapper" class="net.sf.jukebox.jmx.JmxWrapper">
  <constructor-arg index="0" type="java.util.Set">
    <set>
      <ref bean="temperature_sensor-6EE055000000"/>
      <ref bean="temperature_sensor-cpu1"/>
      <ref bean="temperature_sensor-cpu2"/>
      <ref bean="temperature_sensor-mobo"/>
      <ref bean="temperature_sensor-sda"/>
      <ref bean="temperature_sensor-sdb"/>
      
      <ref bean="rrdlogger_sensors"/>
      <ref bean="rrdlogger_thermostats"/>
      <ref bean="rrdlogger_dampers"/>
    </set>
  </constructor-arg>
</bean>

Not all entities are JMX enabled at this time. Adding one to the configuration won't do any harm, but you'll get a warning message in the log.

It's worth mentioning that you need to add DZ3 classes to JConsole's classpath in order to get complex entities like DataSample<Double> rendered by JConsole, like this:

${JAVA_HOME}/bin/jconsole \
-J-Djava.class.path=${JAVA_HOME}/lib/jconsole.jar:\
${JAVA_HOME}/lib/tools.jar:$CLASSPATH

where CLASSPATH is the same that is used in DZ3 Runner.

The recommended value of I...

...is 0.0000002.

Don't ask.

You'll know what it is about when you get to it.

DZ3 Data Logger: Splitters

Some entities (thermostat, PID controller that is a part of it, zone controller to name a few) produce signal that is more complicated than DataSample<Double>, which is just about as complex as DataLogger can accept.

In order to break down complex signals to simple components, splitters exist. Here's how you can configure one (irrelevant beans omitted):

<bean id="thermostat-6EE055000000" class="net.sf.dz3.device.model.impl.ThermostatModel">
   <constructor-arg type="java.lang.String" value="1-Wire 6EE055000000"/>
   <constructor-arg type="net.sf.dz3.device.sensor.TemperatureSensor" ref="temperature_sensor-6EE055000000"/>
   <constructor-arg type="net.sf.dz3.controller.pid.AbstractPidController" ref="pid_controller_6EE055000000"/>
</bean>
<bean id="splitter-thermostat-6EE055000000" class="net.sf.dz3.device.model.impl.ThermostatSignalSplitter">
   <constructor-arg index="0" ref="thermostat-6EE055000000"/>
</bean>
<bean id="rrdlogger_thermostats" class="net.sf.jukebox.datastream.logger.impl.rrd.RrdLogger" init-method="start">
   <constructor-arg index="0" type="java.util.Set">
       <set>
           <ref bean="splitter-thermostat-6EE055000000"/>
       </set>
   </constructor-arg>
   <constructor-arg index="1" type="java.io.File" ref="rrdbase_thermostats"/>
   <constructor-arg index="2" type="java.io.File" ref="rrdtool"/>
</bean>

The above example will make the rrdlogger_thermostats data logger automatically subscrube to all signals that are produced by thermostat-6EE055000000 and create a channel for each of them (in this particular case, they will be named "1-Wire 6EE055000000" and "1-Wire 6EE055000000.calling").

IMPORTANT: Make sure you don't create the controller you're trying to monitor with scope="prototype" - you will end up with a new instance, not connected to anything and not doing anything till the end of time. Goes without saying that you can not reuse controller instances for different entities - you will have to create diferent instances with diffent bean IDs.

Splitters for other complex entities will be written as need arises - keep your eye on the code base. One good way to do that is to subscibe to commits RSS feed.

DZ3 Data Logger: Logging *Everything*

During DZ2 development it became clear tat there's no such thing as too much instrumentation data. Hence, DZ3 was architected in such a way that any object that produces DataSample<E extends Number> can have their data logged.

Here's an example how you can log temperature sensor data (this has been added to the bottom of this example):

<!-- Loggers -->
<bean id="rrdbase" class="java.io.File">
       <!-- This is the path in the file system. If in doubt, make it absolute --/>
       <constructor-arg type="java.lang.String" value="./rrd"/>
</bean>
<bean id="rrdtool" class="java.io.File">
       <constructor-arg type="java.lang.String" value="/usr/bin/rrdtool"/>
</bean>
<bean id="rrdlogger" class="net.sf.jukebox.datastream.logger.impl.rrd.RrdLogger" init-method="start">
       <constructor-arg index="0" type="java.util.Set">
               <set>
                       <ref bean="temperature_sensor-6EE055000000"/>
                       <ref bean="temperature_sensor-cpu1"/>
                       <ref bean="temperature_sensor-cpu2"/>
                       <ref bean="temperature_sensor-mobo"/>
                       <ref bean="temperature_sensor-sda"/>
                       <ref bean="temperature_sensor-sdb"/>
               </set>
       </constructor-arg>
       <constructor-arg index="1" type="java.io.File" ref="rrdbase"/>
       <constructor-arg index="2" type="java.io.File" ref="rrdtool"/>
</bean>

And here's the data that it produced during its first 3 hours of operation:



UPDATE (2009/11/08): Added comment to rrdbean constructor argument - thanks to David for clarification.

DZ3 Configuration: Teaser

DZ3 is about to start working as a forced ventilation controller for a computer located in a closed, but ventilated compartment under a staircase. Picture on the left is of the computer itself (fan assembly is visible in the background). Picture on the right is of the fan assembly that provides ventilation (by the way, not loud at all).



And below is the configuration that reads three lm_sensors supported sensors and one 1-Wire sensor (mapped to the file system by owfs-fs) and feeds the data to the rest of the system. If you want to see it, copy & paste somewhere else - there's no way to make it readable here.

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://www.springframework.org/schema/beans
           http://www.springframework.org/schema/beans/spring-beans-2.5.xsd">

    <!-- Sensors -->
    <bean id="temperature_sensor-6EE055000000" class="net.sf.dz3.device.sensor.impl.ShellSensor" init-method="start">
        <constructor-arg index="0" value="6EE055000000"/>
        <constructor-arg index="1" value="1000"/>
        <constructor-arg index="2" value="/bin/cat /mnt/onewire/28.6EE055000000/temperature12|tr -s ' '"/>
    </bean>
    <bean id="temperature_sensor-cpu1" class="net.sf.dz3.device.sensor.impl.ShellSensor" init-method="start">
        <constructor-arg index="0" value="cpu1"/>
        <constructor-arg index="1" value="1000"/>
        <constructor-arg index="2" value="/usr/bin/sensors|grep Int|tr -s ' '|cut -f 3 -d ' '|tr -d '+°C'"/>
    </bean>
    <bean id="temperature_sensor-cpu2" class="net.sf.dz3.device.sensor.impl.ShellSensor" init-method="start">
        <constructor-arg index="0" value="cpu2"/>
        <constructor-arg index="1" value="1000"/>
        <constructor-arg index="2" value="/usr/bin/sensors|grep AMD|tr -s ' '|cut -f 3 -d ' '|tr -d '+°C'"/>
    </bean>
    <bean id="temperature_sensor-mobo" class="net.sf.dz3.device.sensor.impl.ShellSensor" init-method="start">
        <constructor-arg index="0" value="mobo"/>
        <constructor-arg index="1" value="1000"/>
        <constructor-arg index="2" value="/usr/bin/sensors|grep M/B|tr -s ' '|cut -f 3 -d ' '|tr -d '+°C'"/>
    </bean>

    <!-- Thermostats -->
    <bean id="pid_controller_6EE055000000" class="net.sf.dz3.controller.pid.SimplePidController" scope="prototype">
        <constructor-arg index="0" value="22.5"/>
        <constructor-arg index="1" value="1"/>
        <constructor-arg index="2" value="0"/>
        <constructor-arg index="3" value="0"/>
        <constructor-arg index="4" value="0"/>
    </bean>
    <bean id="thermostat-6EE055000000" class="net.sf.dz3.device.model.impl.ThermostatModel">
        <constructor-arg type="java.lang.String" value="1-Wire 6EE055000000"/>
        <constructor-arg type="net.sf.dz3.device.sensor.TemperatureSensor" ref="temperature_sensor-6EE055000000"/>
        <constructor-arg type="net.sf.dz3.controller.pid.AbstractPidController" ref="pid_controller_6EE055000000"/>
    </bean>

    <bean id="pid_controller_cpu1" class="net.sf.dz3.controller.pid.SimplePidController" scope="prototype">
        <constructor-arg index="0" value="56"/>
        <constructor-arg index="1" value="1"/>
        <constructor-arg index="2" value="0"/>
        <constructor-arg index="3" value="0"/>
        <constructor-arg index="4" value="0"/>
    </bean>
    <bean id="thermostat-cpu1" class="net.sf.dz3.device.model.impl.ThermostatModel">
        <constructor-arg type="java.lang.String" value="CPU1"/>
        <constructor-arg type="net.sf.dz3.device.sensor.TemperatureSensor" ref="temperature_sensor-cpu1"/>
        <constructor-arg type="net.sf.dz3.controller.pid.AbstractPidController" ref="pid_controller_cpu1"/>
    </bean>

    <bean id="pid_controller_cpu2" class="net.sf.dz3.controller.pid.SimplePidController" scope="prototype">
        <constructor-arg index="0" value="44"/>
        <constructor-arg index="1" value="1"/>
        <constructor-arg index="2" value="0"/>
        <constructor-arg index="3" value="0"/>
        <constructor-arg index="4" value="0"/>
    </bean>
    <bean id="thermostat-cpu2" class="net.sf.dz3.device.model.impl.ThermostatModel">
        <constructor-arg type="java.lang.String" value="CPU2"/>
        <constructor-arg type="net.sf.dz3.device.sensor.TemperatureSensor" ref="temperature_sensor-cpu2"/>
        <constructor-arg type="net.sf.dz3.controller.pid.AbstractPidController" ref="pid_controller_cpu2"/>
    </bean>

    <bean id="pid_controller_mobo" class="net.sf.dz3.controller.pid.SimplePidController" scope="prototype">
        <constructor-arg index="0" value="38"/>
        <constructor-arg index="1" value="1"/>
        <constructor-arg index="2" value="0"/>
        <constructor-arg index="3" value="0"/>
        <constructor-arg index="4" value="0"/>
    </bean>
    <bean id="thermostat-mobo" class="net.sf.dz3.device.model.impl.ThermostatModel">
        <constructor-arg type="java.lang.String" value="Motherboard"/>
        <constructor-arg type="net.sf.dz3.device.sensor.TemperatureSensor" ref="temperature_sensor-mobo"/>
        <constructor-arg type="net.sf.dz3.controller.pid.AbstractPidController" ref="pid_controller_mobo"/>
    </bean>

    <!-- Dampers: None for now -->

    <!-- Zone Controllers -->
    <bean id="zone_controller-mx" class="net.sf.dz3.device.model.impl.SimpleZoneController">
        <constructor-arg type="java.util.Set">
            <set>
                <ref bean="thermostat-6EE055000000"/>
                <ref bean="thermostat-cpu1"/>
                <ref bean="thermostat-cpu2"/>
                <ref bean="thermostat-mobo"/>
            </set>
        </constructor-arg>
    </bean>

    <!-- HVAC Units -->
    <bean id="unit-mx" class="net.sf.dz3.device.model.impl.UnitModel">
        <constructor-arg type="java.lang.String" value="mx"/>
        <constructor-arg type="net.sf.dz3.device.model.ZoneController" ref="zone_controller-mx"/>
    </bean>

    <!-- Damper Controllers: None for now -->

</beans>

DZ3 Configuration, Part 1

This is a simplified representation of how things are connected in DZ3:



Every entity can act as a signal producer, and as a signal consumer. Exceptions are temperature sensors that can only be producers, and dampers that can only be consumers (unless there's instrumentation feedback).

Temperature Sensor

• Consumes nothing
• Produces Temperature

Thermostat

• Consumes Temperature
• Produces Zone Demand
• Has 1:1 relation with the Damper
  

Zone Controller

• Consumes Zone Demand
• Produces Total Demand

HVAC Unit

• Consumes Total Demand
• Produces Running

Damper Controller

• Consumes Zone Demand, Running
• Produces Damper Position
• Has to be aware about Thermostat:Damper relations
  

Damper

• Consumes Damper Position
• Produces nothing
• Has a 1:1 relation with the Thermostat
  

Data Logger

• Consumes everything

HOW TO PUT IT ALL TOGETHER

At this moment, the DZ3 container is based upon Spring Framework. If you are familiar with Spring, then a quick look at the sample configuration will tell you everything you need to know (provided you're somewhat familiar with the the code base).

If not, stay tuned.

UPDATE: Configuration Guide is now available.