Friday, January 27, 2012

Competition: Nest: Let Me Try To Help

Like I was saying earlier, Nine out of ten people trying to change the temperature on the webapp and mobile app after being exposed to the actual hardware are observed making circular motions with their index finger and having a puzzled face. It was more difficult on the webapp than it was on the mobile app, because the latter has visible arrows (which some people managed to miss) whereas the former doesn't.

Nest webappNest mobile app

So I wonder, why didn't Nest designers implement this feature and where their usability expert was looking? I'd call this a major goof for a company that paid so much attention to slick hardware design. Or maybe they just underestimate the number of users (of which just Android user base, according to Nest page at Android Market, is between 1000 and 5000 - and I bet that this is a small fraction of users for it looks like the device is targeted at iOS user base)? Or they just don't care?

Well, let me try to help.

You'd think that tracking a circular gesture on a square screen would be difficult? Not at all.
  1. Take the touch coordinate. Convert it to polar coordinate system against the center of your thermostat image and work in it thereafter.
  2. Take the drag coordinate. Keep the thermostat pointer directed at the drag point - it is easy because after polar conversion you have the angle, hence, the setpoint. You don't care much about the radius.
  3. Take the release coordinate. Send it back to hardware.
Voila.

Now, I wonder, how much time it'll take them to implement it? Or they declare the clickety-click the new standard for UI usability?

Competition: Nest: Three Weeks Later

Disclaimer

  • Of course I'm fiercely biased, this is competition. However, observations below are based on objective and reproducible measurements. And you've been warned.
  • Some observations here will duplicate those listed in Black Box Analysis, however, those were initial impressions and these are long term annoyances.
  • "Apple hater" comments will be marked as spam.

The Good

  • Yes, it looks good on the wall.
  • Yes, the ring feels good.
  • Yes, it does recognize when you come close and turn on.
  • Yes, you can control the thermostat via webapp and iOS application.
  • There seems to be no problems with WiFi connectivity (Update 01/29: I jinxed it. It lost the connectivity earlier today and stayed offline for over three hours. Had to manually reconnect, don't know if it would do it automatically) (Update 02/08: I was told that this issue was related to Nest service outage. Nest support had an article with explanation at bit.ly/z0siLQ, but the article is now gone - though it is referred to in many Nest tweets. The hearsay description I was given was a bit bizarre - "Nest service outage caused thermostats to disconnect from WiFi, and reconnect attempts drained the battery". Check their tweets around this time, you'll find many references to it).
  • Nest updates its software automatically.
  • It works.

The Bad

  • [dup] Temperature displayed is the setpoint temperature, not the actual temperature. You can have a vague idea about what the actual temperature is by looking at it, but there's no indication when it will call for heat or cool - which causes people to jerk it unnecessarily - and the end up overcompensating most of the time (yes, even 0.5°C too hot is too hot).
  • [dup] Both setpoint and actual temperature display resolution is 1°F and 0.5°C, this is not sufficient for some conditions.
  • [dup] There doesn't seem to be a way to change the hysteresis from the default. This will cause short cycling for smaller rooms, and stale air for bigger.
  • [dup] Neither web app nor mobile app honor the °C measurement unit for ambient temperature - three weeks later and counting.
  • It is not possible to see whether the thermostat is in heating or cooling mode - there are several months of up to 20°C (40°F) daily temperature swings, it would be useful to learn how to tune the thermostat to minimize energy use.
  • Nine out of ten people trying to change the temperature on the webapp and mobile app after being exposed to the actual hardware are observed making circular motions with their index finger and having a puzzled face. Read more....
  • Nest webapp  doesn't let the browser remember the password. On purpose. Folks, please let *me* decide whether my thermostat password is *that* important and remove that "automcomplete=off".
  • Webapp freezes in "Nest is standing by" state if left alone for long enough.
  • Nest updates its software automatically with no notification to the user whatsoever (see the changelog). I'd rather have it at least notify me about the fact that it is about to update, so I don't get stuck trying to find the root cause.

The Ugly

  • Schedule is not practical to edit on the thermostat itself, if I were them, I'd just skip it on the device altogether - but that would be losing a bragging point, so they had to do it.
  • Schedule is painful to edit even on the webapp and iOS app - of course it is, if all you have is one button. Clickety-click. No gesture support. Just for fun, calculate the number of clicks it'll take to change 7 schedule points two degrees up, half degree at a time (no, I'm not doing it for you).
  • You can't specify "away" as a schedule item. You'll have to manually specify a low or high temperature, but then if you have to change the away temperature for some reason, you'll have to comb through all the schedule again. One click at a time.
  • There is no "hold" functionality. You are at the mercy of the schedule. Yes, you can specify a set "away" temperature and set it to "away" state, but that's an ugly hack.
  • And this hack won't work on an Android webapp.
  • Speaking of which, its functionality is severely limited compared to webapp an iOS app. The only thing you can do is to bring the thermostat out of "away" mode (but not put it back) and change the setpoint (see update at the bottom).
  • (see the above about WiFi) It is a bit scary when the infrastructure fault can bring down individual devices. Let's just hope that there won't be a catch-22 situation when a fault knocks devices off the infrastructure and down, and the only way to get it back is a firmware update (a.k.a. bricking).

It Is Not That Learning

The main selling point of the thermostat (so important, it is the part of the name) is grossly overstated.
  • The schedule it learned has little to do with what is actually happening.
  • The "auto-away" feature takes anywhere FROM FOUR TO SEVEN EIGHT HOURS to activate. Given the fact that I'm spending on average 10-12 hours and away for the rest, that is 30-70% waste as opposed to 4-5% waste in case when auto-away was kicking in after 30 minutes.

Verdict

It's just a fancy thermostat.

Is it worth $250 plus shipping? Not to me. However, if following conditions were met, it would make think again:
  • Setpoint and temperature display resolution at 0.1°C/°F. Or, at least, configurable resolution.
  • Indication of how soon the thermostat will call.
  • Indication of whether it is heating or cooling.
  • Configurable auto-away timeout.
  • "Away" option in the schedule.
  • "Hold" feature.
  • Working Android application (see update at the bottom).
  • Same circular gesture changing the setpoint on the webapp and mobile app, instead of clickety-click.
  • Gesture support in schedule editing.
  • Added bonus, open APIs that will allow Nest interoperability with other systems

Afterword

All the features I want can be implemented either in firmware or software. Whereas I'm not holding my breath for it, this article will be updated if/when these features arrive.

UPDATE: +sid informed me that it is not that Android application is inferior to iOS, it's that both of them (when running on the phone) are inferior to their tablet counterparts (can't say anything about Android tablet, however, haven't seen it), but, in portrait orientation. To get to full functionality, you need to switch the phone orientation to landscape - then all the functionality becomes available. My apologies to Nest folks, I simply never turn my phone sideways unless I have to type. And there's no indication whatsoever that anything beyond what is visible on the screen is available.

This, however, highlights the fact that mobile applications didn't get the attention they deserved. And the webapp still doesn't look right on small screens.

More comments from +sid (verbatim) with my smart aleck comments in italic:
  • Nest pretends to follow Apple in packaging, but they lack attention to details - uneven space between the box and box cover, color paper over the box does not cover the box from edge to edge. You will see that Nest is far from Apple in packaging if you unpacked products from both companies.
  • Learning finished within two days (on weekend) and the only thing I got in schedule - 68 °F on Friday night - so much for learning
  • It is not obvious that if you want to have more settings on iPhone then you need to turn your phone into landscape
  • I have no idea how long it takes to get into "Auto-Away" mode, but three-four hours after I left home I see Nest with Auto-Away
  • Regular Away mode does not get terminated when I walk back home - I have to manually turn Away off
  • I am not 100% certain but I get a feeling it learns and estimates how long it will take to heat the house to set temperature
  • I am really enjoying accessing my thermostat on the go :) - that's because you aren't using the DZ app :)
  • °F/°C is not consistent - when you switch to °C some other stuff still shown in °F (like outside temp in iOS app)
  • I had no issues with getting visual indications on Nest calling for heat (it turns orange) - I didn't say I didn't see when Nest was calling, I said that it doesn't show any indication of how soon it will. Subtle difference.
  • You need to get used to their way to show current and set temp.
UPDATE (2012/02/14): Lack of "hold" feature and mismatch between "learned" and actual schedule made me remove the schedule altogether. So now it is not a learning thermostat, is not a programmable thermostat, just good old Round with a remote control. Or, rather, I tried. It is not possible to remove the schedule completely, it just restores all of it. Too smart for its own good.

DZ 3.6.5 Release Is Out

Changes since 3.6.4

The most visible change is the Swing console color scheme change from Swing (light) to Android (dark). Also, after a long wait the space "intentionally left blank" has shrunk, and the console now displays the temperature trail for the last several hours. Not only that, the chart line color varies depending on system state, and it is now trivial to visually identify trouble spots. More about that later.

Sick Room

The most important change, however, is the HTTP connector v.2 which allows a DZ installation to be controlled from anywhere in the world via webapp and Android application.

Thursday, January 5, 2012

Competition: Nest Learning Thermostat: Black Box Analysis

Since Nest is the only competing product you don't have to take a loan in the bank to own (well, maybe just a small loan), it was interesting to get a first hand experience with it. So, I bought it and installed next to one of DZ sensors.

I am reluctant to talk to my competition because the amount of information they can extract out of my questions significantly exceeds the amount of information they can extract of my most honest answers to their most detailed questions. --GT
Well, the experience was definitely interesting.

The Excellent

The most impressive (from usability standpoint) thing about the installation process is how they implemented pairing the device with the online account. I created the account (took a split second) and several seconds later, when I turned  to face the thermostat (less than a meter away), it was already displaying the message asking whether I want to associate it with the account I just created. Under the wraps, it is pretty simple to someone in the trade (they probably matched the WAN IP to serial number and account name to the IP it was created from), but nevertheless very impressive. Definitely karma bonus for creativity.

The Good

  • No-brainer installation for someone with hands growing out of shoulders.
  • It was impressive that it correctly determined wiring.
  • It was really simple to configure it to work with the existing system (those who know how many variations of thermostat wiring and systems are out there will be definitely impressed by this).
  • Upon connecting to WiFi, the device immediately downloaded the update.
  • There's just a few seconds latency between the device and the web app.
  • Control algorithm seems to be smarter than just a simple hysteresis loop, will see how smart later.

The Neutral

  • Temperature displayed is the setpoint temperature, not the actual temperature.
  • Both setpoint and actual temperature display resolution is 1°F and 0.5°C, this is not sufficient for some conditions (good example: the room with little air getting to it, while the other room next to the HVAC unit gets more than it bargained for). Making this happen, though, will destroy the "Apple" aesthetics.
  • Not sure how successful will be WiFi connection when a DHCP server is not present. Too lazy to try, hope they did something smart.
  • Initially, I thought that the device was blind as a bat and doesn't sense presence right, but then realized that it's just doing it in its own way.

The Bad

  • It was quite painful to enter a 21 character passphrase for the wireless network.
  • It took two reboots and about 10 minutes to download and install the update.
  • There is no indication of how close the thermostat is to calling - itchy fingers *will* try to bump the thermostat if they feel too hot or too cold.
  • Settings change sensitivity is, unexpectedly, extremely high compared to setpoint change sensitivity - the latter is less than 1:1, and the former is almost trigger fast. Uncomfortable, and is a violation of the principle of least astonishment.
  • There doesn't seem to be a way to change the hysteresis from the default. This will cause short cycling for smaller rooms, and stale air for bigger.
  • The maximum "away" temperature setting for cooling mode is just 90°F. Folks, this is not enough for hot climates - I *work* comfortably at 90°F, and won't be afraid to set the away temperature to 100°F on the second floor.
  • Actual temperature is not easily visible over the "grill" (and for °C, good luck trying to distinguish "22.5" and "22°").
  • Web app and mobile app don't honor the °C measurement unit for ambient temperature. While we're at that, why don't they display other data available from Weather Underground (which is their data source)? (The weather is displayed as a background picture)
  • Standard 18 AWG thermostat wire doesn't fit into the base, had to use thinner wire. I had a patch panel for wiring so it wasn't a problem, but good luck doing this if all you have is the cable sticking out of the wall.

The Ugly

  • Fingerprints all over the glass.
  • Guess what two users tried to do (independently from each other) when asked to change the setpoint on iPad app. Correct, they tried to replicate the circular movement that it takes to change the temperature on the actual device. Instead, they were presented with ancient up and down arrows. Come on, folks, *I* did better than that, gesture detection is not that complicated.
  • Entering several spaces in the passphrase was most excruciating. Hint to Nest developers: it is possible to use all that nice space between the alphabet and done/cancel/backspace symbols that you already have, and plant some spacebars right there. Yes, in every shift mode. Won't hurt. Took quite a while to figure out that those spaces are currently just for decoration and aren't functional.
  • And it was completely infuriating when after spending about two minutes of entering the passphrase the thermostat happily reported "failed to connect" (it was a transient failure) and... displayed the EMPTY prompt again.
  • Let me just say this... Don't even try to use their webapp on a mobile phone. Their "beta" is not Google's "beta". More like "alpha". At $250 plus shipping a pop.
  • Where is my Android app? Right here: Nest Mobile. Pretty basic, though, nothing to write home about.

The Verdict

Short version: Meh.

Long version:
  • Hardware looks to be decent (though, just like I suspected, there's not much stainless steel, it's mostly plastic).
  • Embedded software (on the thermostat itself) is perfect as far as usability is concerned, but passable as far as climate control is concerned (no comments about learning ability, that'll be covered later).
  • Web front end is definitely in alpha stage. Let's hope that they are planning to advance it, or at least make it work right on a wider variety of devices.

Applicability

All the information above is pertinent to model Diamond-1.10, display software ver. 1.0.4, base ver. 3.5.2/1.1, as of January 5 2012.