DIY Zoning: News & Status

DZ is feeling well on JDK 1.5.0_04. The version of the code without 1.5 specific features ran for 12 days with no problems, and now the code that had been enhanced with 1.5 features (starting with generics, but the rest (concurrent, I/O, etc.) will follow) runs just as smooth.

Memory consumption has increased, but it seems that it's mostly due to shared JVM core size increase - DZ DAC runs perfectly even if given as little as -Xms2m -Xmx2m memory.

Those willing to be a guinea pig are welcome to get the JDK_15 branch from the CVS.

A timestamp was added to xAP and xPL producers. The format is ISO 8601 yyyy-MM-dd'T'HH:mm:ss.SSSZ.

A package with a serial and USB controllers that came as a donation from Parallax, Inc. is on my desk. Details here.

Replaced jUSB with javax.usb. This was done for Servomaster back in October 2004, so far, there were no complaints - time to finalize the changes.

Added xAP and xPL home automation protocol support modules. Neither has yet been tested with a live xAP/xPL application - waiting for Chris "madmax" to verify the implementation.

Pololu Corporation has generously donated a 16-Servo USB Controller. Support is coming - keep watching Servomaster project pages.

Fixed some obsolete and invalid links.

Documented changes to Dallas OWAPI.

Received the 1-Wire® USB Adapter sample from Dallas. Support is coming.

<rant>

O JavaSpaces, where art though?

</rant>

As DZ progresses, it ages. Original concept stays, but it becomes overwhelmed with details, most of which don't fit into the arcitecture, and while it is still sound and relatively future-proof, the need to simplify and optimize the architecture becomes more and more nagging.

One good concept is the aforementioned JavaSpaces. Unfortunately, it is a stillborn child because it depends on Sun's Jini technology, which didn't deliver what it promised, and, therefore, is basically dead.

Well, since the JavaSpaces implementation can't be used within the constraints imposed on DZ by its environment, at least the original idea (ComponentBus) can be, and will be.

It will be a while until the new architecture becomes the reality, meanwhile, v.2 notes chapter will accumulate notes to myself and later will be used as a requirements document for the new architecture.

Added a chapter on Fault Tolerance.

0.1p7dev3 release is out. This release introduces support for multiple logical devices per one physical 1-Wire® device in general and for TAI8540A humidity sensor in particular.

Necessarily, DAC2CORE protocol has been changed. Backward compatibility couldn't be maintained, so you'll have to upgrade your configuration - see release notes.

Started working on changing the DAC2CORE protocols to support pressure and humidity sensors.

Added DAC support for multiple logical 1-Wire® devices per single physical device. Such is the case with at least DS2438 which can be used as a temperature container and humidity container.

Added initial support for AAG Electronica TAI8540A humidity sensor.

August had been slow.

Not only a certain event occured, which definitely slowed down everybody concerned, but some other things happened as well.

One of them, quite catastrophic, was a workstation rebuild. RedHat9 had reached end-of-life and had to be replaced, so couple of weeks were spent on selecting a new distribution. Eventually, Mandrake 10 was installed as a decent compromise between being fresh enough, stable enough, and usable enough. RedHat's successor, Fedora Core, doesn't impress much. SuSe 9.1 out-of-the-box lacks development tools, and there's no GNOME, of which I'm fond (for lack of an old good FVWM). RxTx 2.x still dumps core, and I don't know yet if 1.4-15 still works.

Well, "software rot" is a known phenomenon, and it'll take some time to sort out the consequences, most important of which is: jUSB stopped compiling because of the kernel version change (still works, though). Since a USB servo controller is the best supported device at this point, this definitely poses a problem that needs fixing ASAP.

Another thing worth mentioning is that AAG Electronica had generously donated a pressure sensor and a humidity sensor, which allows to start working on optimizing static pressure control.

0.1p7dev2 release is out. This is a development release intended to serve as a test drive for a CORE configuration wizard. See Release Notes section for more information.

0.1p7dev1 release is out. This is a development release intended to serve as a test drive for a DAC configuration wizard. See Release Notes section for more information.

The current CVS trunk contains a working DAC configuration wizard (the GUI still needs to be polished, but it is able to produce a valid configuration). The shell script for it is at ${install_prefix}/bin/dz_dac_wizard.

JRobin was upgraded from 1.2.2 to 1.3.1. Performance increase is about tenfold. However, it is possible that there's a bottleneck in DZ RRD regeneration code somewhere - JRobin authors (who's using DZ raw trace as a benchmark data sample) reports much faster times than I was able to achieve. I believe him.

0.1p6 release is out. See Release Notes section for more information. One item worth mentioning here, though, is: PnP is now operational.

Followup to DS18B20 story: I completely forgot that there are other devices that implement TemperatureContainer. Just in case, all of them were added (for compatibility), even though not all of them were created equal (see notes in Parts List section).

Also, support for DS2408 and DS2405 (obsolete, but oh well, let it be) was added, as well as support for DS2890 digital potentiometer.

TODO: Figure out how do switch containers that aren't DS2405...DS2409 interfere with the DAC operation.

Ouch! Turns out that DS18B20 doesn't work, as opposed to DS18S20 (a.k.a. DS1920). Bug #978381 had been created, and a DS18B20 sample ordered to verify the fix. The root cause is funny: these two, even though the names are similar, belong to different container families.

Absolutely frustrated with attempt to make any version of RxTx compile on any of my boxes, started a replacement driver based on OWFS. Initial draft seems quite a bit slower than the OWAPI based code, mostly due to the fact that the whole directory subtree has to be traversed all the time, even the empty subdirectories, but since there's a high degree of cooperation from the OWFS development team, I guess, there will be improvement. Also, as a side effect, 1-Wire® USB adapter will be available.

Or someone fixes RxTx, which I gave up on.

Added schematics and PCB layout for 1-Wire temperature sensors and DS2890 based servo controller (generously contributed by Nic van der Walt) to the Sensor Network page.

JRobin author, Sasa Markovic, contacted me a while ago with questions regarding JRobin performance (see March 5 notes). Turns out, DZ code has a bottleneck somewhere, so JRobin is not THAT slow, but anyway, he says that my criticizm motivated him to stress test JRobin, and as a result, JRobin is about 20 times faster as of release 1.3.1 (soon to be released).

The new site layout problems are eventually fixed (hopefully), and it is (again, hopefully) now ready for public consumption.

Sic transit gloria mundi, take 2 (site access statistics illustrating "Slashdot Effect"):

Makes interesting comparison with the first coming, two years ago:

JRobin is not just slow, it's SLOW. It takes up to 15 second to create the set of graphs RRDTool creates in about 2 seconds. It takes over 100 seconds to create a standard RRD for 8 sensors, something that is negligible with RRDTool. The RRD regeneration script gave me something of a shock when I realized that rrdupdate manages to get called more than 500 times per second on a trace file containing 567,686 (that's five hundred sixty seven thousand and six hundred eighty six) entries (on Athlon XP 2100+ box with 512MB DDR memory). Sketch of an RRD regeneration process with JRobin shows 2 records per second. I guess re-rendering that big trace file of mine is out of question, since it's going to take over three days.

Oh well, maybe I didn't do something right.

In any case, some creative thinking about queueing the incoming data and synchronizing RRD creation, updates and regeneration will be required to make it work seamlessly (as in "fast enough and so different threads don't stand in each other's way").

Still, JRobin may be an acceptable solution since the RRD database creation and regeneration doesn't happen that often - just be careful when you add a new sensor after your sensor network's been up for a long time.

The verdict stays: both JRobin and RRDTool will be supported, the former for beginners, the latter for advanced users. What a disappointment.

Implemented native Java RRD database driver based on JRobin. Well, my hopes were high, but the reality turned out to be bleaker than the dreams.

Of course, it is very, very convenient not to have a platform dependent tool, which, in addition to that, doesn't come in RPM and has to be installed manually (all right, it's not a problem for me, but what about the end users?).

But, there is a price to be paid.

First of all, JRobin RRD is not binary compatible with RRDTool RRD. Sure, platform independent JRobin RRD is better, but it also means incompatibility with RRDTool. Not good, especially if you keep in mind the next paragraph.

Even though JRobin seems to support more advanced graph generation features, RRDTool has a significant advantage - it is scriptable. You don't have to modify Java code and recompile it every time you want to change the way the graphs look.

The verdict: For now, both RRDTool and JRobin based loggers will be supported. For beginners, seamless JRobin. For advanced users, whatever they choose. They can take care of themselves.

UPnP? Who needs UPnP?

Not so fast...

All right, it seems that UPnP is at least somewhat supported in the HVAC world (at least Trane Tracer, mentioned here, is UPnP compliant and proud of it). However, deeper analysis shows that UPnP has some serious problems, not just implementation, but architectural problems: this article was written by UPnP network architect and SSDP lead author. Not a pretty sight.

On the other hand, there's this immortal phrase coined by one of my friends: "It sucks, but it's the best". Indeed, it seems that at least the UPnP *promise* is exactly what's needed to make the integration seamless. Jini seems to be DOA, it's too heavyweight, and locked in into Java (which I don't appreciate even though Java is the current implementation language), and others (Apple Rendezvous and Salutation, to mention a few) seem weird (don't hasten to judge me, though, it takes time to learn all these things).

All right, so the bottomline (for now) is: it took me 10 minutes to write a simple multicast socket broadcaster/listener pair, and it'll take a week to polish it up, and this is what will be used in the 0.2p0 release. In the future, either the necessity will show itself, or a new protocol will emerge - then, let's talk again.

Uh-oh... Slashdotted again :) 76,826 visits in a day - not bad at all.

What a can of worms I've just opened... I regret not seeing the HVAC V1.0 Standardized DCP specification the day it was released...

Bottomline: I'm not sure it would be reasonable to make DZ support Universal Plug-n-Play specification right away (just because this will take quite a bit of effort, and it is an intimidating task on its own), but this is definitely something to keep in mind.

Usability first, everything else be damned... Working on RRD database autogeneration. From now on, if only rrdtool is found on the system, the RRD database will be created and graphs will be drawn automatically. No more hacking RRD creation and graphing scripts.

(about 20 hours later)

I'll be damned, Eric Raymond didn't waste his time. Since I do have delusions of grandeur (What? You didn't notice?), I can afford to say "great minds work alike"... As I was struggling with RRD autogeneration, he was writing a nice article about clueless people like myself: The Luxury of Ignorance: An Open-Source Horror Story. Well, just go and read it. Meanwhile, I'm working on improving usability.

A little search engine saga...

DZ is #2 on brand new Yahoo search engine, right after Honeywell...

I looked at that, and out of curiosity, decided to do some ego surfing and see where DZ stands with other search engines. Well, here are the ones I tried (in order Google gave them to me, plus the results of "search engine" Google query):

• AltaVista
• Ask Jeeves
• Teoma
• WiseNut
• Gigablast
• Scrub The Web
• TrueSearch
• Search Hippo
• Walhello
• Deepindex
• Wotbox
• Dogpile
• Lycos
• Excite
• AlltheWeb
• WebCrawler

Well, turns out that most of them have DZ on the first page, some of them list it as #2, some don't know about it at all (even with a direct search). It's evident that there are fewer search engines than there are search sites - results on different sites look alarminly identical. Quality of the search is vastly different - some of the sites listed in the Google category aren't really search engines, but retailer portals. However (and this is the reason for the results of this dubious exercise are recorded here), some of the obscure and little-known search engines may present a different point of view at the subject, and they'll give you links that you would never find on Google or Yahoo.

Communication between DAC, CORE and VIEW is now encrypted using SSL.

A rough draft of HOWTO is now available.

Release Notes page was created, with upcoming 0.2p0 changes summary. Navbar has to be changed when the time allows.

Bug #734976 has been fixed.

Bug #866567 has been fixed.

The DAC, which now runs on OWAPI 1.00 with RxTx 1.4.15, seems to be stable - not a single core dump, and uptime was reaching two days.

OWAPI 1.00 migration turned out to be tougher than expected. RxTx 2.x is nowhere near as stable as 1.4.15 was - with 1.4.15, uptime was reaching months, and the reason for the termination was never a JVM core dump, but either a power failure, or normal development restart. With RxTx 2.1, uptime barely reaches a few minutes.

Consequently, RxTx 1.4.15 was reinstated, and OWAPI 1.00 code was backported to using javax.comm - good thing that the only change required (at least on the surface) was to replace the import statement.

To be continued...

Bug #864816 is gone for good, hopefully. Indeed, it required a redesign - all the signal path from the sensor to the thermostat to the zone controller and beyond had to be modified. But the effort was worth it - right now, a partial or complete sensor failure will not cause any undesirable behavior - if a zone sensor dies, the damper for that zone will just stay open, and if all the sensors die, the system is gracefully shut down.

Down the same road, the next target is upgrade to OWAPI 1.00 (hope to fix bug #724534 and suchlike atrocities). Unfortunately, this will cause major headache because in order to do so RxTx has to be upgraded, which in turn requires autoconf/automake upgrade and most probably breaks ServoMaster (which will also require upgrade).

Oh well, it was a good time to catch up with the DalSemi folks and fix their bugs anyway.

Bug hunt for bug #864816 has started. Some redesign will be required to make the solution elegant, so the code is branched out. The branch name is Bug_864816.

The project is being revised and put to life, thanks to Tim Small - the first person to actually make it work. I can't let him down ;)

Unfortunate side effect - turned out that SourceForge changed their server setup in such a way that it is not possible to scp(1) stuff to or from the shell server anymore. Javadocs will not be updated until I figure out a way to bypass this limitation.

All the stuff that was served from freehold.crocodile.org had to be moved to another port. It's available again.

I already miss it. The new house has much better HVAC equipment (two variable speed units, one is Trane and other is high-end Lennox), so theoretically everything should be better. In reality, though, once you've tried the zoning system, there's no way back. My family is complaining again. It's not impossible for me to be eventually allowed to staple the blue CAT-5 cables on the white walls again ;)

Today, the system has been shut down after almost two years of seamless operation. Reason for shutdown: relocation.

DZ is again #2 on Google :)

Someone had recently brought to my attention that DZ takes a second place after Honeywell on Google search for "temperature zoning". It was an ego boost all right, but today I've found that it slipped one step down, after a company I've never heard of, Princeton Air Conditioning, Inc.. Oh well...

Oops... HVAC-Talk is expiring threads... Good I have backups. Updated.

Over 18 months of uninterrupted service... The system still works in passive mode. Turns out that controlling the A/C wasn't that critical, and the system is almost as good without it. Read more...

Access statistics: Sic transit gloria mundi...

This project made Slashdot ;)

Rudimentary TCP client view, able to present the system overview and control the system across the TCP connection, is complete.

Rudimentary TCP server view, sufficient to start working on the TCP client view, is complete.

The GPL core is operational. Release 0.1p0 is available for download.

The GPL onewire_server has replaced the proprietary one. Release 0.1p0 is on its way.

DIY Zoning project started.

Motorized registers are installed, the system is activated in a passive mode.

More sensors installed, now covering all the areas of interest.

The cabling is done, the initial set of sensors is installed. Data acquisition started.