Minor update on moisture sensors

Turns out I was utterly soaking my lawn.   Guess these might have been a good investment.  I installed them, and watched them for a few days, got nothing but 0's.  (0 being "utterly soaked").  I decided to turn the sprinklers off for a bit, and make sure the sensors were working, however, I accidentally left one of the schedules on, a 5 minute run per zone daily.  Here is what happened:

The last watering they would have gotten was around 5am.  You can see the steady march upward as the lawn dried out starting around 14:30.  The blue line is the sprinklers going off. (ignore the smaller blips, those are the drips, the one we care about is the big spike at 5am).  At 5am the sprinklers fire off for 5 minutes each.  You can see how two of the sensors respond to this immediately (1 and 4), but the other two are a bit more slow.  1 and 4 are the outside pair, and 2,3 are the innermost two sensors, closer to the center of the lawn.

The sensors responded within about 25 minutes on those outer pairs, especially #4.  I will be watching this over the next few weeks, and start writing some code to auto-adjust the sprinklers depending on conditions.

According to the Davis sensor manual, "turf does not need to be watered until 25-40cb".  I suspect that there needs to be some kind of hysterisys in there, so maybe let it rise to 60 and then knock it back down to 25?  Mostly, I need to watch the graphs for a bit, so I can determine exactly how much each valve contributes to the moisture, and how much time per valve is required to move the moisture a specific amount.  Once I have this data, I think I can start programming code to make everything automatic.

The benefit of course, should be that the water will automagically self-adjust to deal with the changing of the seasons.  That's the theory at least.

Also, side note, given the way sensor 4 responded so radically, it makes me suspect an underground leak.  Hrmm.

Trenchin' ain't easy

Today I finally got around to installing my moisture sensors in the lawn.  I'm using the one-wire moisture reader from http://www.hobby-boards.com/store/ (who, btw, are sadly going out of business and fire-selling all the inventory, so if you want one, buy it now.

It uses the probes from a Davis moisture meter.  I think they were about $75 each, so a bit pricey!   Sadly, I purchased the setup about 2 years ago, but have kept putting it off, because, well, digging a trench sucks.

However, the trench is done, and the probes are installed.  Luckily, literally 5 minutes before the lawn guys showed up to mow.

A long time ago, I buried a length of direct burial wire right up to the edge of the lawn.  After playing forgetful pirate for a few hours, I located the buried bundle, and pulled it all out.  I needed a way to connect the sensors to this Cat6 that wouldn't end up with water inside the sensor.  I think I came up with a good way, we will find out over time.

The wires, all hooked up.

For each one, I stripped the middle of the wire, hooked the sensor up to the chosen pair, and then jammed it through a hole I drilled in a 3/4" PVC end cap.  I then filled the little chunk of PVC with silicone, let it dry, and then installed the other cap.  Hopefully this will keep the sensor connections dry.  I also did use shrink wrap on them, so it should be fine.

I was also apparently smart enough to bury an extra wire, so I used this to hook up a DS18B20 temperature sensor too.  I did essentially the same thing for that one, except this time,I put a huge glob of silicone all inside the hole, and pushed the sensor through that hole, so it would be close to the outside, without actually touching any water.

Finally, each sensor was buried in the ground in it's final resting place.  I was able to find the manual online for the sensors, and it suggested a 45' angle, 3-5" deep for turf.  So that's what I did.  Rather than digging a full trench, I just used a shovel to make a slit in the ground, and then found some random tool and shoved the wire down into the slit.  Basically, it just needs to be deep enough so a mower doesn't eat it.  They said to install the sensors soaking wet, so right now, I'm not getting any useful data from them.  Well, the ground temp sensor is reading 61F, and it's about 70F out right now, so that's something I guess.

As part of the prep for installing the sensors, you are supposed to soak them for 30 minutes, and let them dry for 12 hours, over and over.  While doing this, I noticed they dry out very slowly.  They were not quite fully dry by morning, maybe half way.  So they probably respond rather slowly.

Also, in what was a bit of a surprise to me, they read in cb, or centibars, which is a unit of pressure.  Basically they tell you how much pressure (like suction on a straw) is required to remove moisture from the surrounding soil, into the sensor.  I guess this simulates roots drawing water in, and how difficult or easy it is.  Bone dry, a sensor reads 199cb.  Soaking wet, 0cb.

Once I have some accumulated data, I'll post pretty graph pics.

New version of gnhast - 0.4

Gnhast 0.4 has been released today, and while it has a bunch of fun new features, it also has one really big one.  A web/tablet interface!

Lets look at the changelog, and then back to the pretty pictures:

0.4 - Release Version
        Added Collectors:
                moncoll
                jsoncgicoll

        Added Tools:
                gnhastweb
                gtk-insteonedit (partially complete)

        Added Commands:
                cfeed
                setalarm
                dumpalarms
                listenalarms
                ping
                imalive
       
        New Features:

        Support for new style insteon HUBs.

        Collectors now have self-health checks, and gnhastd monitors them.  A collector will be flagged as bad if it doesn't send updates, or fails to respond to "pings".  This can help with certain devices that get  wedged up after awhile, or collectors that die off mysteriously.

        moncoll collector is a special collector that interfaces with the collector devices created on the gnhast server, and when a collector is marked bad, it can restart the collector, forcibly if needed.

        cfeed is a new type of feed to monitor switches.  Unlike the usual timed feed, cfeed will notify the listeners immediately upon change.

        Add an alarm subsystem.  This is very basic.  Each alarm has a user-generated uid, a blob of text, and an arbitrary severity number. This can be used by scripts to post generic alarms to other things. Currently nothing uses this, eventually the web interface will be made aware of it.

        gtk-insteonedit is a tool to allow link editing of insteon devices directly.  It is still partially complete.

        gnhastweb is a full fledged web GUI for gnhast.  It allows basic editing of devices, and placing devices into groups.  However, what it is actually designed for, is to give you a tablet/phone based control panel for your house.  Each device is a small "card" on the UI,       and you can interact with them.  They are dynamically updated with the current status, and you can do things like flip switches on and off, adjust dimmers, etc.  It can also interface with your rrd files generated by rrdcoll to display simple graphs of sensor data.  The interface is completely configurable via conf file.  You can even substitute hand-generated html for some, or all of the groups, to do something like use an image map of your house as the navigation.  More documentation on this will need to be provided.

So what kinds of things does gnhast support now?  All kinds of fun stuff.  ad2usb Ademco/Honeywell alarm interfaces, IrrigationCaddy sprinkler controllers, Venstar thermostats, and of course everything it previously supported.  Insteon, one-wire, Brultech, etc.  And now you have a nice interface for a tablet to control it all, that you could mount on your wall.  Hint.  Hint.

Download it here

So, what does it look like when an A/C breaks? The graphs below tell the story.  I'm hoping I can use this data to write a script to detect this condition. I'll add more detail once I know what the actual breakage is.  Maybe it just iced up?  Those spikes are a little odd on the 1hr graph.

This is the AC trying to run, me giving up on it, and turning it to fan only.

Looks like it broke around midnight:

Now you can see below my vacation, returning, AC fires up as normal, and then clunk?

You can see the last working run, and then it starts to diverge

Here you can see vacation mode, and then the failure.

Why the Venstar T5900 is the answer to DIY HA thermostat needs

So, I recently picked up a Venstar T5900.  I've been looking for a good "high-end" thermostat for awhile now, that would work with DIY Home Automation.  I think the Venstar is about the only answer out there.

There is of course the 3M one from RadioThemostat, and yes, thats a pretty good choice.  It has an open API, and a few advanced features.  But it's not very pretty.  Old style LCD.  Meh.

Nest is cute, but they will probably never make an API.  Screw them.

The Venstar is color LCD, touch screen, tons of features, scheduling, has the web-connect interface stuff, easy to install, and best of all, a documented API.  They even did a pretty good job of documenting it.  Now admittedly, the API won't let you fiddle every single bit of the t-stat, but it's good enough for most HA uses.  It lets you fiddle the set-temps, and turn the unit on/off/auto/etc as you please.  It also lets you query the onboard sensors.

So what are the quirks?

1) The SSDP discovery routine is wonky.  It doesn't work to the spec.  Normally you send an SSDP query on the multicast, and get a direct response.  The venstar replies back to the multicast.  This can be worked around by just listening for the NOTIFY messages instead.

2) If you want to shut the unit OFF, you have to turn off the schedule first.  No big deal really.

3) If you send it too many queries too quickly it ignores them.  Just slow down your sends.

Which should you pick?

The T5800 is nice, but the T5900 is the one you should buy.  $10 more, and has a humidity sensor.  Even if you don't have a humidity system to control, for $10 you get one more sensor to read.  Yay data.  You definitely want the skyport wifi bit, without that, no API access for you.  Total cost shipped was about $225.  Took about 30 minutes to install, and 20 minutes to program locally.

The API is a simple JSOM interface for grabbing data, and a simple HTTP POST interface for changing settings.  No big deal to write code for at all. They even supply sample code in a few languages.

I've been playing with the API for the Irrigation Caddy lately, and I've discovered a few things not listed on the API page.

1) If you broadcast "Discovery: Who is out there?" to port 30303 UDP, all the irrigation caddies on the network will respond with hostname and macaddr.

2) Sending a stop=active will only stop the current zone, not the program.  So far the only way I've found to stop a program is to send a full stop then a full start.

3) The json requests are standard GET's, but all the zone fiddling, start/stop stuff is POST.

Just writing them down here for now so others can find them in the future.

gnhast 0.2.3 released.

New release of gnhast-0.2.3 today.  Changelog posted below.   Available at:
http://sourceforge.net/projects/gnhast/files/gnhast-0.2.3.tar.gz/download

0.2.3 - Release 7/9/13
    Added Collectors:
        wupwscoll
        wmr918coll
    Added Handlers:
        nightlight
        switchon
        switchoff
    New features:
        Make gnhastd aware of the unit that data is stored in.  Now a collector tells
gnhast what scale it is feeding data in (ex, Celcius). When requesting data, you can
request it in a different scale, and the server will auto-caclulate.
        Add a wupwscoll collector.  This collector will publish weather data to
weather underground, and pwsweather.com.
        Add wmr918coll collector.  Collects data from oregon scientific weather stations.
Works on wx200's wmr918's, and can also connect to a running wx200d instance and
gather data from that.
        Insteon i2cs code now tested and working fully.
        Added basic handlers for a nightlight routine, and switching devices on and off.
        Add a modify command to gnhastd, so you can edit the config file on the fly by
sending it simple commands.  This allows you to add a handler or tweak names/watermarks
on a running instance.
        Port to Linux.

Insteon I2CS notes.

Turns out, the i2cs protocol is slightly more annoying than I was led to believe.  At first glance of the available docs out there, you would think that the only difference is everything gets sent with an extended command.  That is where you would be wrong. It turns out *some* commands are extended, and some are still std.

I recently purchased a pair of the outdoor modules, which are the new i2cs protocol.  Doing the ALDB stuff, and the initial setup was easy, just send all extended commands.  Send it fast on/off, on/off, (as extended) everything works great.  Send it 0x19, for status, and it just ignores it.  Turns out, the status command has to be sent as std, or it ignores it completely.

Sigh.

Power usage

Yesterday, we had a bit of a scorcher.  117F.  The day before was 110, so a difference of 7 degrees.  Look at the difference in power usage.  It's absolutely nuts.

7 degrees translated into 2kw of power.  That's crazy.  Today will be even hotter, I'm sure it will be even nuttier.

Helpful Links

While writing all this automation code, I found tons of useful information out there.  I'll try to keep this post updated with some of the things I found that we especially useful.

Insteon Command Reference
AD2USB Forums (vista alarm panel interface)
One Wire FS
LibConfuse Documentation
LibEvent Documentation
BrulTech
Bitfield decoder
http://marc.merlins.org/perso/linuxha/
http://homewireless.org/wp/2010/03/insteon-commands/
http://irrigationcaddy.com/blog/
WMR968 Serial Protocol
http://hobby-boards.com/store/

Playing with sensors:

I've been goofing around lately with some one-wire sensors.  I have RJ45 jacks all over the house, and I've been wiring them into the one-wire hub, and getting temperature data on all the rooms.  The problem is, I purchased a bag of 25 raw DS18B20 chips, so I've been soldering the things onto little stub wires, and then crimping them onto rj45's and it takes forever.

Then I had the idea of just crimping the chip right into the RJ45.  It actually fits perfectly.  But I was worried that the data would be messed up, because it would basically be measuring the temperature of the wall, not the room.

So I crimped one stub, one 4" wire, and put them right next to eachother for awhile, and recorded the results:

 

The long wire is much more sensitive.  The way it jumps up and down, is actually the air conditioner going off repeatedly.  The stub seems to smooth the results out more.  It does seem however, that the stub is consistently about .5 degrees higher on average, so I guess that's the effect of the wall holding heat.  Based on these results, unless I need actual detail, these stubs will be fine, and are much easier to make.

This will eventually be the home of my journey through building a home automation system, from scratch.  I am DIY'ing everything, from the installation of all the hardware, right down to writing the software for it from scratch.

This week I've released the initial beta version of gnhast - Garbled's NetBSD Home Automation Scripting Tools.  Download it here.

About gnhast:

A collection of daemons that work together to build an event-based home automation system. The core set of daemons is written in C, however, any event (such as a light being turned on) can be handled by an external script or program. These programs can be written in any language, and the central daemon handles all the intercommunication. It is designed to be easily extensible for new device types and protocols.

The core system is BSD Licensed (3-clause), however scripts or additional data collectors may be written under any acceptable lic.

While the primary development environment for this is NetBSD, it can be used on other UNIX variants, such as Linux, or other BSD's.

gnhast wiki (in progress)

The initial version has support for one-wire sensors (temp, humidity, lux, lightning strike, general counters), Brultech GEM's, and Insteon v2 switches and dimmers.  Support is planned for NuTech AD2USB devices, and a few other things, as I buy and install them in my house. The general plan is to co-develop the system as I physically install devices, so more support will be added as my wallet allows.

Tim