We have moved at community.getvera.com

Author Topic: Guidance for nest plugin?  (Read 26920 times)

Offline watou

  • Moderator
  • Hero Member
  • *****
  • Posts: 889
  • Karma: +44/-12
Guidance for nest plugin?
« on: November 29, 2012, 02:21:26 pm »
Hi, I've given myself the project of making a plugin for the nest thermostat.  I've got Lua code working that gets all the info I need from nest.com, and can set target temperature, set home/away, set fan mode, etc.

But the learning curve for Vera plugin development is wrecking my head today, and I am looking for pointers.

So...  are there any examples out there of a plugin that implements HVAC_System, containing one or more HVAC_ZoneThermostats?  To properly factor the output from nest.com, the "home" or "structure" is a parent to one or more thermostat devices.  The user would set "home" or "away" on the HVAC_System device (which also implements HouseStatus), but would control the temperature, fan, etc. on the individual thermostat(s).

I've been digging through RTCOA_Wifi thermostat source as an example, but it's complicated enough to make it very hard to follow and is not factored the way I want.  Ideally there would be a much "clearer" thermostat plugin implementation (or even just an empty shell of one) to follow that let me do these things:

- on plugin initialization, fetch data from nest.com and traverse existing HVAC_System devices or create newly discovered ones.
  - within each HVAC_System parent, traverse existing HVAC_ZoneThermostats or create newly discovered ones.

- periodically re-poll nest.com, compare new data with old values for all of the above devices, and set variables accordingly (or preferably, if I could block for minutes at a time on an HTTPS request, use nest.com's polling interface for "instant" updates).

- of course implement all the set* functions.

The only user data needed by the plugin would be the username and password that you use to login to nest.com.  It would not make sense to store it with any of the above devices, since there are potentially multiple HVAC_Systems and HVAC_ZoneThermostats.

In short, I need to find a simpler path for learning how to build this plugin.  I know there are a few very seasoned plugin developers here who could probably make short work of this, but I very much want to 1) learn how to write this myself using the best practices others have gleaned, and 2) not make users wait for weeks or months until I slog through ill-suited examples and trip over all the same pitfalls.

Any guidance is appreciated.  Thanks!

Offline garrettwp

  • Master Member
  • *******
  • Posts: 6371
  • Karma: +227/-128
  • Vera 3, Lite, ISY994
Re: Guidance for nest plugin?
« Reply #1 on: November 29, 2012, 02:30:48 pm »
watou,

You may want to look at the simpler plugins first to see how they do things. This will help you with the basic concepts of plugin development. You can have a look at a few plugins of mine from apps.mios.com (wolplusping and webpowerswitch). If you log into apps.mios.com, you can view the plugin files and see how they tick.

Best of luck and if you have any trouble many of us on the forum are here to help.

- Garrett

Offline futzle

  • Beta Testers
  • Master Member
  • *****
  • Posts: 3260
  • Karma: +192/-9
Re: Guidance for nest plugin?
« Reply #2 on: November 29, 2012, 04:07:58 pm »
Hi watou,

For examples of parent/child device programming, you could look at the security system plugins, which have all got Zones (and most of them have Partitions too).  Also the energy monitoring plugins may do this (CurrentCost certainly does).  Some of the plugins try to do auto-discovery of child devices at startup, and some of the plugins do manual discovery while the plugin is running, and store the child device list in variables in preparation for the next Luup restart.  Which you do depends on how reliable child detection is; if it's possible that the list of children might get truncated through bad communication then initialization at startup is bad and users will see their devices periodically vanish as they failed to get created at Luup startup.  My gut feeling on the Nest plugin is that you should do manual discovery, because you're relying on the Internet.

I believe that there's only parents and children in the MiOS device model, no grandchildren, so my advice would be to use one top-level parent device corresponding to the Nest user account (store the username and password here) and then have flat sets of child devices—zone thermostats, HVAC systems, whatever else Nest exposes through its API.  This doesn't map to the logical hierarchy of thermostats being "in" systems, but you don't gain anything by representing the true hierarchy that way to MiOS.  This is a bit like how the security system devices have separate lists of partition and zone child devices: the zones aren't "in" partitions from Vera's perspective.  Users won't notice this; it's just an implementation detail.

To break the development down into parts, I'd recommend starting with getting the top-level "account" device working (this will be a brand new device type), having it do the discovery calls to Nest's servers and log what you get.  Then create the child devices (auto or manual discovery as discussed above).  Only then worry about populating the devices with actual data.  At this stage you'll have a read-only mirror of a user's Nest devices on the Vera dashboard.  Then you can start programming in interactivity and handling actions so that you can feed information back to the Nest servers when the user presses a button on the dashboard.

Offline watou

  • Moderator
  • Hero Member
  • *****
  • Posts: 889
  • Karma: +44/-12
Re: Guidance for nest plugin?
« Reply #3 on: November 29, 2012, 04:32:02 pm »
garretwp and futzle, thank you very much for your clear advice.  I think that, between you both, you have given me a much firmer footing on building a clean and robust plugin.  I feel like I can dive back in now without getting overwhelmed.  I will certainly have more questions after a bit.  Regards, watou

Offline garrettwp

  • Master Member
  • *******
  • Posts: 6371
  • Karma: +227/-128
  • Vera 3, Lite, ISY994
Re: Guidance for nest plugin?
« Reply #4 on: November 29, 2012, 05:15:33 pm »
Please keep us posted. I very well might pick up a nest. I was going write a plugin, but glad someone else decided to give it a go.

- Garrett


Offline watou

  • Moderator
  • Hero Member
  • *****
  • Posts: 889
  • Karma: +44/-12
Re: Guidance for nest plugin?
« Reply #5 on: December 01, 2012, 04:09:13 pm »
Hopefully quick question: My device icon doesn't change when I change a variable, even though my .json file looks to be accurate.  Is there some other requirement besides the "DisplayStatus," imgIconMin/Max and state_icons entries?  Does my .json also need a sceneList entry?  I'm stumped as to what I'm missing here. 

Thanks in advance.  I'm diligently climbing this learning curve, and it's all seeming simpler the higher I go....

Offline guessed

  • Community Beta
  • Master Member
  • ******
  • Posts: 5301
  • Karma: +92/-22
  • Release compat is not a bolted-on afterthought
Re: Guidance for nest plugin?
« Reply #6 on: December 01, 2012, 04:30:35 pm »
You may need to post the [JSON] file, and the snippet of code that changes the variable, for folks to see what you've got setup.  It'll be hard to work it out otherwise.

That said, you need a data binding also, so it knows which variable to use to change the icon.  The variable needs to be a number, it cannot take domain (string) values.

See this line:
    http://code.mios.com/trac/mios_dscalarmpanel/browser/trunk/D_DSCAlarmPartition2.json#L14

Offline watou

  • Moderator
  • Hero Member
  • *****
  • Posts: 889
  • Karma: +44/-12
Re: Guidance for nest plugin?
« Reply #7 on: December 01, 2012, 06:53:59 pm »
Thanks!  I figured it out -- I was putting full URLs in the state_icons array in the JSON file, not just the bare file names.  Everything was working when I copied from D_BinaryLight1.json, but only just noticed the lack of the relative "icons/" path in the state_icons array.  So problem solved.  But...

Is it possible to package the icons with the plugin, so the use of custom icons doesn't require reaching over the Internet for them?  It's a small thing, but really....

Offline guessed

  • Community Beta
  • Master Member
  • ******
  • Posts: 5301
  • Karma: +92/-22
  • Release compat is not a bolted-on afterthought
Re: Guidance for nest plugin?
« Reply #8 on: December 01, 2012, 07:04:27 pm »
Is it possible to package the icons with the plugin, so the use of custom icons doesn't require reaching over the Internet for them?  It's a small thing, but really....
Not without a hack, no.  MiOS has very poor package/plugin mgmt (none, basically)

Offline RichardTSchaefer

  • Community Beta
  • Master Member
  • ******
  • Posts: 10091
  • Karma: +764/-143
Re: Guidance for nest plugin?
« Reply #9 on: December 01, 2012, 07:10:35 pm »
Yes you can package these with the plugin.
In the plugin page there is a place to put the file during install.
It talks about this at the bottom of the Plugin Guide:
http://wiki.micasaverde.com/index.php/Apps.mios_Developer%27s_Guide#Publish

I also UNCHECK the commpress option.
The problem is that you have to do this every time you create a version!
Otheriwise the icons end up in the /etc/cmh-ludl area.

Offline guessed

  • Community Beta
  • Master Member
  • ******
  • Posts: 5301
  • Karma: +92/-22
  • Release compat is not a bolted-on afterthought
Re: Guidance for nest plugin?
« Reply #10 on: December 01, 2012, 07:33:42 pm »
Yes, but then you go UI5 only, and your filenames must be 100% unique (even for images)...

What's fundamentally missing from the system is the ability to package up everything into a unit (like an WAR file) and not have it impact anything else, and for it to know that the image sections need not be compressed (etc), and for it to place the contents in all the correct locations.

Given how clunky the apps.mios.com tool is, and that this mechanism forces you to use it during the development process (or to understand how to get the images there manually yourself), I consider it a hack (one invented by the MiOS team 8) )...

Offline RichardTSchaefer

  • Community Beta
  • Master Member
  • ******
  • Posts: 10091
  • Karma: +764/-143
Re: Guidance for nest plugin?
« Reply #11 on: December 01, 2012, 07:56:10 pm »
I totally agree!
I also agree it would be nice for each plugin to have the option to be in it's own dir so there are no filename claches.
But we need a common area for shared stuff !

There are lots of land mines to navigate with the apps website ...
I over wrote one file because I clicked the wrong upload file (and of course there is no indication that this happened ... because the selector does not indicate the file you will replace with the upload!)


Offline watou

  • Moderator
  • Hero Member
  • *****
  • Posts: 889
  • Karma: +44/-12
Re: Guidance for nest plugin?
« Reply #12 on: December 03, 2012, 04:41:03 pm »
Hi, I'm trying to use the dkjson module in my plugin but it's giving me fits.  Testing with a lua interpreter at the command line is no problem, but I'm getting a mess when trying to require("L_nest_dkjson") in my local testing on the Vera.  What is the voodoo incantation needed to get this module to load properly for local testing on my Vera?  The RTCOA_Wifi plugin uses it but goes through some wild decompression step that I think applies to a deployed plugin, not my local testing?  Thanks in advance for any guidance.  Far too much black magic in this plugin development business. :P


Offline watou

  • Moderator
  • Hero Member
  • *****
  • Posts: 889
  • Karma: +44/-12
Re: Guidance for nest plugin?
« Reply #13 on: December 03, 2012, 05:04:28 pm »
Answering my own question here, just to say I belatedly found another thread on the subject.  Sorry for the noise...

Offline watou

  • Moderator
  • Hero Member
  • *****
  • Posts: 889
  • Karma: +44/-12
Re: Guidance for nest plugin?
« Reply #14 on: December 04, 2012, 01:54:22 pm »
I'm inching along in progress on the Nest plugin, but I've hit an issue with child devices.  I initially tried to create unfound child devices during the startup function, but that went into a loop for some reason so I moved the discovery code to a function run from luup.call_timer.  In any case, now I have device #33 that is found for update during the poll, but I can't see the device in the UI.  I can enumerate all devices to the log with this:
Code: [Select]
for k, v in pairs(luup.devices) do
    for k2, v2 in pairs(v) do
        luup.log("Device #" .. k .. ":" .. k2 .. "=" .. tostring(v2))
    end
end
And I see the device there.  It is not marked invisible/embedded/hidden, but it's not anywhere in the UI.  Do I have a corrupt table of devices?  Why is this child device invisible from the UI?  I saw it briefly during the looping mess, but it disappeared after the luup engine restarted itself.  What gives?