We have moved at community.getvera.com

Author Topic: Thinking of building Record/Playback for Lighting (Vacation Mode Plugin)  (Read 10905 times)

Offline guessed

  • Community Beta
  • Master Member
  • ******
  • Posts: 5301
  • Karma: +92/-22
  • Release compat is not a bolted-on afterthought
After looking at the Energy Monitor codebase, I think there are enough "bits" to build a Vacation-Mode Plugin, something that records a few days worth of Lighting activity into a Calendar, and then can be set to "playback" these events.

I've done some experiments using curl to PUT Calendar events into Google Calendar using CalDAV, and these are working fine.

Currently I'm experimenting using iCal entries like the following for the "Record" function:

    BEGIN:VCALENDAR
    VERSION:2.0
    PRODID:-//MiOS Vacation Plugin//CalDAV Client//EN
    BEGIN:VEVENT
    UID:20100712T182145Z-123401@example.com
    DTSTAMP:20100613T202145Z
    DTSTART:20100613T170005Z
    DTEND:20100613T170100Z
    SUMMARY:Set Kitchen Main Light to 50%
    DESCRIPTION:A Test Calendar entry to be created by the Vera Vacation Plugin.\n\nBEGIN-MiOS\nluup.call_action("urn:upnp-org:serviceId:Dimming1", "SetLoadLevelTarget", {newLoadlevelTarget="30"}, 45)\n\END-MiOS
    CLASS:PRIVATE
    END:VEVENT
    END:VCALENDAR



and I can't make up my mind whether to use the "SUMMARY" Field, with a human-readable command like:

    Set Kitchen Main Lights to 50% - For Dimmers
    Set Living Picture Light to On - For Switches/Dimmers
    Set Master Bedroom Light to Off - For Switches/Dimmers

which I'll parse out and build the Vera command...

Or something like what I've done in the "DESCRIPTION" Field, which contains a pure-lua snippet between BEGIN-MiOS/END-MiOS Tagging:

    A Test Calendar entry to be created by the Vera Vacation Plugin.\n\nBEGIN-MiOS\nluup.call_action("urn:upnp-org:serviceId:Dimming1", "SetLoadLevelTarget", {newLoadlevelTarget="30"}, 45)\n\END-MiOS


The latter is much more flexible, since you could put just about any Lua/Luup code in there and have it execute (all Security issues aside ;)

The former is much easier for folks to read, and potentially manually create, as long as the Device Descriptions are unique enough.  If folks have non-unique Device Labels, then the syntax would need to be augmented with the DeviceID as in:

    Set Kitchen Main Lights #54 to 50% - For Dimmers
   

Since it's going to chew up a bunch of time to write this, anyone have any opinions on which style they'd prefer to see?

Anyhow, my goal is to let Vera record, and then playback, events from a [remote] Calendar (Google, to start with, Yahoo next, and then any CalDAV server eventually).  Letting it capture Lighting information (initially) so you can drive "away/vacation" schedules, and make your house look really lived in (since it will have followed real activity for a few days, recording all the Lighting "action" going on)

Anyone interested and/or have thoughts on it?

Once I get enough bits, I'll request a code.mios.com space for it and my test files.

Offline guessed

  • Community Beta
  • Master Member
  • ******
  • Posts: 5301
  • Karma: +92/-22
  • Release compat is not a bolted-on afterthought
If anyone wants to do similar experiments, here's the curl command I used to push the previous Calendar data to my GMail account (after enabling Calendaring for my GMail account):

Code: [Select]
curl  --verbose --user "xxxxxxx@gmail.com:yyyyyy" --insecure --basic ^
      --data-binary @test.ics --output test.txt --request PUT ^
      --header "If-None-Match: *" ^
      --header "Content-Type: text/calendar; charset="utf-8"; component=VEVENT" ^
      --url https://www.google.com/calendar/dav/xxxxxxx@gmail.com/events/test1.ics

Where xxxxxxx is your GMail userid, and yyyyyy is your password for that account. 

All tests were run under a Windows version of cURL, in a .bat file, but presumably the same stuff works on Vera also (since it uses cURL for a bunch of stuff)

Offline 325xi

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1101
  • Karma: +0/-0
  • V1, V2, still V2...
Not sure about recording-for-playback, but playback from GCal would be extremely useful - assuming user will be able to type event details into GCal

Have you looked into RememberTheMilk service? IMHO it's tailored for this, and they have rich API too.

Offline guessed

  • Community Beta
  • Master Member
  • ******
  • Posts: 5301
  • Karma: +92/-22
  • Release compat is not a bolted-on afterthought
Right, either way there's a "record" half and a "playback" half of the code, with a shared repository in the middle (CalDAV Calendar).

Folks who don't want to record, could manually type in the Calendar entries, in the right format, and just enable the Playback component.  It's likely that the Playback bit would be built first anyhow, but the experiments above were done to ensure that I could eventually "record", since that will likely be simpler (and more realistic looking) than anything created by hand.  If "manual" is going to be common, then the simple syntax below would be the better "first bet" for implementation.

The CalDAV choice was deliberate.  It's an open standard, which is run in common (free) hosted environments (eg, Google, Yahoo, etc) but can also be run on a locally-hosted server (various vendors) should people want tighter security (folks have already desire for better "off-net" options)

Additionally, it's automatically exposed (editable) on an iPhone/iPad, and other phone brands, as well as via Thunderbird/Lightning, Sunbird, Web Browsers and even some plugins for Outlook.

This combo makes any "scheduling" activity very accessible to all the devices that anyone might consider running on.

Offline mikeholczer

  • Sr. Member
  • ****
  • Posts: 413
  • Karma: +0/-0
I had thought about building this same functionality, but have not have the time to even design it out.

Offline mikeholczer

  • Sr. Member
  • ****
  • Posts: 413
  • Karma: +0/-0
Some of the ideas I had include:
1) Not limiting recording and playback to lighting, but support all actions on all devices. When in record mode the user would pick the devices and their actions that should be recorded.

2) Having playback handled by a virtual device, so one could add multiple playback devices and playback multiple schedules. I was thinking that using script playback might be an easier way to schedule climate and light schedules than using scenes with timers. The would be particularly true if integrated with a CalDAV server, as it would be easier to see a full day/week/etc of scheduled actions.

Offline guessed

  • Community Beta
  • Master Member
  • ******
  • Posts: 5301
  • Karma: +92/-22
  • Release compat is not a bolted-on afterthought
The initial limitation will be lighting, but since I'll check it into code.mios.com, folks will be able to extend it to cover other devices/device types.

My original prioritization was for things needed for "Vacation mode", in this order:

    1. Lighting (Dimmers and Regular Switches)
    2. Scenes (for things like "All off")
    3. Thermostats (since I just installed 2 :)

Folks should be able to add other types once the baseline functionality is in place.

Note, since there's no UI to drive this within Vera, control will be extremely simple.  It'll be a Plugin, which can be instantiated as often as you like (Multiple Scheduler devices), each having the following characteristics:

    An "On/Off" Switch to enable Record Mode
    An "On/Off" Switch to enable Playback Mode
    URL Parameter for CalDAV Account
    Username/Password information for CalDAV Account

any filtering of Recorded data can be done manually by going into your favorite Calendar client and deleting stuff  :-)  You'll also use your native calendar client to "move" any Recorded events to the right date range for Playback.

We also have to assume that both the Clock and TZ are setup correctly in Vera (similar to how the Weather Plugin assumes that Locale/Region is set correctly).

Quote
I was thinking that using script playback might be an easier way to schedule climate and light schedules than using scenes with timers. The would be particularly true if integrated with a CalDAV server, as it would be easier to see a full day/week/etc of scheduled actions.

Exactly, except that there's no notion of Sunset/Sunrise in Calendar, so that Timer-type has no counterpart.

This will, however, start small being focussed mostly on Vacation-oriented stuff, but it should be possible to grow the codebase over time with more scheduling functionality.

I was thinking of having it "check in" with the Calendar about every 15-30 minutes in order to pickup the next 1 hr worth of "work".  This should cutdown on the overheads, but also means that "instant" scheduling wouldn't work (this timing will be a parameter so people can tweak)

The next step is to work out how to get stuff "out" of Calendar, over CalDAV, including recurring events (etc). 

Getting Calendar entries created was relatively easy.... luckily the Mrs is away for a few weeks so I have time to research ;)



JAVIER: Can you create me a code.mios.com space for a "Scheduler" Plugin?  I can use this to at least upload the CalDAV test scripts prior to the Plugin's creation.

guest4690

  • Guest
JAVIER: Can you create me a code.mios.com space for a "Scheduler" Plugin?  I can use this to at least upload the CalDAV test scripts prior to the Plugin's creation.

sure, but 'Scheduler' is a little too generic.  what about 'CalDAV Scheduling' or even 'CalDAV storage' ?

Offline 325xi

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1101
  • Karma: +0/-0
  • V1, V2, still V2...
As a potential extension of the recording feature - human readable log of human-initiated events (lights on/off, thermostat manual changes, sensors) could be useful as part of "nanny surveillance system".  (Video is better, but only if one has a camera in every corner of the house, which is rarely the case.)

Offline Ap15e

  • Beta Testers
  • Hero Member
  • *****
  • Posts: 1998
  • Karma: +12/-0
I would like to suggest decomposing your plugin into two components:

Component 1/Plugin 1 (only one instance allowed): Notification service
Clients (other plugins) subscribe to the notification service and get notified about events (light on/off, run scene, set dimmer, ...).

Component 2/Plugin 2 (multiple instances allowed): CalDAV Scheduling
Uses Component 1/Plugin 1

Offline guessed

  • Community Beta
  • Master Member
  • ******
  • Posts: 5301
  • Karma: +92/-22
  • Release compat is not a bolted-on afterthought
Hey Javier,
Let's shoot for:

     CalDAV Scheduler


BTW: Do you have a set of internal "rules" you go by for plugin naming?  I'd like to avoid Apple-Store style rejections ;)

Offline guessed

  • Community Beta
  • Master Member
  • ******
  • Posts: 5301
  • Karma: +92/-22
  • Release compat is not a bolted-on afterthought
@Ap15e,
No need to split it in two if all you want is notification of standard Device events within Vera.  After looking at this code for a while:

    http://code.mios.com/trac/mios_energry-reporting/browser/energy%20monitor%20plugin/L_EnergyMonitor_j.lua#L138

and then this method implementation:

    http://code.mios.com/trac/mios_energry-reporting/browser/energy%20monitor%20plugin/L_EnergyMonitor_j.lua#L25

it finally dawned on me how luup.variable_watch works.  Basically it's an event hook that lets you subscribe to changes in a named-variable, on a specified DeviceID (more or less).

Anyhow, this makes a much better example than the documentation.



For the CalDAV Scheduler, we'd simply subscribe to the ON/Off or Dimmer events on ALL lighting fixtures, similar to how the Energy Monitor works.  Over time, we'd end up subscribing to various other Variables on other devices so they can be watched also (SetPoints and such for Thermostats etc)


So any Plugin that wants to see changes in standard Vera Devices can also just subscribe itself in a similar manner.  No need to split this out, since it already exists are core Luup functionality.


Now if we ever wanted to make the CalDAV Scheduler do "stuff" on other devices, ones that dont already implement a standard service, all we'd need to do is implement a new Service for the Scheduler to look for using:

    luup.device_supports_service

and then it'll call that.  you could imagine an eventual syntax like:

     Call SqueezeBox Living Room #58 with <arbitraryString>

where this invokes some custom Service, defined by the CalDAV Scheduler, that passes an arbitrary string over a generic Service Action.  The Plugins could do whatever they want with it, but it would be a step above calling a "Scene" since you'd be able to pass a Parameter.

But I'm getting ahead of myself here.

First I have to work out how to use CalDAV REPORT to form a query to extract the next hours worth of data ;)

guest4690

  • Guest
Let's shoot for:

     CalDAV Scheduler

done

BTW: Do you have a set of internal "rules" you go by for plugin naming?  I'd like to avoid Apple-Store style rejections ;)

not really. i have just been bitten too many times by overly broad names.

Offline guessed

  • Community Beta
  • Master Member
  • ******
  • Posts: 5301
  • Karma: +92/-22
  • Release compat is not a bolted-on afterthought
I'd like to create a plugin called "Stuff"    (just kidding)

Offline Ap15e

  • Beta Testers
  • Hero Member
  • *****
  • Posts: 1998
  • Karma: +12/-0
@guessed,

Thanks for pointing out the existence of luup.variable_watch!

With this function you are just a few lines of code away from a notification service:
Just iterate over the devices and subscribe to the relevant variables.

Just for reference: the formal specification is at:
http://wiki.micasaverde.com/index.php/Luup_Lua_extensions#function:_variable_watch
« Last Edit: June 15, 2010, 05:25:21 pm by Ap15e »