We have moved at community.getvera.com

Author Topic: Context and thread for executing actions  (Read 951 times)

Offline rigpapa

  • Beta Testers
  • Hero Member
  • *****
  • Posts: 1121
  • Karma: +187/-3
Re: Context and thread for executing actions
« Reply #15 on: January 20, 2019, 04:55:25 pm »
My next try might be to co-ordinate the processing in Vera.  If I use <run> to receive the info then call_delay to schedule the processing of same, I can ensure that I space out the processing.  This won't eliminate concurrent event calls but might minimise their effect.  Like jumping through hoops!!!

I do something like this in many of my plugins, so much so that it's become one of my design patterns. I have a "task queue", which is a table of objects containing a time to next run and a function to be called. The plugin has a single "call_delay" thread that only calls the "task scheduler". When called, it rips through the task queue and builds a list of ready tasks, then runs that ready list by calling the task-specific callback function for each. It then finds the next earliest waiting task, and schedules the next call_delay accordingly. When the tasks run, they can call to reschedule themselves for another future run on whatever schedule they need. When tasks "register", they can include not just the callback function to use, but an array of arguments of any valid Lua type, so I can pass in a lot more than just a single string to my time-delayed tasks (a limitation in call_delay). Tasks can also be cancelled or rescheduled externally (i.e. an action implementation can call for the periodic update task to run immediately). It's been a very helpful wrapper on Luup's call_delay.
Author of Reactor, DelayLight, SiteSensor, Rachio, Deus Ex Machina II, Intesis WMP Gateway, Auto Virtual Thermostat and VirtualSensor plugins. Vera Plus w/100+ Z-wave devices. Vera3, Lite. Hassio, Slapdash.

Offline rafale77

  • Community Beta
  • Hero Member
  • ******
  • Posts: 1749
  • Karma: +101/-27
  • HA ≠ IoT as a blue sky is cloudless.
Re: Context and thread for executing actions
« Reply #16 on: January 20, 2019, 04:59:47 pm »
This is an awesome improvement (the wrapper) on the luup engine which should really be integrated in the vera firmware. Maybe AK can use it in OpenLuup too?
openLuup (79 devices, 141 scenes, 19 apps) master to VeraPlus (142 zwave nodes, 8 Zigbee nodes, 221 devices,  20 scenes , 2 apps) +  Hubitat (15 Zigbee nodes) + Home-Assistant (API Integrations). Bridged to Siri and Alexa. Homewave. VeraPlus ExtRooted and mios server independent.

Offline rigpapa

  • Beta Testers
  • Hero Member
  • *****
  • Posts: 1121
  • Karma: +187/-3
Re: Context and thread for executing actions
« Reply #17 on: January 20, 2019, 05:25:43 pm »
This is an awesome improvement (the wrapper) on the luup engine which should really be integrated in the vera firmware. Maybe AK can use it in OpenLuup too?

openLuup does a good job of scheduling tasks. Without trying to speak for akbooer, I think he needs his scheduler to work the Vera way because that's what's compatible. Nothing he does prevents my approach from working on openLuup, and mine is just one way of getting things done.

I could publish it as a module or encapsulated-function to embed inline; it's only about 160 lines of loosely-spaced code (including trace/debug statements that could be removed).
Author of Reactor, DelayLight, SiteSensor, Rachio, Deus Ex Machina II, Intesis WMP Gateway, Auto Virtual Thermostat and VirtualSensor plugins. Vera Plus w/100+ Z-wave devices. Vera3, Lite. Hassio, Slapdash.

Offline akbooer

  • Beta Testers
  • Master Member
  • *****
  • Posts: 6387
  • Karma: +291/-70
  • "Less is more"
Re: Context and thread for executing actions
« Reply #18 on: January 20, 2019, 05:38:35 pm »
I do something like this in many of my plugins, so much so that it's become one of my design patterns. I have a "task queue", which is a table of objects containing a time to next run and a function to be called. The plugin has a single "call_delay" thread that only calls the "task scheduler". When called, it rips through the task queue and builds a list of ready tasks, then runs that ready list by calling the task-specific callback function for each. It then finds the next earliest waiting task, and schedules the next call_delay accordingly. When the tasks run, they can call to reschedule themselves for another future run on whatever schedule they need. When tasks "register", they can include not just the callback function to use, but an array of arguments of any valid Lua type, so I can pass in a lot more than just a single string to my time-delayed tasks (a limitation in call_delay). Tasks can also be cancelled or rescheduled externally (i.e. an action implementation can call for the periodic update task to run immediately). It's been a very helpful wrapper on Luup's call_delay.

This is exactly how the openLuup scheduler works, and, strangely (or perhaps not) I had done the same thing previously with a startup code library that I used ubiquitously on my Veras.

This is an awesome improvement (the wrapper) on the luup engine which should really be integrated in the vera firmware. Maybe AK can use it in OpenLuup too?

As per the above, I think I already do?  It may just be a cosmetic tweak to the packaging which you're after.

@akbooer has helped many, many people along the way.  Don't forget to give him Karma.

Very much appreciated!  Thanks @GeekGoneOld (and Karma.)
3x Vera Lite-UI5/Edge-UI7, 25x Fibaro, 23x TKB, 9x MiniMote, 2x NorthQ Power, 2x Netatmo, 1x Foscam FI9831P, 9x Philips Hue,
Razberry, MySensors Arduino, HomeWave, AltUI, AltHue, DataYours, Grafana, openLuup, ZWay, ZeroBrane Studio.

Offline rafale77

  • Community Beta
  • Hero Member
  • ******
  • Posts: 1749
  • Karma: +101/-27
  • HA ≠ IoT as a blue sky is cloudless.
Re: Context and thread for executing actions
« Reply #19 on: January 20, 2019, 09:11:28 pm »
Sorry akbooer, I should have looked at the scheduling code in openLuup closer before making this comment. It is indeed more or less how it works. I made the assumption that it behaved more like the vera which openLuup was trying to mimic.
openLuup (79 devices, 141 scenes, 19 apps) master to VeraPlus (142 zwave nodes, 8 Zigbee nodes, 221 devices,  20 scenes , 2 apps) +  Hubitat (15 Zigbee nodes) + Home-Assistant (API Integrations). Bridged to Siri and Alexa. Homewave. VeraPlus ExtRooted and mios server independent.

Offline reneboer

  • Beta Testers
  • Hero Member
  • *****
  • Posts: 1574
  • Karma: +110/-31
Re: Context and thread for executing actions
« Reply #20 on: January 21, 2019, 05:00:39 am »
... changed <run> to <job> and changed the return true to return 4,nil.  Amazing how much code there is out there that has no return at all in either <run> or <job> tags of actions.  Even MCV code.  Lots of it.

AFAIK, this is not an issue.  As for scenes, a blank return from either run or job code should do the expected thing: exit successfully.

That's what I found in my testing whilst reverse engineering the Luup engine for openLuup.
By testing this weekend I found out why so many probably return blank. You can put as much code in the I..xml file returning values as you like, but those are not returned with luup.call_action. It are the values from the variables you defined in the S...xml file. I first ran tests on openLuup and repeated them on Vera all with that result. So that odd design 'logic' is why most call_actions return nothing if you ask me.

Cheers Rene
2xVeraLite, VeraEdge, openLuup, ALTUI, 20 switches, 10 dimmers, 20 sensors, 10 scene controllers, 1 Harmony Hub, many plug-ins. Not enough time.

Offline akbooer

  • Beta Testers
  • Master Member
  • *****
  • Posts: 6387
  • Karma: +291/-70
  • "Less is more"
Re: Context and thread for executing actions
« Reply #21 on: January 21, 2019, 05:24:16 am »
No, no, these are two separate things entirely.

The return codes from run or job functions control the status/flow of the action.  However, as you say, the named associated state variables are used to return their values to the original action caller.

Note that these values reflect their state after a run action, but before a job action, which makes them tricky to use correctly. 

Also note that an action can have both run and job parts, in which case the return status of the run part is used, as in scene Lua code, to determine whether the following job should be run at all.
3x Vera Lite-UI5/Edge-UI7, 25x Fibaro, 23x TKB, 9x MiniMote, 2x NorthQ Power, 2x Netatmo, 1x Foscam FI9831P, 9x Philips Hue,
Razberry, MySensors Arduino, HomeWave, AltUI, AltHue, DataYours, Grafana, openLuup, ZWay, ZeroBrane Studio.

Offline reneboer

  • Beta Testers
  • Hero Member
  • *****
  • Posts: 1574
  • Karma: +110/-31
Re: Context and thread for executing actions
« Reply #22 on: January 21, 2019, 11:03:17 am »
Totally separate indeed, but if you look at the logic in some existing plugins (only source to really try and understand how things work due to lack of proper documentation) some do think that that return (on run) is the call_action return value.

I should realize I best ask you as you had to learn all ugly internals building openLuup  ;D

Thanks for to continued learning. Truly appreciate all your knowledge sharing.  I have some more plugins to check and possible update.

Cheers Rene
2xVeraLite, VeraEdge, openLuup, ALTUI, 20 switches, 10 dimmers, 20 sensors, 10 scene controllers, 1 Harmony Hub, many plug-ins. Not enough time.

Offline akbooer

  • Beta Testers
  • Master Member
  • *****
  • Posts: 6387
  • Karma: +291/-70
  • "Less is more"
Re: Context and thread for executing actions
« Reply #23 on: January 21, 2019, 11:12:37 am »
...if you look at the logic in some existing plugins ... some do think that that return (on run) is the call_action return value.

Makes you wonder that they work at all!

Which reminds me that, as per the previous warning...

Quote
Note that these values reflect their state after a run action, but before a job action, which makes them tricky to use correctly. 

...this is a possible cause of unexpected side-effects if you do actually change <run> to <job> and the action has relevant return parameters!
3x Vera Lite-UI5/Edge-UI7, 25x Fibaro, 23x TKB, 9x MiniMote, 2x NorthQ Power, 2x Netatmo, 1x Foscam FI9831P, 9x Philips Hue,
Razberry, MySensors Arduino, HomeWave, AltUI, AltHue, DataYours, Grafana, openLuup, ZWay, ZeroBrane Studio.

Offline rigpapa

  • Beta Testers
  • Hero Member
  • *****
  • Posts: 1121
  • Karma: +187/-3
Re: Context and thread for executing actions
« Reply #24 on: January 21, 2019, 11:37:28 am »
...this is a possible cause of unexpected side-effects if you do actually change <run> to <job> and the action has relevant return parameters!

This is a very important point.

On a side note, I haven't seen a lot of use of actions to return values in live code. It seems most people just go after the related state variables directly, as the use case seems most often as a "getter" for a related to state value, rather than a computed result. I can't think of an example where I've ever had to use an action that returns a result other than something that can be derived from a state variable directly, in fact. But, I can see the utility of having that capability. Any idea where one might see such a thing in actual use?
Author of Reactor, DelayLight, SiteSensor, Rachio, Deus Ex Machina II, Intesis WMP Gateway, Auto Virtual Thermostat and VirtualSensor plugins. Vera Plus w/100+ Z-wave devices. Vera3, Lite. Hassio, Slapdash.

Offline akbooer

  • Beta Testers
  • Master Member
  • *****
  • Posts: 6387
  • Karma: +291/-70
  • "Less is more"
Re: Context and thread for executing actions
« Reply #25 on: January 21, 2019, 11:51:24 am »
It's immediately useful for actions which use the <job> tag, since it returns the actual job number.  This allows a plugin to follow the (asynchronous) status of the action, to know when it finishes, and, if necessary, to abort it.

Having said that, aside from the job termination notification, I know of no plugins which have tried to be that sophisticated with actions, probably because the documentation has been so poor.

For actions which are invoked via HTTP requests, the possibility of returning useful parameters as part of the response has obvious benefits in terms of I/O and response time.
3x Vera Lite-UI5/Edge-UI7, 25x Fibaro, 23x TKB, 9x MiniMote, 2x NorthQ Power, 2x Netatmo, 1x Foscam FI9831P, 9x Philips Hue,
Razberry, MySensors Arduino, HomeWave, AltUI, AltHue, DataYours, Grafana, openLuup, ZWay, ZeroBrane Studio.

Offline rigpapa

  • Beta Testers
  • Hero Member
  • *****
  • Posts: 1121
  • Karma: +187/-3
Re: Context and thread for executing actions
« Reply #26 on: January 21, 2019, 12:37:51 pm »
It's immediately useful for actions which use the <job> tag, since it returns the actual job number.  This allows a plugin to follow the (asynchronous) status of the action, to know when it finishes, and, if necessary, to abort it.

OK, but that comes back in the first three result values of call_action(), and that's not really what I'm talking about. I'm referring to the contents of the fourth argument, and using that direcly from Lua.

Quote
Having said that, aside from the job termination notification, I know of no plugins which have tried to be that sophisticated with actions, probably because the documentation has been so poor.

Amen to that. Couple with a rather loose interpretation of UPnP, well, you know...

Quote
For actions which are invoked via HTTP requests, the possibility of returning useful parameters as part of the response has obvious benefits in terms of I/O and response time

Agreed. Being able to return that data in any form is also huge for me.
Author of Reactor, DelayLight, SiteSensor, Rachio, Deus Ex Machina II, Intesis WMP Gateway, Auto Virtual Thermostat and VirtualSensor plugins. Vera Plus w/100+ Z-wave devices. Vera3, Lite. Hassio, Slapdash.

Offline reneboer

  • Beta Testers
  • Hero Member
  • *****
  • Posts: 1574
  • Karma: +110/-31
Re: Context and thread for executing actions
« Reply #27 on: January 21, 2019, 12:38:37 pm »
Hi rigpapa,

In my new release of the Harmony plugin I thought it was a good idea to add some mapping functions one could call to map between names and ID's as I added the possibility to use either to start actions or send commands. I did not have a variable for that (what is the point, right) and could not figure out why the (run) action had no return value. So now it ends in a generic temp variable, and the S...xml definitions make it a nice return name.

As you say there are very few examples, mine now is one  :o

Cheers Rene
2xVeraLite, VeraEdge, openLuup, ALTUI, 20 switches, 10 dimmers, 20 sensors, 10 scene controllers, 1 Harmony Hub, many plug-ins. Not enough time.

Offline rigpapa

  • Beta Testers
  • Hero Member
  • *****
  • Posts: 1121
  • Karma: +187/-3
Re: Context and thread for executing actions
« Reply #28 on: January 21, 2019, 12:46:43 pm »
Hi rigpapa,

In my new release of the Harmony plugin I thought it was a good idea to add some mapping functions one could call to map between names and ID's as I added the possibility to use either to start actions or send commands. I did not have a variable for that (what is the point, right) and could not figure out why the (run) action had no return value. So now it ends in a generic temp variable, and the S...xml definitions make it a nice return name.

As you say there are very few examples, mine now is one  :o

Cheers Rene

That's exactly the kind of thing I'd expect. I've done that as well in a plugin, to return the current plugin version, which is just a constant not stored in state. But I doubt that's ever been used by anybody.
Author of Reactor, DelayLight, SiteSensor, Rachio, Deus Ex Machina II, Intesis WMP Gateway, Auto Virtual Thermostat and VirtualSensor plugins. Vera Plus w/100+ Z-wave devices. Vera3, Lite. Hassio, Slapdash.