We have moved at community.getvera.com

Author Topic: Luup Service IDs  (Read 551 times)

Offline Buxton

  • Full Member
  • ***
  • Posts: 209
  • Karma: +12/-0
Luup Service IDs
« on: July 16, 2018, 02:46:45 am »
Hey AK,

I searched quite a bit for this topic with no real results.  Here's what I'm after.

I want to save the on/off state of my Zwave devices to a table, so that after a given event (that will change multiple device states), I can poll the table and restore the overall state of my zwave devices to what they were prior to the event.  To do so, I'm thinking that I would iterate through the device table, and using the obtained device number, retrieve the status of the device using variable_get.  However, programmatically getting the service IDs of devices is proving to be difficult. I imagine this is easy, but I'm not seeing it.   Here is the code I have so far which gets me most of the way there:
Code: [Select]
local idx = {}

for openLuupNo in pairs(luup.devices) do
    if openLuupNo > 10000   then
        idx[#idx+1] = openLuupNo
    end
end
table.sort (idx)

for _, openLuupNo in ipairs(idx) do
    local d = luup.devices[openLuupNo]
    local devType = d.device_type
    local devDescription = d.description
    local devNodeID = d.id
    local devString  = (string.format('[%03d / %s] %s %s',  openLuupNo, devNodeID, devDescription, devType))
   
    local devServiceID = "Some sort of transform of devType to devServiceID"
    local w = luup.variable_get(devServiceID, "Status", openLuupNo)

    if d.id > "1" then
--      print (w)
      print (devString)
    end
end

As always, any help greatly appreciated.

Offline akbooer

  • Moderator
  • Master Member
  • *****
  • Posts: 6387
  • Karma: +292/-70
  • "Less is more"
Re: Luup Service IDs
« Reply #1 on: July 16, 2018, 12:07:30 pm »
    I want to save the on/off state of my Zwave devices to a table, so that after a given event (that will change multiple device states), I can poll the table and restore the overall state of my zwave devices to what they were prior to the event.

    I assume you have good reasons for this, but suggest that you are about to be skating on thin ice...

    Quote
    To do so, I'm thinking that I would iterate through the device table, and using the obtained device number, retrieve the status of the device using variable_get. 

    So far, so good.

    Quote
    However, programmatically getting the service IDs of devices is proving to be difficult. I imagine this is easy, but I'm not seeing it.   

    You are not missing anything, for there is nothing to see.  It's a sad fact that Vera/MiOS forgot to include a request which gives you what you want: "what serviceIds does this device support?"   The nearest you can get using the luup.xxx() API is:

    The only other thing you can do on Vera, from a programmatic point of view would be to make an HTTP request to your own machine asking for the (huge) user_data file
    [/list]
    Code: [Select]
    local _, user_data = luup.inet.wget "http://127.0.0.1:3480/data_request?id=user_data2"
    ...and then parse the user_data with a JSON decoder and search through all the device services.

    Strangely, you CAN do what you want from a browser using the request:
    Code: [Select]
    http://127.0.0.1:3480/data_request?id=invoke
    ...but this returns HTML and requires you to follow device links to get at the data.

    Quote
    Here is the code I have so far which gets me most of the way there:

    Two comments on this:
    • you shouldn't really use the device boundary 10000 when searching for local devices, this is not a guaranteed number.  The right way to do it would be to check that its parent (or higher ancestor) is not a VeraBridge.
    • "Some sort of transform of devType to devServiceID" - this is where you fall through the ice, so to speak.  There simply is no such unique mapping: serviceIds are totally separate from device types.  However, if you do know your device types well, you know what services they generally provide.  But code like this is likely to break.

    Even if you get to this point, how are you going to restore the old states?  You can't just change the variables in question, you have to go through a service action call, and you have to know what that is for each variable type that you are using.

    HAVING SAID ALL THAT...

    You are running on openLuup, and not Vera.  It is, in fact, trivial to find the services and variables that belong to a device.  This would, however, mean delving slightly below the thin veneer of inadequate luup.xxx() calls to the object-oriented layer which is the real heart of openLuup.

    I will gladly tell you how to do that if you have an answer to the really difficult question of how you're planning to restore the states.
    « Last Edit: July 16, 2018, 12:33:27 pm by akbooer »
    3x Vera Lite-UI5/Edge-UI7, 25x Fibaro, 23x TKB, 9x MiniMote, 2x NorthQ Power, 2x Netatmo, 1x Foscam FI9831P, 9x Philips Hue,
    Razberry, MySensors Arduino, HomeWave, AltUI, AltHue, DataYours, Grafana, openLuup, ZWay, ZeroBrane Studio.

    Offline Buxton

    • Full Member
    • ***
    • Posts: 209
    • Karma: +12/-0
    Re: Luup Service IDs
    « Reply #2 on: July 16, 2018, 07:32:09 pm »
    Quote
    Vera/MiOS forgot

    LOL!  AK you are polite to a fault. 

    As for using a device boundary, I completely agree and it is not the way I'd like to go, but it seems to be the way most code handles obtaining ZWave only devices.  Checking the parent works, (I believe the Info-Viewer plugin uses this method) but it seems overly complex for what I'm doing.

    As for setting the device state, I intended to use this routine only for lights, so it would be relatively uncomplicated--using set_device.  But I can imagine someone stumbling across this thread who might want to apply the same logic to other devices--say a group of thermostats in an event where there is a temporary break in the normal daily routine that requires a temperature adjustment, or even  restoring values to non-zwave devices.  In my case, a group of lights come on (depending on the time of day) triggered by a security event.  When the event is cleared, the lights that came on (and were not on previously) are shut off from data supplied by a state table.

    As you point out, both of these quandaries display weaknesses in the underlying vera luup engine.  For example, one should simply be able to poll a device flag that would signify that it is indeed a zwave device. Likewise, one should also be able to easily determine the serviceids.  So I'd like to propose that openLuup correct these deficiencies with a series of luup.xxx extensions.

    When I was looking through the openLuup code in search of answers, I could make out the contours of a solution as the OOP model in Lua relies on tables, and accordingly, this type of data is stored in openLuup.  But IMHO, making direct calls to the openLuup engine is also prone to failure as a change here or there could break the call.  A luup extension , on the other hand, would offer relative simplicity to ones code, and safeguard, to a degree, against future disruption.

    Anyway, my two cents for what it's worth.  If the above borders on heresy, I can certainly kluge a solution with some hard mapping.  Let me know your thoughts and I'll go forward accordingly.

    Offline akbooer

    • Moderator
    • Master Member
    • *****
    • Posts: 6387
    • Karma: +292/-70
    • "Less is more"
    Re: Luup Service IDs
    « Reply #3 on: July 17, 2018, 06:15:54 am »
    @Buxton

    Thanks for your very thoughtful response.  Much appreciated.  Comments below...

    As you point out, both of these quandaries display weaknesses in the underlying vera luup engine.  For example, one should simply be able to poll a device flag that would signify that it is indeed a zwave device. Likewise, one should also be able to easily determine the serviceids.  So I'd like to propose that openLuup correct these deficiencies with a series of luup.xxx extensions.

    Well this, indeed, is my own quandary.  I am very, very, loathe to make extensions to luup.xxx() calls.  It's only very recently that I've added a luup.openLuup flag to signify that you're running on openLuup and not Vera.  The whole point was that you shouldn't be able to tell the difference.  BUT... people always push things to the limit, and there are clearly some cases where you need to know this.

    One option would simply to be to hang extensions off the, now extant, luup.openLuup branch.  Thus you could have:

    Code: [Select]
    luup.openLuup.services (device_number)
    luup.openLuup.actions (device_number)
    luup.openLuup.variables (device_number)

    Using luup.openLuup as a flag would still work if you test for non-nil, rather than true.

    Quote
    When I was looking through the openLuup code in search of answers, I could make out the contours of a solution as the OOP model in Lua relies on tables, and accordingly, this type of data is stored in openLuup.  But IMHO, making direct calls to the openLuup engine is also prone to failure as a change here or there could break the call.  A luup extension , on the other hand, would offer relative simplicity to ones code, and safeguard, to a degree, against future disruption.

    As it is, the OO approach at the moment would be:

    Code: [Select]
    local device_number = 2
    local d = luup.devices[device_number]
    for serviceId, svc in pairs(d.services) do
      print ("\n--- Actions for serviceId " .. serviceId)
      for action in pairs (svc.actions) do print (action) end
      print ("\n--- Variables for serviceId " .. serviceId)
      for variable, var in pairs (svc.variables) do print(variable, var.value) end
    end
    -- or your can just get to a list of variables by id directly from the device
    print "\n--- Variables by id"
    for id, var in ipairs(d.variables) do print(id, var.name, var.value) end

    which, for me, gives:

    Code: [Select]
    --- Actions for serviceId openLuup
    EmptyTrash
    SetHouseMode
    SendToTrash
    Test
    RunScene

    --- Variables for serviceId openLuup
    Memory_Mb 8.6
    HouseMode 1
    CpuLoad 4.1
    MemAvail_Mb 816.5
    MemTotal_Mb 949.6
    Version v18.7.15
    Test 123
    MemFree_Mb 655.9
    StartTime 2018-07-17T07:31:02
    Uptime_Days 0.15
    Vnumber 180715

    --- Actions for serviceId urn:upnp-org:serviceId:altui1

    --- Variables for serviceId urn:upnp-org:serviceId:altui1
    DisplayLine1 9 Mb, cpu 4.1%, 0.15 days
    DisplayLine2 [Home]

    --- Variables by id
    1 HouseMode 1
    2 DisplayLine2 [Home]
    3 Version v18.7.15
    4 StartTime 2018-07-17T07:31:02
    5 Memory_Mb 8.6
    6 CpuLoad 4.1
    7 Uptime_Days 0.15
    8 DisplayLine1 9 Mb, cpu 4.1%, 0.15 days
    9 MemFree_Mb 655.9
    10 MemAvail_Mb 816.5
    11 MemTotal_Mb 949.6
    12 Vnumber 180715

    But you are absolutely right.  This opens a whole new can of worms.  The variable object is currently totally unprotected... don't for one moment suspect that simply changing var.value to something else would correctly update the variable.  The variable is the most complex structure in the system and has to handle callbacks, timestamps, user_data version numbers, data history, ...

    Quote
    Anyway, my two cents for what it's worth.  If the above borders on heresy, I can certainly kluge a solution with some hard mapping.  Let me know your thoughts and I'll go forward accordingly.

    Worth more than two cents, I would say.  Perhaps I will upgrade you from Beta-tester to Alpha-tester?
    3x Vera Lite-UI5/Edge-UI7, 25x Fibaro, 23x TKB, 9x MiniMote, 2x NorthQ Power, 2x Netatmo, 1x Foscam FI9831P, 9x Philips Hue,
    Razberry, MySensors Arduino, HomeWave, AltUI, AltHue, DataYours, Grafana, openLuup, ZWay, ZeroBrane Studio.

    Offline Buxton

    • Full Member
    • ***
    • Posts: 209
    • Karma: +12/-0
    Re: Luup Service IDs
    « Reply #4 on: July 17, 2018, 09:52:47 pm »
    Yes, having luup extensions somewhat separated by the openLuup object, but nevertheless available, is a good way to maintain the integrity of the engine.  It would also allow you to sandbox some of the internals where you see the necessity. So a good compromise, and documenting such additions would be more compact.

    Alpha away, though like most, my attention span is limited by how chaotic my worklife is!

    Offline Buxton

    • Full Member
    • ***
    • Posts: 209
    • Karma: +12/-0
    Re: Luup Service IDs
    « Reply #5 on: July 20, 2018, 01:25:19 am »
    One more similar aspect that came to mind (but I forgot about when writing the above) is obtaining scene data.  This post touches on it: http://forum.micasaverde.com/index.php/topic,8537.msg54699.html#msg54699

    Another way to accomplish saving a state table would entail using scenes as the foundation.  Per what I wrote above, does openLuup have access to at least "native" AltUI scene data?  I know you don't bring across vera scene data during bridging, rather you create a reference to scenes with a simple http call to the remote vera. 

    For example, one would create a scene where specific devices are turned on or off during an event.  During the event and prior to running the scene, save the current status of the devices that are encapsulated in the scene via a lua script.  Then run the response to the event which includes firing the scene, and when the event clears, use the state table originally derived from the scene to reinstate the status of the various devices that may (or may not) have been effected.  This allows a great deal of flexibility over what devices to use in an event response, and as well, the logic following an event's aftermath.

    The problem of course is that luup.xxx does not give you direct access to this type of data....

    I promise this is my last post on the topic!

    Offline akbooer

    • Moderator
    • Master Member
    • *****
    • Posts: 6387
    • Karma: +292/-70
    • "Less is more"
    Re: Luup Service IDs
    « Reply #6 on: July 20, 2018, 02:38:15 am »
    does openLuup have access to at least "native" AltUI scene data?

    In fact, scenes are stored and implemented natively in openLuup, not AltUI.  The only feature of Vera scenes that I intentionally omitted was triggers, since AltUI does a much better job with device variable watches.

    Quote
    I know you don't bring across vera scene data during bridging, rather you create a reference to scenes with a simple http call to the remote vera. 

    That's the default bridge behaviour, yes.  But also, it's possible to import Vera scenes and have them automatically converted to 'native' Lua scenes.  VeraBridge's GetVeraScenes action will do this for you.

    Quote
    For example, one would create a scene where specific devices are turned on or off during an event.  During the event and prior to running the scene, save the current status of the devices that are encapsulated in the scene via a lua script.  Then run the response to the event which includes firing the scene, and when the event clears, use the state table originally derived from the scene to reinstate the status of the various devices that may (or may not) have been effected.  This allows a great deal of flexibility over what devices to use in an event response, and as well, the logic following an event's aftermath.

    Interestingly, after some arm-twisting, forum member @DesT persuaded me to implement user-defined global callbacks before and after any scene is run.  These are simply global functions defined in Lua Startup and activated by defining their names in system attributes openLuup.Scenes.Prolog and .Epilog.

    Quote
    The problem of course is that luup.xxx does not give you direct access to this type of data....

    No, but, once again, at the openLuup OO layer, it's all instantly available.  By way of example, a method call on a scene object reveals for one of my scenes:

    Code: [Select]
    {
    Timestamp = 1483028673,
    favorite = false,
    groups = {{
    actions = {{
    action = "SetTarget",
    arguments = {{
    name = "newTargetValue",
    value = "1"
    }},
    device = "50",
    service = "urn:upnp-org:serviceId:SwitchPower1"
    }},
    delay = 0
    }},
    id = 5,
    last_run = 1532027621,
    lua = "",
    modeStatus = "0",
    name = "Landing Table ON",
    paused = 0,
    room = 0,
    timers = {{
    days_of_week = "1,2,3,4,5,6,7",
    enabled = 1,
    id = 1,
    last_run = 1532027621,
    name = "new timer",
    next_run = 1532113949.9436,
    time = "-01:00:00T",
    type = 2
    }},
    triggers = {},
    triggers_operator = "OR"
    }

    You have everything: timers, delayed actions, etc.  This is exactly the structure (although in Lua and not in JSON) defined here: http://wiki.micasaverde.com/index.php/Scene_Syntax

    This could, of course, be wrapped as a function call in luup.openLuup.xxx

    Quote
    I promise this is my last post on the topic!

    I do hope not.  This turns out to be quite thought-provoking.
    3x Vera Lite-UI5/Edge-UI7, 25x Fibaro, 23x TKB, 9x MiniMote, 2x NorthQ Power, 2x Netatmo, 1x Foscam FI9831P, 9x Philips Hue,
    Razberry, MySensors Arduino, HomeWave, AltUI, AltHue, DataYours, Grafana, openLuup, ZWay, ZeroBrane Studio.