We have moved at community.getvera.com

Author Topic: openLuup - running unmodified plugins on any machine  (Read 127050 times)

Offline akbooer

  • Moderator
  • Master Member
  • *****
  • Posts: 6387
  • Karma: +292/-70
  • "Less is more"
Re: openLuup - running unmodified plugins on any machine
« Reply #435 on: October 25, 2015, 09:58:04 am »
guess thats the best way to go ...
well while the transition i gonna stick with the Scenes only and move "some" of the scripts/plugins ..
That's a reasonable starting point.

Quote
as the new machine also uses new ID's (i just hope they wont change on a random basis)
so i have a lot of id-numbers to replace while copy the scenes over ...
They won't change if you don't change anything.  The key would be to have ALTUI as device 3, and the VeraBridge as device 4.  I would always build a startup.lua file which recreates the system you want.  Always backup your user_data.json file.

To get around the device ID problem, I've posted elsewhere a simple function which you can put into the startup Lua script which maps between device names and IDs.  For me, the names are much more stable.

Quote
my heavy parts are tons of CURL's to other systems connected in my house ...
like tell transmission to pull the parking break so i got some bandwith to work ;)
turn on Airconditioner in the office if its after 13:00 and my wife has late-shift
push notifications and so on and so on ..
Why would you use curl and not luup.inet.wget ?

Quote
usally only 10% of the things do happen .. rest falles trough the reload loop
I hope we're on the way to fixing that.

Quote
oh while on useless use-cases ..
how do i start a scene within a scene ?
Code: [Select]
luup.call_action("urn:micasaverde-com:serviceId:HomeAutomationGateway1", "RunScene", {SceneNum = "32"}, 1)
does not really work

Can you elaborate?  So far as I know, this does work.  A log would help (openLuup logs are much better than Vera ones!)
3x Vera Lite-UI5/Edge-UI7, 25x Fibaro, 23x TKB, 9x MiniMote, 2x NorthQ Power, 2x Netatmo, 1x Foscam FI9831P, 9x Philips Hue,
Razberry, MySensors Arduino, HomeWave, AltUI, AltHue, DataYours, Grafana, openLuup, ZWay, ZeroBrane Studio.

Offline CudaNet

  • Beta Testers
  • Hero Member
  • *****
  • Posts: 1401
  • Karma: +42/-11
  • Chimichanga !
Re: openLuup - running unmodified plugins on any machine
« Reply #436 on: October 25, 2015, 10:11:43 am »
The good news is openLuup initializes without an error (version 5.3a). Unfortunately I seem to be having issues, for example (simple test):
Using AltUI, I set the dimmer to 100% on the cloned device [13]. I don't see it making the call to Vera at all to make the change to device [44].

Code: [Select]
2015-10-25 09:01:34.354   openLuup.server:: /data_request?id=action&output_format=json&DeviceNum=13&serviceId=urn:upnp-org:serviceId:Dimming1&action=SetLoadLevelTarget&newLoadlevelTarget=100 tcp{client}: 0x1be5000
2015-10-25 09:01:34.355   luup.call_action:: 13.urn:upnp-org:serviceId:Dimming1.SetLoadLevelTarget
2015-10-25 09:01:34.357   openLuup.server:: request completed (2 bytes, 1 chunks, 2 ms) tcp{client}: 0x1be5000

Scenes seem to be running but it looks like most of my devices simply aren't updating from Vera. Should I start clean ?

@all

Can you try this fix (which is also on GitHub) - changes to the openLuup modules in that subdirectory.

It addresses a recent XML problem (I hope) and it works really well for my Sonos.  I would hope that DLNA and Squeeze box would be similarly affected.  Let me know (I'm sure you will.)

Is everyone OK with pulling stuff off GitHub and then I can stop posting the code here?
openLuup, AltUI, Zway and HomeWave, enough said...

Offline akbooer

  • Moderator
  • Master Member
  • *****
  • Posts: 6387
  • Karma: +292/-70
  • "Less is more"
Re: openLuup - running unmodified plugins on any machine
« Reply #437 on: October 25, 2015, 11:52:50 am »
Scenes seem to be running but it looks like most of my devices simply aren't updating from Vera. Should I start clean ?

Well, it's no consolation, but for a scene running on openLuup, triggered by a 1 minute timer, toggling a remote RGBW dimmer I see this:
Code: [Select]
2015-10-25 15:49:27.269   luup.scenes:0: running 1, Toggle Green, initiated by 1 min timer
2015-10-25 15:49:27.272   luup.call_action:0: 33.urn:micasaverde-com:serviceId:HaDevice1.ToggleState
2015-10-25 15:49:27.274   luup.call_action:0: action will be handled by parent: 4
2015-10-25 15:49:27.777   luup_log:4: http://172.16.42.14:3480/data_request?id=action&serviceId=urn:micasaverde-com:serviceId:HaDevice1&action=ToggleState&DeviceNum=44
2015-10-25 15:49:31.339   luup.variable_set:4: 33.urn:upnp-org:serviceId:SwitchPower1.Status was: 0 now: 1 #hooks:0
2015-10-25 15:49:31.342   luup.variable_set:4: 33.urn:upnp-org:serviceId:Dimming1.LoadLevelTarget was: 0 now: 100 #hooks:0
2015-10-25 15:49:31.344   luup.variable_set:4: 33.urn:upnp-org:serviceId:Dimming1.LoadLevelStatus was: 0 now: 100 #hooks:0

and, of course, it's actually making the device turn on/off too.

I wonder what's different?

3x Vera Lite-UI5/Edge-UI7, 25x Fibaro, 23x TKB, 9x MiniMote, 2x NorthQ Power, 2x Netatmo, 1x Foscam FI9831P, 9x Philips Hue,
Razberry, MySensors Arduino, HomeWave, AltUI, AltHue, DataYours, Grafana, openLuup, ZWay, ZeroBrane Studio.

Offline CudaNet

  • Beta Testers
  • Hero Member
  • *****
  • Posts: 1401
  • Karma: +42/-11
  • Chimichanga !
Re: openLuup - running unmodified plugins on any machine
« Reply #438 on: October 25, 2015, 12:14:13 pm »
I'm going to restore (v5.0), test to see if all is well then upgrade again. I'll post back my results later. This will at least tell me if I made a mistake on my end.

Scenes seem to be running but it looks like most of my devices simply aren't updating from Vera. Should I start clean ?

Well, it's no consolation, but for a scene running on openLuup, triggered by a 1 minute timer, toggling a remote RGBW dimmer I see this:
Code: [Select]
2015-10-25 15:49:27.269   luup.scenes:0: running 1, Toggle Green, initiated by 1 min timer
2015-10-25 15:49:27.272   luup.call_action:0: 33.urn:micasaverde-com:serviceId:HaDevice1.ToggleState
2015-10-25 15:49:27.274   luup.call_action:0: action will be handled by parent: 4
2015-10-25 15:49:27.777   luup_log:4: http://172.16.42.14:3480/data_request?id=action&serviceId=urn:micasaverde-com:serviceId:HaDevice1&action=ToggleState&DeviceNum=44
2015-10-25 15:49:31.339   luup.variable_set:4: 33.urn:upnp-org:serviceId:SwitchPower1.Status was: 0 now: 1 #hooks:0
2015-10-25 15:49:31.342   luup.variable_set:4: 33.urn:upnp-org:serviceId:Dimming1.LoadLevelTarget was: 0 now: 100 #hooks:0
2015-10-25 15:49:31.344   luup.variable_set:4: 33.urn:upnp-org:serviceId:Dimming1.LoadLevelStatus was: 0 now: 100 #hooks:0

and, of course, it's actually making the device turn on/off too.

I wonder what's different?
openLuup, AltUI, Zway and HomeWave, enough said...

Offline nullx8

  • Hero Member
  • *****
  • Posts: 870
  • Karma: +9/-3
  • Trouble !
Re: openLuup - running unmodified plugins on any machine
« Reply #439 on: October 25, 2015, 10:53:19 pm »
To get around the device ID problem, I've posted elsewhere a simple function which you can put into the startup Lua script which maps between device names and IDs.  For me, the names are much more stable.

could you hint me to that again i could not find it (google and forum search is pretty much useless for finding tiny bits :( )
i think its time for sub-forum for openLuup ...

ID mapping for core devices is defently somethings should be considered ..
i use svn to backup (cronjob simply commit to my repro)

variable Id's do change basically every time i hit reload. (i have not yet seen a device ID change .. but better not to risc it)
« Last Edit: October 25, 2015, 11:53:57 pm by nullx8 »
1 VeraLite, 2 RaspberryPI, Harmony Remote, openLuup, 85 Devices, 34Nodes, 63 Triggers, 49 Scenes

Offline CudaNet

  • Beta Testers
  • Hero Member
  • *****
  • Posts: 1401
  • Karma: +42/-11
  • Chimichanga !
Re: openLuup - running unmodified plugins on any machine
« Reply #440 on: October 26, 2015, 12:05:25 am »
OK, so I did a couple of tests.

  • Version 5 files.
  • Version 5.3a files.
  • Version 5.3a and 5.0 files.

v5.0 initialized and had no issues whatsoever loading the Vera devices.
v5.3 would initialize but wouldn't call Vera to load devices, so - no devices at all.
v5.3 and v5.0 (combination of files from both versions) initialized and loaded all my devices from Vera.

I have all the log data available for all tests but it came down to the VeraBridge files (from Git; v5.3). The last test (v53 and v5) was a combination of all the v5 VeraBridge files (/etc/cmh-ludl) and openLuup files (latest posted by you).

Unrelated topic, I vote for fetching the files off GitHub moving forward...
« Last Edit: October 26, 2015, 12:11:39 am by CudaNet »
openLuup, AltUI, Zway and HomeWave, enough said...

Offline CudaNet

  • Beta Testers
  • Hero Member
  • *****
  • Posts: 1401
  • Karma: +42/-11
  • Chimichanga !
Re: openLuup - running unmodified plugins on any machine
« Reply #441 on: October 26, 2015, 12:37:14 am »
And then there was sound !

I need to do some research on this error...
Code: [Select]
device 0 '_system_' requesting reload
modprobe: ERROR: could not insert 'ip_tables': Operation not permitted
iptables v1.4.21: can't initialize iptables table `nat': Table does not exist (do you need to insmod?)
Perhaps iptables or your kernel needs to be upgraded.
« Last Edit: October 26, 2015, 01:39:16 am by CudaNet »
openLuup, AltUI, Zway and HomeWave, enough said...

Offline nullx8

  • Hero Member
  • *****
  • Posts: 870
  • Karma: +9/-3
  • Trouble !
Re: openLuup - running unmodified plugins on any machine
« Reply #442 on: October 26, 2015, 02:20:59 am »
Nat is disabled or connection tracking is off

Make sure nf_conntrack is set to 1 if use iptables with nat-rules
1 VeraLite, 2 RaspberryPI, Harmony Remote, openLuup, 85 Devices, 34Nodes, 63 Triggers, 49 Scenes

Offline akbooer

  • Moderator
  • Master Member
  • *****
  • Posts: 6387
  • Karma: +292/-70
  • "Less is more"
Re: openLuup - running unmodified plugins on any machine
« Reply #443 on: October 26, 2015, 04:06:42 am »
could you hint me to that again i could not find it (google and forum search is pretty much useless for finding tiny bits :( )
The basic idea was here: http://forum.micasaverde.com/index.php/topic,14743.msg111814.html#msg111814 but these days I wrap it into a function which simply returns the ID given a name.  You can do the same for scenes.  I had thought of using the UUID option for devices (ie. strings as an alternative to numeric IDs, possibly also allowing device names.  But that diverges too much from Vera, I think.)

Quote
i think its time for sub-forum for openLuup ...
I think we are just about at critical mass for that.

Quote
ID mapping for core devices is defently somethings should be considered ..
i use svn to backup (cronjob simply commit to my repro)
If you use the same user_data.json (ie. don't run a startup.lua file) then device IDs will not change.

Quote
variable Id's do change basically every time i hit reload. (i have not yet seen a device ID change .. but better not to risc it)
I don't know what variable IDs are for.  They're not used by the system (openLuup) or any plugin AFAIK.  They're there simply for cosmetic value in the user_data, since they not documented and I didn't know if they would make a difference.
3x Vera Lite-UI5/Edge-UI7, 25x Fibaro, 23x TKB, 9x MiniMote, 2x NorthQ Power, 2x Netatmo, 1x Foscam FI9831P, 9x Philips Hue,
Razberry, MySensors Arduino, HomeWave, AltUI, AltHue, DataYours, Grafana, openLuup, ZWay, ZeroBrane Studio.

Offline akbooer

  • Moderator
  • Master Member
  • *****
  • Posts: 6387
  • Karma: +292/-70
  • "Less is more"
Re: openLuup - running unmodified plugins on any machine
« Reply #444 on: October 26, 2015, 07:45:56 am »
@nullx8, This is the code I use for translating between device and scene names and IDs.

Code: [Select]
local deviceLookupTable    -- table of [device.description] = deviceNo for name lookup
local sceneLookupTable    -- table of [scene.description]  = sceneNo for name lookup

-- generic reverse lookup table constructor for element table.field
function lookupTable (table, field)
  field = field or "description" -- default field name
  local lookup = {} 
  for i,d in pairs (table) do  -- create the lookup table
    lookup [d[field] ] = i
  end
  return lookup
end

-- deviceNo (deviceName) returns matching device number
function deviceNo (name)
  deviceLookupTable = deviceLookupTable or lookupTable (luup.devices)
  return deviceLookupTable[name]
end

-- sceneNo (sceneName) returns matching scene number
function sceneNo (name)
  sceneLookupTable = sceneLookupTable or lookupTable (luup.scenes)
  return sceneLookupTable[name]
end

Put it in your startup code (not the startup.lua.)
 
Use it like this:
Code: [Select]
local devNo = deviceNo "My Device Name"
local scnNo = sceneNo  "A useful scene"

I find this very useful in scene and startup code.

« Last Edit: October 26, 2015, 08:16:52 am by akbooer »
3x Vera Lite-UI5/Edge-UI7, 25x Fibaro, 23x TKB, 9x MiniMote, 2x NorthQ Power, 2x Netatmo, 1x Foscam FI9831P, 9x Philips Hue,
Razberry, MySensors Arduino, HomeWave, AltUI, AltHue, DataYours, Grafana, openLuup, ZWay, ZeroBrane Studio.

Offline akbooer

  • Moderator
  • Master Member
  • *****
  • Posts: 6387
  • Karma: +292/-70
  • "Less is more"
Re: openLuup - running unmodified plugins on any machine
« Reply #445 on: October 26, 2015, 07:58:45 am »
OK, so I did a couple of tests.

...and some!  I apologise.  By way of excuse, I have just discovered that somehow the GitHub repository that I was using was in the trash can on my Mac  :(

How this happened, I have no idea, but this has led me into some sort of version control hell.  I believe that the VeraBridge in the 5.3 release was, in fact, an experimental one I had offered to @mda to try and solve a problem there.

Can you confirm that the attached is, in fact, the version that works for you?

Quote
Unrelated topic, I vote for fetching the files off GitHub moving forward...

That's one vote.  If you confirm that the attached is the right file, I'll update GitHub and move onwards [Edit: you can find it at https://github.com/akbooer/openLuup]

I think we need to regroup and assess what's working.  Might be a good idea to share those configuration docs you have? (pull request on GihHub??)
« Last Edit: October 26, 2015, 08:21:38 am by akbooer »
3x Vera Lite-UI5/Edge-UI7, 25x Fibaro, 23x TKB, 9x MiniMote, 2x NorthQ Power, 2x Netatmo, 1x Foscam FI9831P, 9x Philips Hue,
Razberry, MySensors Arduino, HomeWave, AltUI, AltHue, DataYours, Grafana, openLuup, ZWay, ZeroBrane Studio.

Offline akbooer

  • Moderator
  • Master Member
  • *****
  • Posts: 6387
  • Karma: +292/-70
  • "Less is more"
Re: openLuup - running unmodified plugins on any machine
« Reply #446 on: October 26, 2015, 08:14:45 am »
And then there was sound !

I need to do some research on this error...
Code: [Select]
device 0 '_system_' requesting reload
modprobe: ERROR: could not insert 'ip_tables': Operation not permitted
iptables v1.4.21: can't initialize iptables table `nat': Table does not exist (do you need to insmod?)
Perhaps iptables or your kernel needs to be upgraded.

This is, I think, the Sonos plugin checking for, or trying to install, the UPnP Discovery patch.  I haven't tested that, and without installing it, the system seems to be able to find my various bits of Sonos hardware without difficulty (both on OpenWRT and Debian.)

I wouldn't bother with the UPnP Event Proxy either, unless someone can tell me that it's necessary.


3x Vera Lite-UI5/Edge-UI7, 25x Fibaro, 23x TKB, 9x MiniMote, 2x NorthQ Power, 2x Netatmo, 1x Foscam FI9831P, 9x Philips Hue,
Razberry, MySensors Arduino, HomeWave, AltUI, AltHue, DataYours, Grafana, openLuup, ZWay, ZeroBrane Studio.

Offline CudaNet

  • Beta Testers
  • Hero Member
  • *****
  • Posts: 1401
  • Karma: +42/-11
  • Chimichanga !
Re: openLuup - running unmodified plugins on any machine
« Reply #447 on: October 26, 2015, 11:38:26 am »
Good news, the file works perfectly... You're making excellent progress !

OK, so I did a couple of tests.

...and some!  I apologise.  By way of excuse, I have just discovered that somehow the GitHub repository that I was using was in the trash can on my Mac  :(

How this happened, I have no idea, but this has led me into some sort of version control hell.  I believe that the VeraBridge in the 5.3 release was, in fact, an experimental one I had offered to @mda to try and solve a problem there.

Can you confirm that the attached is, in fact, the version that works for you?

Quote
Unrelated topic, I vote for fetching the files off GitHub moving forward...

That's one vote.  If you confirm that the attached is the right file, I'll update GitHub and move onwards [Edit: you can find it at https://github.com/akbooer/openLuup]

I think we need to regroup and assess what's working.  Might be a good idea to share those configuration docs you have? (pull request on GihHub??)
« Last Edit: October 26, 2015, 12:29:46 pm by CudaNet »
openLuup, AltUI, Zway and HomeWave, enough said...

Offline CudaNet

  • Beta Testers
  • Hero Member
  • *****
  • Posts: 1401
  • Karma: +42/-11
  • Chimichanga !
Re: openLuup - running unmodified plugins on any machine
« Reply #448 on: October 26, 2015, 12:05:42 pm »
Quite possibly, I just used the [discover] button which quickly located the devices - never touched the patch. The err is a 1:1 for each device on init..

And then there was sound !

I need to do some research on this error...
Code: [Select]
device 0 '_system_' requesting reload
modprobe: ERROR: could not insert 'ip_tables': Operation not permitted
iptables v1.4.21: can't initialize iptables table `nat': Table does not exist (do you need to insmod?)
Perhaps iptables or your kernel needs to be upgraded.

This is, I think, the Sonos plugin checking for, or trying to install, the UPnP Discovery patch.  I haven't tested that, and without installing it, the system seems to be able to find my various bits of Sonos hardware without difficulty (both on OpenWRT and Debian.)

I wouldn't bother with the UPnP Event Proxy either, unless someone can tell me that it's necessary.
openLuup, AltUI, Zway and HomeWave, enough said...

Offline CudaNet

  • Beta Testers
  • Hero Member
  • *****
  • Posts: 1401
  • Karma: +42/-11
  • Chimichanga !
Re: openLuup - running unmodified plugins on any machine
« Reply #449 on: October 26, 2015, 12:25:37 pm »
This is a fairly clean system configured for sole purpose of testing openLuup. Nothing that wasn't added by a default Ubuntu install is on this system unless it was needed for openLuup. However, It appears from /proc/net that I do have entries within nf_conntrack.

Code: [Select]
unknown  2 503 src=172.16.5.25 dst=224.0.0.251 [UNREPLIED] src=224.0.0.251 dst=172.16.5.25 mark=0 use=2
udp      17 123 src=172.16.5.25 dst=91.189.89.199 sport=123 dport=123 src=91.189.89.199 dst=172.16.5.25 sport=123 dport=123 [ASSURED] mark=0 use=2
udp      17 9 src=172.16.0.71 dst=224.0.0.252 sport=61914 dport=5355 [UNREPLIED] src=224.0.0.252 dst=172.16.0.71 sport=5355 dport=61914 mark=0 use=2
udp      17 4 src=172.16.0.31 dst=172.16.255.255 sport=138 dport=138 [UNREPLIED] src=172.16.255.255 dst=172.16.0.31 sport=138 dport=138 mark=0 use=2
udp      17 20 src=172.16.5.104 dst=172.16.255.255 sport=138 dport=138 [UNREPLIED] src=172.16.255.255 dst=172.16.5.104 sport=138 dport=138 mark=0 use=2
unknown  2 502 src=0.0.0.0 dst=224.0.0.1 [UNREPLIED] src=224.0.0.1 dst=0.0.0.0 mark=0 use=2

Nat is disabled or connection tracking is off

Make sure nf_conntrack is set to 1 if use iptables with nat-rules
openLuup, AltUI, Zway and HomeWave, enough said...