We have moved at community.getvera.com

Author Topic: Porting Vera scenes to openLuup  (Read 1259 times)

Online a-lurker

  • Hero Member
  • *****
  • Posts: 872
  • Karma: +66/-8
Porting Vera scenes to openLuup
« on: July 18, 2017, 12:23:18 am »
Caveats: This may not work for you, for reasons I haven't envisaged. You should be able to save yourself. All care and no responsibility. Need to be able to use WinSCP or similar. Can make backups of the data_data.json file in openLuup. Need to be confident editing json files. Have an understanding of how to execute Linux commands. Have AltUI installed on your Vera. Know how to make a scene be triggered by a "watched" variable. This may all become redundant if akbooer ever automates this process somehow or changes openLuup in some unexpected way.

The openLuup Vera bridge conveniently mirrors Vera scenes, so they can be shown in AltUI, when browsing the openLuup server. We can make use of this functionality, to port Vera scenes to the openLuup environment.

1) Triggers: Events vs watches:
openLuup does not use "Event" triggers. It instead (and quite rightly) makes use of "Watched" variables. On Vera you need to modify any scene that uses an Event trigger to use a "Watch" instead. These can be found via AltUI:

Using AltUI-->More-->Scenes: Use the table checkbox list and check "id", "triggers", "watches", "lua", "room". Any scene that has a trigger (an Event) needs to be converted to use a watch. Make the appropriate changes to the scene(s)

It's too complicated to explain the process here. Note that this needs to be done before continuing and one needs to be confident your set up is still functioning, as it should, before proceeding. May need a few days to make sure all is OK. If you can't do this - then time to have a rethink about continuing.

2) So one doesn't get confused about what is what; make a record of the scenes used by Vera and the openLuup device:

Using AltUI-->More-->Scenes: Use the table checkbox list and check "id", "triggers", "watches", "lua", "room"
Set the display to show all scenes and sort the table by "id".
Cut and paste the info into a text file; one file each for Vera and openLuup.

3) Stop the openLuup server using this URL in your browser:

http://openLuup_IP_Address:3480/data_request?id=exit

Web page will report:
requested openLuup exit at <date & time>

You can no longer access openLuup via the browser as you have just stopped the openLuup server. Plus openLuup is no longer doing any automation.

4) Back up the user_data.json file:
Just before openLuup stops, it writes out the most recent "user_data.json" file to "/etc/cmh-ludl"
Use WinSCP to copy this file to your PC of choice. MAKE A BACKUP OF THIS FILE somewhere.
If you can't do this - then time to have a rethink about continuing.

5) Edit the file to copy Vera scenes to the openLuup environment:
Using you favorite text editor (that would be Notepad++) start hacking the json data.

Search for this including the brackets/braces:

Code: [Select]
scenes":[{
This will be found very close to the end of the file. This is the start of the scenes array. Each scene and what is does is defined in this array, eg a single scene looks like so:

Code: [Select]
{
    "Timestamp":1492299092,
    "favorite":false,
    "groups":[],
    "id":10025,
    "lua":"luup.inet.wget \"http://Vera_IP_Address:3480/data_request?id=action&serviceId=urn:micasaverde-com:serviceId:HomeAutomationGateway1&action=RunScene&SceneNum=25\"",
    "modeStatus":"0",
    "name":"Scene sw momentary",
    "paused":"0",
    "room":1,
    "timers":[],
    "triggers":[]
  },

Two items to note:
a) scenes that are bridged have 10,000 added to the Vera scene id. So in the above, the Vera scene id is "25" since the bridged id is "10025". If more than one Vera is bridged then 20,000 is added, etc, per Vera.
b) the "Lua" tag is in use for every bridged scene. The Lua code just runs the scene on the bridged Vera. See above.

Make the changes:
a) select the scene json text, eg as shown above, to be ported to openLuup and just copy it immediately underneath. Be very careful with the curly brackets and commas.
b) change the scene id to an id not being used by openLuup. Be careful to keep the double quotes in place. Your chosen new id will not be a number above 10,000 and will be unused by openLuup. Refer to you scene lists you made in section 2). Accidentally having two scenes with the same id is not a good idea.
c) the Lua json section, that calls the scene in Vera, must be replaced with the Lua used by the scene in Vera. This may be none, so an empty string would be used or it would be the Lua found in Vera. You can use the scene editor on Vera to find the Lua to copy into to place or you could directly access Vera's "user_data.json" file (more complicated procedure not explained here). If the Lua may need to be re arranged to sort out any referencing problem after porting (user specific - you need to work this out)

Notes:
a) don't duplicate scene numbers
b) openLuup sorts the scenes by scene id, next time it writes out the user_data.json file. So the order of scene ids is unimportant when you do your editing. But of course, must still be part of the scenes array.
c) you can port one scene at a time or if feeling confident, as many as you like at the one time.
d) you may encounter some user specific problem that may need to be cleaned up using AltUI scene editor at a later stage.
e) it may not make sense to port some scenes depending on what they do (user specific). Think about what each scene does.
f) scenes will probably need to be altered to reference the Vera devices, especially Z-Wave devices.

6) Check the json file:
Use https://jsonlint.com/ or similar to be sure you have not introduced any parse errors into the json file.

7) Return file to openLuup hardware
Copy the changed "user_data.json" file back to "/etc/cmh-ludl"

8 ) Restart openLuup
Using WinSCP change the display so that it's showing "/etc/cmh-ludl". Execute this command:

Code: [Select]
/etc/cmh-ludl/openLuup_reload

Note that WinSCP looses access at this stage and you will probably have to log back in.

9) Check openLuup
Using your browser navigate to: http://openLuup_IP_Address:3480/console?page=log&version=startup
Make sure all looks OK. Refresh AltUI and see if the scenes show up.

10) NOT working? Then stop openLuup, restore your "user_data.json" back up and restart openLuup. Have a rethink on how to proceed.

11) Cleaning up:
If the scenes appear you will now have the original bridged scenes and your new local copies in openLuup. Delete the unwanted scenes in Vera. Restart the Luup engine in Vera using: AltUI-->Misc-->Reload_Luup_Engine. Refresh the Vera browser page. Now do the same for the openLuup. The Bridge plugin will see the scenes are no longer in Vera and will change its mirroring, so they no longer appear in the openLuup browser page.

12) All may need tinkering to get all this to work.

NOTE you can use the same concept for porting devices but there are definitely extra steps that need to be completed to do so. Perhaps in another post.
« Last Edit: July 18, 2017, 05:43:31 pm by a-lurker »

Offline akbooer

  • Moderator
  • Master Member
  • *****
  • Posts: 6387
  • Karma: +290/-70
  • "Less is more"
Re: Porting Vera scenes to openLuup
« Reply #1 on: July 18, 2017, 03:55:01 am »
Quote
This may all become redundant if akbooer ever automates this process somehow...

I would say that 95% of this process can be very easily automated.  The mechanical aspects of transferring the Vera scene code (very prone to error if done manually) are relatively straight-forward.

It would be possible to allocate a block of scene numbers (say the 90,000s) for the copied scenes and mark them as "paused" so that they can't run.  UPnP triggers could be removed, or perhaps left as documentation (they are, anyway, not executed in openLuup.)  Actions could be updated with the mirrored device numbers.  The only things that can't be easily automated are:
  • adding device variable watch triggers
  • fixing any Lua code changes (like hard-coded device numbers)

If there's demand, I could do this.
3x Vera Lite-UI5/Edge-UI7, 25x Fibaro, 23x TKB, 9x MiniMote, 2x NorthQ Power, 2x Netatmo, 1x Foscam FI9831P, 9x Philips Hue,
Razberry, MySensors Arduino, HomeWave, AltUI, AltHue, DataYours, Grafana, openLuup, ZWay, ZeroBrane Studio.

Offline rafale77

  • Community Beta
  • Hero Member
  • ******
  • Posts: 1749
  • Karma: +101/-27
  • HA ≠ IoT as a blue sky is cloudless.
Re: Porting Vera scenes to openLuup
« Reply #2 on: July 18, 2017, 11:44:53 pm »
@akbooer. That would be a great feature facilitating migration from vera to openLuup. I think there is value to it.
openLuup (79 devices, 141 scenes, 19 apps) master to VeraPlus (142 zwave nodes, 8 Zigbee nodes, 221 devices,  20 scenes , 2 apps) +  Hubitat (15 Zigbee nodes) + Home-Assistant (API Integrations). Bridged to Siri and Alexa. Homewave. VeraPlus ExtRooted and mios server independent.

Offline akbooer

  • Moderator
  • Master Member
  • *****
  • Posts: 6387
  • Karma: +290/-70
  • "Less is more"
Re: Porting Vera scenes to openLuup
« Reply #3 on: July 19, 2017, 01:00:10 pm »
OK, that's done...  VeraBridge now has an action GetVeraScenes which creates local copies.

Things to note:
  • new scene numbering has 100,000 added to the local scene numbering
  • scenes are placed into the appropriate openLuup Vera room
  • these scenes don't survive a reload (unless you explicitly renumber them)
  • because of the above you can play to your heart's content and not damage anything
  • all event triggers are redirected to local devices, but disabled
  • any scenes with triggers have an extra one added, warning that they're unused!
  • all device actions are redirected to local devices
  • scene Lua is unchanged

This is available as v17.7.19 in the development branch.

All you have to do is go to the Actions tab for the VeraBridge you're interested in and press GetVeraScenes which runs a job (very quickly.)  switch immediately to the Scenes page and you should find them

Would appreciate feedback on this from anyone, but especially @a-lurker who has such experience of doing this the hard way.
3x Vera Lite-UI5/Edge-UI7, 25x Fibaro, 23x TKB, 9x MiniMote, 2x NorthQ Power, 2x Netatmo, 1x Foscam FI9831P, 9x Philips Hue,
Razberry, MySensors Arduino, HomeWave, AltUI, AltHue, DataYours, Grafana, openLuup, ZWay, ZeroBrane Studio.

Offline rafale77

  • Community Beta
  • Hero Member
  • ******
  • Posts: 1749
  • Karma: +101/-27
  • HA ≠ IoT as a blue sky is cloudless.
Re: Porting Vera scenes to openLuup
« Reply #4 on: July 19, 2017, 06:57:21 pm »
Just tested and I am seeing 2 problems:

1. The duplicate scene (the ones with triggers?) come up with the correct name but completely empty. Meaning there is no action, trigger or Luup code
2. Without renumbering any scene, I did a luup reload which wiped out the scenes with a content but not the empty duplicates.

edit:

Came to realize that the empty scenes are actually bridged to the scenes on the vera. So no wonder they survived. I did not pay attention to the 100,000 adder. Let me test further
« Last Edit: July 19, 2017, 07:01:54 pm by rafale77 »
openLuup (79 devices, 141 scenes, 19 apps) master to VeraPlus (142 zwave nodes, 8 Zigbee nodes, 221 devices,  20 scenes , 2 apps) +  Hubitat (15 Zigbee nodes) + Home-Assistant (API Integrations). Bridged to Siri and Alexa. Homewave. VeraPlus ExtRooted and mios server independent.

Online a-lurker

  • Hero Member
  • *****
  • Posts: 872
  • Karma: +66/-8
Re: Porting Vera scenes to openLuup
« Reply #5 on: July 19, 2017, 07:26:17 pm »
Yes the overall process is not too complicated but requires considerable care when doing by hand. Probably spent more time on writing the post above. ;) However fine adjustments are still needed, here & there.

Quote
these scenes don't survive a reload (unless you explicitly renumber them)
Just out of interest how do you renumber scenes? And as side line: in Vera you end up with ID numbering holes as scenes are added & deleted. When a new scene is created, does openLuup look for ID holes and fill them first. If not, can I request that it does. Not so simple for devices because of the unknown amount of children that may be added, along with the parent.

Quote
NOTE you can use the same concept for porting devices...

Do I write a post on this or just wait till you've done it programmatically :D
« Last Edit: July 19, 2017, 07:32:35 pm by a-lurker »

Offline akbooer

  • Moderator
  • Master Member
  • *****
  • Posts: 6387
  • Karma: +290/-70
  • "Less is more"
Re: Porting Vera scenes to openLuup
« Reply #6 on: July 20, 2017, 06:08:27 am »
Probably spent more time on writing the post above. ;)

...I probably spent about the same writing the code, so if a couple more people use it then that will all have been worthwhile!

Quote
Just out of interest how do you renumber scenes?

Use the 'Copy' button on the Scene and it will create a new scene, with a low scene number, called 'Clone of XXX'.

Quote
And as side line: in Vera you end up with ID numbering holes as scenes are added & deleted. When a new scene is created, does openLuup look for ID holes and fill them first. If not, can I request that it does.

That is exactly what it does already.  A new scene, created normally or cloned as described above, will have the next-highest available scene number.

Quote
Not so simple for devices because of the unknown amount of children that may be added, along with the parent.

Indeed so.  So many things use device numbers it's folly to try and change too much.

Quote
Quote
NOTE you can use the same concept for porting devices...

Do I write a post on this or just wait till you've done it programmatically :D

See the above comment!  I think you're referring to a PM you sent on April 1st, so I ignored it  ;)   I think your use case (in fact, about moving plugins, not devices) refers to a few very large plugins with complex configuration and many child devices.  This is a minefield and, aside from the very brave (like yourself) I would recommend starting from scratch by installing the plugin and configuring it as per normal.
3x Vera Lite-UI5/Edge-UI7, 25x Fibaro, 23x TKB, 9x MiniMote, 2x NorthQ Power, 2x Netatmo, 1x Foscam FI9831P, 9x Philips Hue,
Razberry, MySensors Arduino, HomeWave, AltUI, AltHue, DataYours, Grafana, openLuup, ZWay, ZeroBrane Studio.