We have moved at community.getvera.com

Author Topic: openLuup : device variables and timestamp  (Read 4068 times)

Offline vosmont

  • Beta Testers
  • Hero Member
  • *****
  • Posts: 687
  • Karma: +60/-8
openLuup : device variables and timestamp
« on: November 09, 2015, 02:25:50 am »
Hello,

it seems that the timestamp of the last update of a device variable is not stored ?
After a reload, "luup.variable_get" doesn't return the right timestamp, for a value that has changed before the restart (seems to be the timestamp of the reload).

Offline akbooer

  • Moderator
  • Master Member
  • *****
  • Posts: 6387
  • Karma: +291/-70
  • "Less is more"
Re: openLuup : device variables and timestamp
« Reply #1 on: November 09, 2015, 02:43:59 am »
Yes, that's true.  It's actually for the same reason that the stored device json file is not retained between restarts.  I'm refactoring the device creation code to fix this, but in the mean time, console yourself with the fact that openLuup does not spontaneously reload like Vera does, so in a running environment, this is not much of an issue. ;)
3x Vera Lite-UI5/Edge-UI7, 25x Fibaro, 23x TKB, 9x MiniMote, 2x NorthQ Power, 2x Netatmo, 1x Foscam FI9831P, 9x Philips Hue,
Razberry, MySensors Arduino, HomeWave, AltUI, AltHue, DataYours, Grafana, openLuup, ZWay, ZeroBrane Studio.

Offline vosmont

  • Beta Testers
  • Hero Member
  • *****
  • Posts: 687
  • Karma: +60/-8
Re: openLuup : device variables and timestamp
« Reply #2 on: November 09, 2015, 03:36:14 am »
 :) That's true.

It's just I'm doing a lot of tests with start and restart, and I wasn't understanding why the timestamp was not as it should be.

Offline greenberry

  • Sr. Newbie
  • *
  • Posts: 37
  • Karma: +0/-0
Re: openLuup : device variables and timestamp
« Reply #3 on: December 01, 2015, 02:48:01 pm »
Yes, that's true.  It's actually for the same reason that the stored device json file is not retained between restarts.  I'm refactoring the device creation code to fix this, but in the mean time, console yourself with the fact that openLuup does not spontaneously reload like Vera does, so in a running environment, this is not much of an issue. ;)

Hi,
Thanks for openLuup!  I have it running on my bbg with (almost) no hassle.
Would the above also be the reason for not having the Thinkspeak settings/variables within AltUi after a reboot?
Or/and it this fixed in release 6?

Sorry if this isn't the correct thread, but couldn't find a more suitable one, and didn't want to create a new one, just for this.
Thnx & grtz,
Berry.

Offline akbooer

  • Moderator
  • Master Member
  • *****
  • Posts: 6387
  • Karma: +291/-70
  • "Less is more"
Re: openLuup : device variables and timestamp
« Reply #4 on: December 01, 2015, 05:37:10 pm »
I'm not aware of this being a problem but, to be honest, I haven't used Thingspeak.  So far as I know the settings are saved in device variables which are restored after reload. 

Do you actually mean reboot or reload?  It's possible for some settings not to have been saved if recently changed and the the system is rebooted without exiting cleanly.  The whole system state is, as on Vera, checkpointed every six minutes.
3x Vera Lite-UI5/Edge-UI7, 25x Fibaro, 23x TKB, 9x MiniMote, 2x NorthQ Power, 2x Netatmo, 1x Foscam FI9831P, 9x Philips Hue,
Razberry, MySensors Arduino, HomeWave, AltUI, AltHue, DataYours, Grafana, openLuup, ZWay, ZeroBrane Studio.

Offline greenberry

  • Sr. Newbie
  • *
  • Posts: 37
  • Karma: +0/-0
Re: openLuup : device variables and timestamp
« Reply #5 on: December 02, 2015, 03:02:42 pm »
Hi.
Ok thnx, I will test again with the latest release and make sure there's at least 6 minutes past before I reboot my bbg.
The reason I ask,  it's quite a hassle to manually input all the read api keys and channel ids, even for a few devices with a few properties.
Reboot happens of a power lost and because I'm still in progress to get my beaglebone working with a mysensors gateway (is wifi gateway also possible btw?)

How can I check (in the altui log?) if they are being saved

Thnx again!
Regards,
Berry.

Offline akbooer

  • Moderator
  • Master Member
  • *****
  • Posts: 6387
  • Karma: +291/-70
  • "Less is more"
Re: openLuup : device variables and timestamp
« Reply #6 on: December 02, 2015, 03:19:47 pm »
I will test again with the latest release and make sure there's at least 6 minutes past before I reboot my bbg.
You don't have to wait 6 minutes, just do one of:
  • AltUI > Misc > Reload Luup Engine, wait for the restart and then nuke it, or
  • make the URL request http://<YourVeraIP>:3480/data_request?id=exit through a browser
either one of these will have forced a checkpoint of the user_data.

Quote
The reason I ask,  it's quite a hassle to manually input all the read api keys and channel ids, even for a few devices with a few properties.
Yes, that's true.  The whole raison d'etre of openLuup is to avoid this hassle!

Quote
I'm still in progress to get my beaglebone working with a mysensors gateway (is wifi gateway also possible btw?)
WiFi gateway to MySensors?  There's isn't a plugin for that, so at the moment I only support Ethernet.  You have a MySensors WiFi gateway?  What's the protocol?

Quote
How can I check (in the altui log?) if they are being saved

There is a device variable VariablesToWatch under AltUI which should contain the data.  If it's there, it's saved in user_data, as above.
« Last Edit: December 03, 2015, 02:41:14 am by akbooer »
3x Vera Lite-UI5/Edge-UI7, 25x Fibaro, 23x TKB, 9x MiniMote, 2x NorthQ Power, 2x Netatmo, 1x Foscam FI9831P, 9x Philips Hue,
Razberry, MySensors Arduino, HomeWave, AltUI, AltHue, DataYours, Grafana, openLuup, ZWay, ZeroBrane Studio.

Offline greenberry

  • Sr. Newbie
  • *
  • Posts: 37
  • Karma: +0/-0
Re: openLuup : device variables and timestamp
« Reply #7 on: December 03, 2015, 03:16:22 pm »
You don't have to wait 6 minutes, just do one of:
  • AltUI > Misc > Reload Luup Engine, wait for the restart and then nuke it, or
  • make the URL request http://<YourVeraIP>:3480/data_request?id=exit through a browser
either one of these will have forced a checkpoint of the user_data.
Yes, indeed thnx, I did read and use the clean exit from browser before, but forgot while fiddling with these variable settings.

Quote
WiFi gateway to MySensors?  There's isn't a plugin for that, so at the moment I only support Ethernet.  You have a MySensors WiFi gateway?  What's the protocol?
No the plugin is the same as for the serial/ethernet gateway but the arduino sketch is different, and in the plugin you just connect to ip:port
So I guess it just should work, I'll try that later.
The MySensors WiFi gateway: http://www.mysensors.org/build/esp8266_gateway

Quote
There is a device variable VariablesToWatch under AltUI which should contain the data.  If it's there, it's saved in user_data, as above.
Yes the data was there after putting them into the Thinkspeak variable settings, and also after a reload Luup now I just checked in release 6.

But after a reboot (via AltUI reboot or just stop the process from within BBG's ssh terminal) it was gone, what is strange as it did survive a reload luup....

Also I didn't have any luck getting the openLuup_reload script run at boot, but that's for another thread I assume.(Maybe nice for others too, to have this in a thread for different platforms? )

Thank you !

regards,
Berry.

EDITED:
Just did a chmod on the user_data.json file (after I checked that data wasn't saved in it) and now after a reboot the variables are saved into AltUi section VariablesToSend, but the weird (well for me..) thing is that there are not visible in the UI webpage, not in the Thinkspeak vars section within a device and not within the AltUI's VariableToSend, so how can I check if the user_data.json is correctly loaded/parsed/used?
File is in the /etc/cmh-ludl dir.
« Last Edit: December 03, 2015, 03:52:37 pm by greenberry »

Offline akbooer

  • Moderator
  • Master Member
  • *****
  • Posts: 6387
  • Karma: +291/-70
  • "Less is more"
Re: openLuup : device variables and timestamp
« Reply #8 on: December 03, 2015, 05:13:40 pm »
But after a reboot (via AltUI reboot or just stop the process from within BBG's ssh terminal) it was gone, what is strange as it did survive a reload luup....

Before I re-read your post and saw the edit I was going to say that you must be restarting the openLuup_reload process incorrectly.  For example, with a startup.lua file or the wrong user_data.json.

Quote
Also I didn't have any luck getting the openLuup_reload script run at boot, but that's for another thread I assume.(Maybe nice for others too, to have this in a thread for different platforms? )

So many to choose from!  I already know of Mac, OS, Windows (several flavours), open-WRT, Debian (several sorts) Synology, Ubuntu, ...

Quote
EDITED:
Just did a chmod on the user_data.json file (after I checked that data wasn't saved in it) and now after a reboot the variables are saved into AltUi section VariablesToSend, but the weird (well for me..) thing is that there are not visible in the UI webpage, not in the Thinkspeak vars section within a device and not within the AltUI's VariableToSend, so how can I check if the user_data.json is correctly loaded/parsed/used?
File is in the /etc/cmh-ludl dir.

So, as per above, I'm not sure what process you're running at reload, but if, for example, you hadn't done a cd to /etc/cmh-ludl/ then things would be in the wrong place (OK, I'm sure it's not that.)

You can check the user_data.json file easily enough, as you know.  You can check the in-memory version of it by doing an HTTP request for user_data. You can restart from AltUI's menu or shut down using the exit HTTP request and check the user_data.json file once again.

It is just possible that that some string parsing in all of the above process is not treating things right, but I have not seen that happen elsewhere - device variables should be saved and restored correctly so long as the system is shut down gracefully.

« Last Edit: December 04, 2015, 03:24:41 am by akbooer »
3x Vera Lite-UI5/Edge-UI7, 25x Fibaro, 23x TKB, 9x MiniMote, 2x NorthQ Power, 2x Netatmo, 1x Foscam FI9831P, 9x Philips Hue,
Razberry, MySensors Arduino, HomeWave, AltUI, AltHue, DataYours, Grafana, openLuup, ZWay, ZeroBrane Studio.

Offline greenberry

  • Sr. Newbie
  • *
  • Posts: 37
  • Karma: +0/-0
Re: openLuup : device variables and timestamp
« Reply #9 on: December 05, 2015, 07:23:56 am »
Before I re-read your post and saw the edit I was going to say that you must be restarting the openLuup_reload process incorrectly.  For example, with a startup.lua file or the wrong user_data.json.
Thanks, yes I did and I guess that was the main issue causing it,I've misunderstood before that that was needed.
Now I got my variables after a reboot.

Quote
So many to choose from!  I already know of Mac, OS, Windows (several flavours), open-WRT, Debian (several sorts) Synology, Ubuntu, ...
Yes that's great!
I meant starting a new thread for how to auto-start at boot,with you explaining it only for BBB for example and so that others can give some info how they did it for their specific platform.

As a side note It turned out that I was missing some D_..xml files etc because the get_files script wasn't working for me, it always gives me a error at line 53 even when trying to get the files from the local same machine. So I'v copied all files from the vera by hand and used pluto-lzo d on them.
error I got:
lua: openLuup_getfiles.lua:53: error reading remote directory listing
stack traceback:
        [C]: in function 'assert'
        openLuup_getfiles.lua:53: in function 'get_directory'
        openLuup_getfiles.lua:60: in function 'get_files_from'
        openLuup_getfiles.lua:79: in main chunk
        [C]: ?


The other thing I've noticed is that device icon files from device_states where not loaded,but gave some 404.
In your OpenLuup user guide it says:
/www/cmh/skins/default/icons/ into which any .png image files (device icons) you need
should go.
C. /www/cmh/skins/default/img/devices/ which should contain a symbolic link to the above icons/ directory.

then:
cd /www/cmh/skins/default/img/devices/
> ln -s /www/cmh/skins/default/icons/device_states


but shouldn't it be?:
> ln -s -T /www/cmh/skins/default/icons/ device_states
so that there's a device_states symlink pointing to the icons directory?
I did try that, and now all icon files from device_states are loaded correctly.

Thank you for all your help!

kind regards,
Berry.

Offline akbooer

  • Moderator
  • Master Member
  • *****
  • Posts: 6387
  • Karma: +291/-70
  • "Less is more"
Re: openLuup : device variables and timestamp
« Reply #10 on: December 05, 2015, 10:15:53 am »
Now I got my variables after a reboot.
Good news.

Quote
I meant starting a new thread for how to auto-start at boot,with you explaining it only for BBB for example and so that others can give some info how they did it for their specific platform.
Yes, I thought you meant that I should write one for every machine and OS under the sun!  I believe that @CudeNet has done a comprehensive configuration guide that he's going to share.

Quote
As a side note It turned out that I was missing some D_..xml files etc because the get_files script wasn't working for me, it always gives me a error at line 53 even when trying to get the files from the local same machine.

This is wrong, and odd, since I've not heard of anyone else having a problem - quite the reverse, in fact.

How are you running it?  What are you entering in response toe the machine IP prompt?

Quote
The other thing I've noticed is that device icon files from device_states where not loaded,but gave some 404.
[...]
shouldn't it be?:
> ln -s -T /www/cmh/skins/default/icons/ device_states
so that there's a device_states symlink pointing to the icons directory?

Yes, looks like I'm missing a space in there.  Thanks.
3x Vera Lite-UI5/Edge-UI7, 25x Fibaro, 23x TKB, 9x MiniMote, 2x NorthQ Power, 2x Netatmo, 1x Foscam FI9831P, 9x Philips Hue,
Razberry, MySensors Arduino, HomeWave, AltUI, AltHue, DataYours, Grafana, openLuup, ZWay, ZeroBrane Studio.

Offline greenberry

  • Sr. Newbie
  • *
  • Posts: 37
  • Karma: +0/-0
Re: openLuup : device variables and timestamp
« Reply #11 on: December 05, 2015, 04:21:31 pm »
This is wrong, and odd, since I've not heard of anyone else having a problem - quite the reverse, in fact.

How are you running it?  What are you entering in response toe the machine IP prompt?

Running in terminal ssh beaglebone green debian wheezy.
As root, cd to /etc/cmh-ludl/ , run openLuup_getfiles
Entering my local ip for vera, can ping that one successfully too?.

Offline akbooer

  • Moderator
  • Master Member
  • *****
  • Posts: 6387
  • Karma: +291/-70
  • "Less is more"
Re: openLuup : device variables and timestamp
« Reply #12 on: December 05, 2015, 04:47:48 pm »
it always gives me a error at line 53 even when trying to get the files from the local same machine.

I don't understand what you meant by that bit?  This is most strange.

3x Vera Lite-UI5/Edge-UI7, 25x Fibaro, 23x TKB, 9x MiniMote, 2x NorthQ Power, 2x Netatmo, 1x Foscam FI9831P, 9x Philips Hue,
Razberry, MySensors Arduino, HomeWave, AltUI, AltHue, DataYours, Grafana, openLuup, ZWay, ZeroBrane Studio.

Offline greenberry

  • Sr. Newbie
  • *
  • Posts: 37
  • Karma: +0/-0
Re: openLuup : device variables and timestamp
« Reply #13 on: December 05, 2015, 05:41:40 pm »
it always gives me a error at line 53 even when trying to get the files from the local same machine.

I don't understand what you meant by that bit?  This is most strange.
As a test I copied a bunch of lzo files from vera to my bbg into the www/cmh-ludl folder and entered as ip 127.0.0.1, to test if the getfiles script could read the dir from my beaglebone, just to see if it could give me more info about this. So no initially I was entering Vera's local ip in my network. 

Offline tsawyer

  • Sr. Newbie
  • *
  • Posts: 35
  • Karma: +2/-0
Re: openLuup : device variables and timestamp
« Reply #14 on: February 12, 2016, 01:13:26 pm »
I figured out the "lua: openLuup_getfiles.lua:53: error reading remote directory listing" error.   It is because local auth is enable on the Vera.  To fix, open up the /etc/lighttpd.conf and comment out mod_auth and then reload lighttpd (/etc/init.d/lighttpd restart).