We have moved at community.getvera.com

Author Topic: New dataMine graphing and logging plugin  (Read 287702 times)

Offline ajlennon

  • Full Member
  • ***
  • Posts: 154
  • Karma: +0/-0
Re: New dataMine graphing and logging plugin
« Reply #150 on: December 07, 2011, 08:30:03 am »

@willstar

I don't know if this is any help, but a '<' at the start of the file could indicate you've somehow got some HTML in there, e.g. <head>

If you log into Vera over SSH and cat the file with something like cat /etc/cmh-ludl/L_DataMine1.lua then you should be able to determine if it looks correct?

Cheers, Alex

Offline Chris

  • Moderator
  • Hero Member
  • *****
  • Posts: 821
  • Karma: +3/-0
Re: New dataMine graphing and logging plugin
« Reply #151 on: December 07, 2011, 08:57:50 am »
Quote
The day and week view is ok. The month view is wrong though.
Ah - I've been found ok!  ;)
This is a partially implemented feature. It was one of the things I initially worked on, but priorities changed as people were asking for graphing non-numeric data, and better support for status variables... The issue is that processing a months data (and sometimes less if the data changes often) can take a huge amount of time, and this can cause Vera to go bye-byes (i.e. Lua restarts). So, the concept I have is to generate historical data - min/max over a period are recorded for hourly and daily data. Then, depending on zoom level dataMine selects the appropriate data set. This speeds things up a huge amount!

So, this is largely implemented, but it needs to run each day - I need to hook this in somehow. If you want to try it out, then go to the config screen, go to the variable, and click on the little clock icon. This will tell Vera to go away and generate the historical data. It can take a few "seconds" (or tens of seconds) depending on the size of the data set. Doing this will make the monthly views work.

I'll try and sort this out next...

Quote
You need to ad a symbolic link to the custom-user-startup file in "/etc/rc.d": ln -s source_file link_name
I'm far from an expert in Linux, and know even less about OpenWRT, but I found something that said that the by running "/etc/init.d/custom-user-startup enable" it would automatically put in the link - it seemed to work for me. If I understand correctly, it looks at the start of the file to get the priority, and then generates the correct name in the rc.d directory.


Offline ajlennon

  • Full Member
  • ***
  • Posts: 154
  • Karma: +0/-0
Re: New dataMine graphing and logging plugin
« Reply #152 on: December 07, 2011, 09:02:29 am »
Quote
If you want to try it out, then go to the config screen, go to the variable, and click on the little clock icon.

That works thanks :)

Offline Qrp

  • Sr. Newbie
  • *
  • Posts: 27
  • Karma: +0/-0
Re: New dataMine graphing and logging plugin
« Reply #153 on: December 07, 2011, 09:24:19 pm »
@chixxi Thanks for the information on creating the symbolic link, it worked

@Chris Might be good to put that in the install wiki in case others run into the same difficulty.

Thanks to everyone who helped, and to Chris for putting this together in the first place.

Chip

Offline willstar

  • Sr. Newbie
  • *
  • Posts: 29
  • Karma: +0/-0
Re: New dataMine graphing and logging plugin
« Reply #154 on: December 09, 2011, 10:22:27 pm »

@willstar

I don't know if this is any help, but a '<' at the start of the file could indicate you've somehow got some HTML in there, e.g. <head>

If you log into Vera over SSH and cat the file with something like cat /etc/cmh-ludl/L_DataMine1.lua then you should be able to determine if it looks correct?

Cheers, Alex

Thanks Alex - I had a look and there is some html there - <head> etc - although that appears in all the versions of the lua files (datamine) that I have tried copying in via SCP - weird.

I'm thinking I might start from scratch (again - did this before) but I think there's a simpler solution somewhere. If nothing else it is helping me up that steep learning curve!

Offline chixxi

  • Hero Member
  • *****
  • Posts: 1036
  • Karma: +37/-14
Re: New dataMine graphing and logging plugin
« Reply #155 on: December 10, 2011, 05:49:31 am »
I tried your plugin in UI5. See Screenshots attached.

It is "almost" working. I did no re install the plugin after the update. The device is still available and I can choose channels etc. However, when trying to view any data (Screenshot2) it seems that the graphs are not generated and the view drops back to Screenshot5. Trying to get something out of the log right now:

Code: [Select]
50 12/10/11 11:48:50.289 luup_log:24: dataMine: ID: 2 Start: 1323427730 Stop: 1323514130 Sample: 123 <0x8008>

50 12/10/11 16:36:19.112 luup_log:24: dataMine: Initialising Data Mining System <0x402>
01 12/10/11 16:36:19.159 LuaInterface::CallFunction_Startup-1 device 24 function initialise failed /usr/lib/lua/json.lua:122: Unterminated JSON encoded object found at position in [] <0x402>
01 12/10/11 16:36:19.160 LuImplementation::StartLua running startup code for 24 I_DataMine1.xml failed <0x402>
« Last Edit: December 10, 2011, 10:39:37 am by chixxi »
Developer of Plugins: Virtual Switch, Variable Container, Popcorn Hour Remote, Vacation Ghost. => PLUGINS HAVE BEEN UNPUBLISHED BY ME.

Offline garrettwp

  • Master Member
  • *******
  • Posts: 6371
  • Karma: +227/-128
  • Vera 3, Lite, ISY994
Re: New dataMine graphing and logging plugin
« Reply #156 on: December 10, 2011, 07:57:56 am »
It is most likely due to the fact that the plugin needs to be updated for UI5. Changes where made on how the plugs work with UI5. Since Chris is not a Beta tester, he would not have known this or been able to make the changes. So Chris have to make the proper changes to make it work.

- Garrett


UI to UI5 migration
has some info on changes especially for plugins.
« Last Edit: December 10, 2011, 08:03:57 am by garrettwp »

Offline chixxi

  • Hero Member
  • *****
  • Posts: 1036
  • Karma: +37/-14
Re: New dataMine graphing and logging plugin
« Reply #157 on: December 10, 2011, 08:06:58 am »
It is most likely due to the fact that the plugin needs to be updated for UI5. Changes where made on how the plugs work with UI5. Since Chris is not a Beta tester, he would not have known this or been able to make the changes. So Chris have to make the proper changes to make it work.

Yes, please don't get me wrong. I was expecting that some things would not work after the update, just wanted to let chris know. I can offer my help if he needs somebody to test. No need to hurry. I am great full for every plugin developed.
Developer of Plugins: Virtual Switch, Variable Container, Popcorn Hour Remote, Vacation Ghost. => PLUGINS HAVE BEEN UNPUBLISHED BY ME.

Offline Chris

  • Moderator
  • Hero Member
  • *****
  • Posts: 821
  • Karma: +3/-0
Re: New dataMine graphing and logging plugin
« Reply #158 on: December 10, 2011, 08:11:01 am »
MCV have sent me UI5, but I've been busy with work over the past week, and I've also had builders start an extension on my house so I've not had the change to look at it yet. I'll try and have a look tonight...

Offline garrettwp

  • Master Member
  • *******
  • Posts: 6371
  • Karma: +227/-128
  • Vera 3, Lite, ISY994
Re: New dataMine graphing and logging plugin
« Reply #159 on: December 10, 2011, 08:30:40 am »
Looking forward to the update Chris. I would like to get it running on my box. But since I have been running UI5 for quite a while, I could not do so.

- Garrett

Offline ajlennon

  • Full Member
  • ***
  • Posts: 154
  • Karma: +0/-0
Re: New dataMine graphing and logging plugin
« Reply #160 on: December 10, 2011, 04:04:48 pm »

I just updated to UI5 and all my logged data files have gone. Might be worth backing them up yourselves before updating just in case

Offline Chris

  • Moderator
  • Hero Member
  • *****
  • Posts: 821
  • Karma: +3/-0
Re: New dataMine graphing and logging plugin
« Reply #161 on: December 10, 2011, 04:23:04 pm »
Quote
I just updated to UI5 and all my logged data files have gone. Might be worth backing them up yourselves before updating just in case
Maybe the USB stick wasn't remapped? Or maybe it wasn't mapped properly previously and now that the system has updated the files have gone.

I "upgraded" to UI5 this afternoon, and I can no longer access my Vera at all, so unfortunately I'm unable to work on this until I get some support from MCV. Sorry.

Offline ajlennon

  • Full Member
  • ***
  • Posts: 154
  • Karma: +0/-0
Re: New dataMine graphing and logging plugin
« Reply #162 on: December 10, 2011, 04:26:50 pm »
It depends on your definition of correctly mapped. It was mapped, it worked. imho software should not be deleting files on external media.

That said this is the least of my worries as very little now appears to be working under UI5. Sigh. I knew I shouldn't have upgraded. Every time.

Offline utz

  • Sr. Member
  • ****
  • Posts: 274
  • Karma: +0/-0
Re: New dataMine graphing and logging plugin
« Reply #163 on: December 22, 2011, 09:35:37 am »
Chris,

any news on dataMine on UI5? I use this for logging which I like, so I only want to upgrade to UI5 if this one is working...

Thanks.

Offline Chris

  • Moderator
  • Hero Member
  • *****
  • Posts: 821
  • Karma: +3/-0
Re: New dataMine graphing and logging plugin
« Reply #164 on: December 22, 2011, 11:01:52 am »
Sorry - no UI5 as yet... I had a load of problems when I upgraded, so I'm back on UI4. I've ordered a Vera3 which should be here very early in the new year and I hope to get dataMine working on UI5 then.

Chris