We have moved at community.getvera.com

Author Topic: New dataMine graphing and logging plugin  (Read 287569 times)

Offline Chris

  • Moderator
  • Hero Member
  • *****
  • Posts: 821
  • Karma: +3/-0
Re: New dataMine graphing and logging plugin
« Reply #510 on: December 23, 2012, 02:16:11 pm »
Hi,
I need to update the overview document. This has changed now, and the config tab has been removed from the plugin itself. Now, you need to go to the GUI (there's a link off the plugin), and there is a "Configuration" tab. Select the variable you want to log, and change the "LoggingEnabled" setting to "true".
I hope this helps - please let me know if you have any more problems...

Cheers
Chris

Offline jpitts

  • Sr. Newbie
  • *
  • Posts: 29
  • Karma: +0/-0
Re: New dataMine graphing and logging plugin
« Reply #511 on: December 23, 2012, 05:48:22 pm »
Ok thanks I got it working now, the other thing that I realized that you have to do is after you enable logging you have to click off of that row and click save at top in order to enable it.
Looks good and works well so far, I remember using this back on ui4 and my vera2, it's working much better on ui5 and a vera3.

Offline jpitts

  • Sr. Newbie
  • *
  • Posts: 29
  • Karma: +0/-0
Re: New dataMine graphing and logging plugin
« Reply #512 on: December 28, 2012, 11:23:54 pm »
Is there any easy way on a sensor to show how many times it triggered for the day, I have a lot of door/motion sensors, that show 0 or 1 on the graph, but I'd really like to just know how many times it's been triggered for the day,week,month etc.

Offline Chris

  • Moderator
  • Hero Member
  • *****
  • Posts: 821
  • Karma: +3/-0
Re: New dataMine graphing and logging plugin
« Reply #513 on: December 29, 2012, 05:37:21 am »
No - at the moment you can't do this, however hour/day/month statistics is the next thing I want to add. Hopefully this will do what you want.

Cheers
Chris

Offline a-lurker

  • Hero Member
  • *****
  • Posts: 872
  • Karma: +66/-8
Re: New dataMine graphing and logging plugin
« Reply #514 on: December 29, 2012, 06:45:16 pm »
Hi Chris

DM is working well over here - thanks for that.

Was wondering if you have thought of allowing an offset to be applied to the variables. In particular I have some temperature measuring devices and they are clearly out of whack. One is too high and the other is too low. An offset could be associated with the measuring device itself but this may be asking too much, given all the other request being made of the Vera OS. Regardless, I suspect that it's best to record the raw data and correct it at will later but there's room for debate on that one. Here's an example where a graph offset has been put in place:

http://board.homeseer.com/showthread.php?t=154432

Also would be possible to turn off the spline interpolation per variable graphed and what are the options for "GraphType" currently?



Offline Chris

  • Moderator
  • Hero Member
  • *****
  • Posts: 821
  • Karma: +3/-0
Re: New dataMine graphing and logging plugin
« Reply #515 on: December 30, 2012, 05:30:49 am »
Was wondering if you have thought of allowing an offset to be applied to the variables. In particular I have some temperature measuring devices and they are clearly out of whack. One is too high and the other is too low. An offset could be associated with the measuring device itself but this may be asking too much, given all the other request being made of the Vera OS.
I tend the agree with you - this probably ought to be done at the measuring device, but this will never happen! I'll take a look at this - it's easy to do, but it will have an impact on display speed as obviously I'd need to go through all data points to add the offset...

Also would be possible to turn off the spline interpolation per variable graphed and what are the options for "GraphType" currently?
Graphtypes are -:
0   Default (spline)
1   Line
2   Scatter
3   Area

Setting the type to line will turn off spline and will also ensure data values are maintained between points. By this, I mean that if a power value goes to 0 at midnight, and then goes to 100W at 6am, the graph will show 0 between midnight and 6am rather than a diagonal line from 0 to 100...

I've actually updated the PDF guide with this and had intended to upload it to the forum yesterday, but it's crept over the 512k filesize limit so I need to work out what to do with it...

Chris

Offline mfp

  • Full Member
  • ***
  • Posts: 103
  • Karma: +3/-2
Re: New dataMine graphing and logging plugin
« Reply #516 on: December 30, 2012, 08:57:36 am »
Chris,

Great plugin!

Saw you had trouble with the file limit for the PDF Guide, here are some ideas:
1) Compress the pictures some more to get under the file limit.
2) Split the document in a userguide and installation guide.
3) Put more info on http://code.mios.com/trac/mios_datamine.

By the way is it possible to make a user option to display both the Config and Graph from within Vera instead on a separate tab in the browser?

Mark

Offline Chris

  • Moderator
  • Hero Member
  • *****
  • Posts: 821
  • Karma: +3/-0
Re: New dataMine graphing and logging plugin
« Reply #517 on: December 30, 2012, 12:20:25 pm »
Saw you had trouble with the file limit for the PDF Guide, here are some ideas:
1) Compress the pictures some more to get under the file limit.
2) Split the document in a userguide and installation guide.
3) Put more info on http://code.mios.com/trac/mios_datamine.
1) I tried, but couldn't get it below the 512k limit. 3) is my default - not actually to add more data to the web page as that's not as easy as writing a document and publishing it as a PDF (the Trak wiki software used for code.mios.com is a bit clunky) - but I could put the PDF on the webpage and link to it from the forum. 2) I like though and I might see if this works...

By the way is it possible to make a user option to display both the Config and Graph from within Vera instead on a separate tab in the browser?
Not really - this is kinda what I had in UI4. The graph was limited to the small properties window, and doing much in a config screen was a bit difficult due to real-estate limitations. I initially tried to work with this when I moved to UI5, but it just wasn't happening. While there are a few limitations with having a separate page, I think in general the benefits outweigh them.

Cheers
Chris

Offline bucko

  • Hero Member
  • *****
  • Posts: 659
  • Karma: +27/-9
Re: New dataMine graphing and logging plugin calibration ?
« Reply #518 on: January 08, 2013, 09:05:34 pm »
Hi Chris,
I have a couple of temp sensors that come built in to a Zwave plug in socket controller. The problem is that they are way off accuracy probably due to the internal heat of the controller. So I want to add an error correction to the datamine channel to adjust for this sensor error. So basically channel 2 (temp sensor) is sensor temp (-12), or whatever number I need to get this sensor closer to the actual room temp.

How can I do this?
Vera 3, 40+ Fibaro relays & dimmers, IPUX cams, Elk M1 Gold, M1XEP, M1SP, XVOR, M1RB, Elk124, Monoprice HDMI matrix & 6 zone amp, Denon AVR, Panasonic A8000, BI DV,On Controls

Offline Chris

  • Moderator
  • Hero Member
  • *****
  • Posts: 821
  • Karma: +3/-0
Re: New dataMine graphing and logging plugin
« Reply #519 on: January 09, 2013, 04:50:13 am »
I have a couple of temp sensors that come built in to a Zwave plug in socket controller. The problem is that they are way off accuracy probably due to the internal heat of the controller. So I want to add an error correction to the datamine channel to adjust for this sensor error. So basically channel 2 (temp sensor) is sensor temp (-12), or whatever number I need to get this sensor closer to the actual room temp.
Hi
This will be available in the next version which is currently awaiting approval by MCV. You will be able to set an offset per channel...

Cheers
Chris

Offline nathanhall

  • Jr. Member
  • **
  • Posts: 53
  • Karma: +0/-1
Re: New dataMine graphing and logging plugin
« Reply #520 on: January 17, 2013, 08:36:59 pm »
I just installed dataMine and (mostly) got it working but I do have a few questions as I'm very excited to start digging into this.

The dashboard doesn't appear to display all three graphs for me - it only displays the energy consumption donut and the whole-house gauge but not what looks to be a plot over time of energy usage in the manual (figure 10, page 9).   

Also - is there a way to combine variables into a single graph plot?  Say for example I want to combine several lights in one room into a single plot - is that possible?  Alternatively, is it possible to plot all energy use over time in the "chart" tab (e.g. the total wattage used over time in a single variable to show total energy usage)?

Offline Chris

  • Moderator
  • Hero Member
  • *****
  • Posts: 821
  • Karma: +3/-0
Re: New dataMine graphing and logging plugin
« Reply #521 on: January 18, 2013, 05:27:05 am »
The dashboard doesn't appear to display all three graphs for me - it only displays the energy consumption donut and the whole-house gauge but not what looks to be a plot over time of energy usage in the manual (figure 10, page 9).   
The third graph should (currently) display the "whole house" data for the past 24 hours. So, it should select the energy monitor with cat-1 (if you have more than 1, then I think it will select the first one).  Currently, I've noticed that this displays a wrong graph anyway, and this graph is really a placeholder for the next stage where I hope to have usage trends displayed here. So, I don't know that I'd get too worried about this at the moment. Oh - one other thing - this third graph is only in the latest version, so if you're a version behind, then you'll just get the other 2.

Also - is there a way to combine variables into a single graph plot?  Say for example I want to combine several lights in one room into a single plot - is that possible?  Alternatively, is it possible to plot all energy use over time in the "chart" tab (e.g. the total wattage used over time in a single variable to show total energy usage)?
This isn't something that can currently be done. I did think about making the donut graph selectable between categories and rooms, but I haven't done this. I'm happy to think about this once I've finished getting the history stuff integrated reliably (it's not so simple having to account for random Vera/Lua restarts when you're post processing data). The history may support some of what you're after in that it will derive usage stats on an hour/daily basis (and maybe month), and I then hope to support combining these in different ways (ie by category, room...). But this isn't currently planned for the raw data...

Cheers
Chris

Offline mrtrilby

  • Newbie
  • *
  • Posts: 1
  • Karma: +0/-0
Re: New dataMine graphing and logging plugin
« Reply #522 on: January 18, 2013, 12:49:23 pm »
Hello, I've been running the dataMine plugin for a while and it's fantastic, but it now seems to have stopped working. The interface just reports "Loading dataMine web application for Vera" but never actually loads. Judging by the files on the USB stick, the plug-in is still running. I thought about uninstalling and re-installing the plug-in to fix it, but I don't see it listed in the "Install Apps" page any more, so I'm a bit worried about not being able to get it back again.

Is its removal from the Vera Apps a temporary thing? Is there a simple way to fix the web application?

Thanks.

Offline Chris

  • Moderator
  • Hero Member
  • *****
  • Posts: 821
  • Karma: +3/-0
Re: New dataMine graphing and logging plugin
« Reply #523 on: January 18, 2013, 01:14:26 pm »
What version of dataMine are you using? The later versions shouldn't sit there with this screen - if they are having trouble, they should report an error instead...

When dataMine starts, it tries to download a bunch of information from Vera - some direct from Vera, and some from the dataMine plugin. That's what's happening when you get this message. The new version will report where it's failing, so I'm wondering if you're using an old version?

Normally this is indicative of the plugin not working, but if the txt files continue to update, then it's probably not this....

I would suggest -:
  • Update to the latest version if you haven't already
  • Make sure the number of parameters being logged isn't 0 (this is just another check to ensure the plugin is starting ok)
  • Get a dump of the dataMine part of the log (from WinSCP, bring up the terminal screen, and type "cat /var/log/cmh/LuaUPnP.log | grep dataMine".

You could try reinstalling - so long as you don't scrape the USB stick, everything should still be there when you restart (to be 100% sure, take a backup of the /dataMine directory by copying it to your PC). Personally, I'd like to try and understand the error if possible rather than just reinstalling so I can fix it if there's an issue, but if you want to try a reinstall, then up to you.

Cheers
Chris

Offline Poe1

  • Newbie
  • *
  • Posts: 7
  • Karma: +0/-0
Re: New dataMine graphing and logging plugin
« Reply #524 on: January 23, 2013, 07:45:57 am »
Hi Chris,

First, thanks for the app - I'm a data junkie so to me, what's a home automation system without the ability to log data?!

Anyway, your datamine plugin was working good for me for a few weeks, but now I just get the message "Downloading Graph Data" (see attachment) on the screen and nothing ever updates. This message will stay up for a while (couple minutes?) and then go away and just leave a blank graph.

I'm using plugin version 0.965 running vera thru Chrome and Datamine thru Firefox.  Any ideas what I could do to fix it?

Thanks,

-Casey