We have moved at community.getvera.com

Author Topic: New dataMine graphing and logging plugin  (Read 287583 times)

Offline Aaron

  • Hero Member
  • *****
  • Posts: 1985
  • Karma: +61/-193
Re: New dataMine graphing and logging plugin
« Reply #315 on: June 25, 2012, 05:28:49 pm »
Also, do you think you could add a feature on the graph to use external data.... I'd like to add outside temperature data and compare it against run times, on peak and off peak - over time.
dataMine can log any variables from any devices installed in Vera. You can therefore install the Google Weather plugin to get outside weather, and log the temperature from that (or any other thermometer). Is that what you mean?

I'll try it out. Thank you... I did not know about the goog weather plugin :) 

The idea is to log the hourly temps and humidity - and compare it against the cycle times of the AC unit during those hours to see if there are any correlations. Thus, how the outside temps/humidity affect the indoor temp rises and cycle times to drop the home indoor temp 2 degrees.

should be interesting!



Offline garrettwp

  • Master Member
  • *******
  • Posts: 6371
  • Karma: +227/-128
  • Vera 3, Lite, ISY994
Re: New dataMine graphing and logging plugin
« Reply #316 on: June 25, 2012, 07:34:05 pm »
From what I gather, the trane graph data is available to be pulled from the thermostat. There is no zwave command to retrieve the data.

- Garrett

Offline nickrwym

  • Full Member
  • ***
  • Posts: 139
  • Karma: +0/-0
Re: New dataMine graphing and logging plugin
« Reply #317 on: June 26, 2012, 09:51:56 am »
Chris, I like the sound of the new Datamine but am reluctant to go to UI5. Are there plans in the pipeline to make the new version work on UI4?

Keep up the great work

Nick

Offline garrettwp

  • Master Member
  • *******
  • Posts: 6371
  • Karma: +227/-128
  • Vera 3, Lite, ISY994
Re: New dataMine graphing and logging plugin
« Reply #318 on: June 26, 2012, 12:05:11 pm »
Chris, I like the sound of the new Datamine but am reluctant to go to UI5. Are there plans in the pipeline to make the new version work on UI4?

Keep up the great work

Nick

Please see this post:

http://forum.micasaverde.com/index.php/topic,7750.msg75507.html#msg75507

- Garrett

Offline Chris

  • Moderator
  • Hero Member
  • *****
  • Posts: 821
  • Karma: +3/-0
Re: New dataMine graphing and logging plugin
« Reply #319 on: June 26, 2012, 02:10:03 pm »
Chris, I like the sound of the new Datamine but am reluctant to go to UI5. Are there plans in the pipeline to make the new version work on UI4?

Hi Nick,
Yes, I'll look into this shortly. It shouldn't be a major issue, but I've been spending time fighting with the java libraries I'm using - I seem to have uncovered a number of bugs in it and this is stopping me releasing a version with the configuration embedded in the GUI.

This of course doesn't impact UI4, it's just a matter of time (and work has taken me abroad again for a week or so  >:()

Chris

Offline nickrwym

  • Full Member
  • ***
  • Posts: 139
  • Karma: +0/-0
Re: New dataMine graphing and logging plugin
« Reply #320 on: June 26, 2012, 02:39:37 pm »
I know the feeling, my life has been rather hectic. As long as it's on the way I'll hold on, this time of year Vera is doing very little

Nick

Offline Chris

  • Moderator
  • Hero Member
  • *****
  • Posts: 821
  • Karma: +3/-0
Re: New dataMine graphing and logging plugin
« Reply #321 on: June 26, 2012, 02:43:17 pm »
I know the feeling, my life has been rather hectic. As long as it's on the way I'll hold on, this time of year Vera is doing very little

Yep - I'll sort out the UI4 version. I just want to get a "real" version running first to avoid having to make too many modifications to the Lua files. The UI side will be the same, so if I can get the backend Lua sorted "once", then I can concentrate on a single UI...

Cheers
Chris

Offline Lonestar10

  • Full Member
  • ***
  • Posts: 226
  • Karma: +8/-2
Re: New dataMine graphing and logging plugin
« Reply #322 on: July 02, 2012, 03:35:24 pm »
Chris,

This looks like a terrific plugin.  Unfortunately (sadly?), some of us don't know a thing about installing codes.  Please let us know when you have a "plug-n-play" version available of UI5 or UI4.

Thanks for your hard work!

IU7 VeraPlus - Ecobee 3 x2 - Kwikset - Nest - GE & Leviton Switches - Alexa

Offline Chris

  • Moderator
  • Hero Member
  • *****
  • Posts: 821
  • Karma: +3/-0
Re: New dataMine graphing and logging plugin
« Reply #323 on: July 02, 2012, 06:08:31 pm »
Chris,

This looks like a terrific plugin.  Unfortunately (sadly?), some of us don't know a thing about installing codes.  Please let us know when you have a "plug-n-play" version available of UI5 or UI4.

Thanks for your hard work!
I agree it's not the easiest plugin to install. Hopefully once the new Vera software is release, it will be easier (I can remove 2 steps out of the process). I'm also trying to automate/simplify the USB installation, and if I can do that, then it should be completely plug and play on UI5. Hopefully this will all be soon........

UI4 will always be partly manual since there just isn't the same functionality.

Chris

Offline Chris

  • Moderator
  • Hero Member
  • *****
  • Posts: 821
  • Karma: +3/-0
Re: New dataMine graphing and logging plugin
« Reply #324 on: July 04, 2012, 07:38:06 am »
I have managed to put all the configuration into dataMine, such that all the messing around loading libraries, and installing USB devices is now removed / simplified. Once the updated Vera software is available, the installation process will be -:

  • Install the app as usual through UI5 the app install tab
  • Format a USB stick (on Windows works fine for me)
  • Plug the USB stick into Vera
  • Configure up one variable in the dataMine properties
  • Restart Lua

The variable that is configured in dataMine is the location of the USB drive. Once dataMine knows this, it checks to see if the USB drive is mounted, and if it isn't, it mounts it. This should only happen the first time dataMine starts each time Vera is powered on, but it takes no time at all anyway.

Hopefully this will prove to be reliable, and it will certainly simplify installation (which I agree right now is not the easiest). I'm also putting together a PDF explaining this (and anything else I can think of) rather than rely on sifting through the forum (which is getting very long).

The new version will also fix/improve on a few issues with the GUI, so hopefully the new Vera software will be released soon!

Once I get the UI5 version out, I'll look at putting this back into UI4. This will simplify the UI4 installation, however as there's no app installer for UI4, it will always be a manual process - I can hopefully just make it easier...

Cheers
Chris

Offline Aaron

  • Hero Member
  • *****
  • Posts: 1985
  • Karma: +61/-193
Re: New dataMine graphing and logging plugin
« Reply #325 on: July 04, 2012, 11:14:07 am »
Chris,
Really good to hear - your changes will make it easier for the masses to use and that is always a good thing!

I really like what I've been using so far, it has helped me fine tune my air conditioning usage.

Are there changes to the graphing and such you will be putting in?  Here's some things I think would be useful to most people..

save graph profiles for ease of recovery - save the channels selected, with orientation, time/date range, etc so we can quickly pull it back up
datapoint value markers - show values on the graph
set a graph range - directly set the day/time range. ths current buttons and arrows are a bit tedious ;)

on graph analysis measurements/data... this one is a bit more in depth.
-- options to auto display items like: min/max values, peak-to-peak & trough-to-trough times, etc
-- custom user markers (one or more sets of markers on single channel) can be placed in the graph, and measurements are take for time and/or value difference for eac h set

save data options... ability to save a date/time range of data from within the graph.

Offline Chris

  • Moderator
  • Hero Member
  • *****
  • Posts: 821
  • Karma: +3/-0
Re: New dataMine graphing and logging plugin
« Reply #326 on: July 04, 2012, 11:26:44 am »
save graph profiles for ease of recovery - save the channels selected, with orientation, time/date range, etc so we can quickly pull it back up
datapoint value markers - show values on the graph
set a graph range - directly set the day/time range. ths current buttons and arrows are a bit tedious ;)

on graph analysis measurements/data... this one is a bit more in depth.
-- options to auto display items like: min/max values, peak-to-peak & trough-to-trough times, etc
-- custom user markers (one or more sets of markers on single channel) can be placed in the graph, and measurements are take for time and/or value difference for eac h set

save data options... ability to save a date/time range of data from within the graph.
Yep - saving graphs is something I intend to put in. It's currently hidden, but there is another pane in the same area as the channel config and options for listing saved graphs.

Other areas I'm looking at are -:
  • how to handle history data. Once I get the configuration tab working (I'm having trouble with some bugs in the java lib) then I hope to have options on how history data is displayed (eg. min/max/av/usage) as different types of data hae different requirements.
  • I've also got part way through an energy visualisation screen to show real-time usage in a more graphical way. This is partly coded, but it needs to be kept separate due to the java library issues.

I'll start a list  ;)

Cheers
Chris

Offline Chris

  • Moderator
  • Hero Member
  • *****
  • Posts: 821
  • Karma: +3/-0
Re: New dataMine graphing and logging plugin
« Reply #327 on: July 05, 2012, 04:25:32 pm »
Since there's now a new version of Vera code available, I can now release the latest dataMine (it will probably take a day or so for MCV to approve).

This version may work if you haven't upgraded Vera, but you may also find that Vera gets into a loop where it can't install the plugin (it depends on your internet speed). If you've got a good internet connection, you'll be fine - if it's rubish (like mine), then you will need to have upgraded Vera before installing this!

If you've installed the previous version, I would recommend deleting the /www/dm directory before starting the install of the new version. This isn't 100% necessary - it will still work, but it's cleaner as I've changed the structure of the GUI to make it more extensible...

To install -:
  • Install the app as usual through UI5 the app install tab
  • Format a USB stick (on Windows works fine for me)
  • Plug the USB stick into Vera
  • Configure up one variable in the dataMine properties (see below)
  • Restart Lua

Once you've installed the plugin, you need to go to the advanced properties, and set the SetMountPount variable. For Vera3, this is /dev/sda1 for the bottom USB slot, or /dev/sdb1 for the top slot. For Vera2, unfortunately, I'm not 100% sure. I've been using /dev/scsi/host0/bus0/target0/lun0/part1 on my Vera2 (top socket I think). For Vera3Lite I'm told it is /dev/sda1. If you are using a USB hub, these may change - refer to the document for a description of how to find the mount point.

I've tested this on a newly installed Vera, so it should work fine, but if you have any issues with the installation, please let me know. I've tried to streamline it as much as possible, but there's always the possibility I've missed something.

I've attached an overview PDF with installation instructions that hopefully helps to answer some questions - it's kind of work in progress, but as I've had a couple of people PM me asking for something like this, I thought I'd put it into a PDF rather than rely on the forums.

If anyone has any questions, let me know and I'll try and answer them.

Chris
« Last Edit: July 06, 2012, 05:30:48 am by Chris »

Offline Aaron

  • Hero Member
  • *****
  • Posts: 1985
  • Karma: +61/-193
Re: New dataMine graphing and logging plugin
« Reply #328 on: July 05, 2012, 05:51:44 pm »
I updated the firmware... it wiped the dataMine channels I had selected?

all my other plugins/settings look untouched.

Offline Chris

  • Moderator
  • Hero Member
  • *****
  • Posts: 821
  • Karma: +3/-0
Re: New dataMine graphing and logging plugin
« Reply #329 on: July 05, 2012, 06:10:31 pm »
I updated the firmware... it wiped the dataMine channels I had selected?

all my other plugins/settings look untouched.
Strange - I had no issues when I updated the firmware. Now, I am running the new dataMine, which doesn't rely on Vera configuration to mount the USB, so if the firmware upgrade has changed something there, then the new dataMine will (hopefully) solve this.

It's unlikely the channels are "wiped". The configuration is stored in a file on the USB stick which (hopefully) the Vera upgrade shouldn't have affected, so it's possible that something has become "temporarily unhinged".

Chris