We have moved at community.getvera.com

Author Topic: New dataMine graphing and logging plugin  (Read 287698 times)

Offline Aegis

  • Full Member
  • ***
  • Posts: 141
  • Karma: +1/-0
Re: New dataMine graphing and logging plugin
« Reply #765 on: January 08, 2014, 09:08:10 am »
Thanks... that seems to be related to the problem...

I ended up doing the following....

1. Removed plugin and USB stick.
2. Powered off
3. Inserted USB stick
4. Powered on
5. Manually created dataMine folder in root directory
6. Installed plugin
7. Launched GUI and set UUID

Because I actually backed up the contents of the USB stick before all of the troubleshooting, I copied back the config and 1 year of associated data.  After completing step 7 above, the 20 channels are back!

Either way, it looks like something related to the dataMine directory.  Manually creating it before installing seemed to do the trick on my end.

Thanks for the help.

I will try this. At what point in your process did u copy back the configuration?

Offline Aegis

  • Full Member
  • ***
  • Posts: 141
  • Karma: +1/-0
Re: New dataMine graphing and logging plugin
« Reply #766 on: January 09, 2014, 08:27:07 am »
If anyone can look at this debug file to review the errors, I would greatly appreciate it.  It just seems like the program doesn't find the USB, however, it seems to work for an hour or so after I set it up.  I set the UUID using PuTTy and the blkid command.  I populate the SetMountPoint with /dev/sda1 for the Vera Lite.  I labeled the drive dataMine in Win7.  I tried to label it while plugged into the Vera Lite and it said it was already created.  The only thing I wonder about is when the drive is plugged into the Vera, when you mkdir, how do you ensure it is making the directory on the USB and not on the Vera?

The times and dates on the USB files make no sense.  They haven't even occurred yet.
« Last Edit: January 09, 2014, 08:31:23 am by Aegis »

Offline a-lurker

  • Hero Member
  • *****
  • Posts: 872
  • Karma: +66/-8
Re: New dataMine graphing and logging plugin
« Reply #767 on: January 09, 2014, 05:10:41 pm »
The report looks far improved indicating you have now successfully mounted a 4GB stick.

"how do you ensure it is making the directory on the USB " That why in the instructions I linked to:  it said, create the directory when the stick is unplugged. Likewise you can't create a directory on it, if it's not mounted. To be sure - plug it into another PC and check there is no directory. From the log however it all looks OK.

"The times and dates on the USB files make no sense". What do you mean here - they look OK.

If dataMine continually stops or restarts it may be something you are trying to log causing problems. I would stop logging all and add them back in one by one.

Offline Aegis

  • Full Member
  • ***
  • Posts: 141
  • Karma: +1/-0
Re: New dataMine graphing and logging plugin
« Reply #768 on: January 10, 2014, 06:53:27 am »
The report looks far improved indicating you have now successfully mounted a 4GB stick.

"how do you ensure it is making the directory on the USB " That why in the instructions I linked to:  it said, create the directory when the stick is unplugged. Likewise you can't create a directory on it, if it's not mounted. To be sure - plug it into another PC and check there is no directory. From the log however it all looks OK.

"The times and dates on the USB files make no sense". What do you mean here - they look OK.

If dataMine continually stops or restarts it may be something you are trying to log causing problems. I would stop logging all and add them back in one by one.


Times and dates were actually in the future to when the activity actually occurred.

I'm monitoring a couple Tstats where 4 variables (temp & modestate) haven't changed for days, so maybe that is the issue.  Thank you.

Offline Aegis

  • Full Member
  • ***
  • Posts: 141
  • Karma: +1/-0
Re: New dataMine graphing and logging plugin
« Reply #769 on: January 12, 2014, 02:56:30 pm »
I believe this plugin is in fact working for me now.  My confusion was over the loss of up-to-date data.  What made me take a deeper look was the dataMine Overview pdf file.  I read this file several times, but what I finally realized today was that the images even show the icons for tick, yellow exclamation & red exclamation for several channels.  I was taking all my red exclamations as complete loss of communication, when in fact I believe that if the device (thermostats in my case) do not change state over a period of time, no new information is sent.  So a couple furnaces running non stop during the deep freeze will appear to have "lost communications."  I uninstalled everything today, deleted all dataMine directories then reinstalled.  All 4 channels I set up would not show anything but red exclamation for quite some time.  I then changed the setting on the thermostat and the channel immediately registered fresh data with a tick mark indicator.

I should add a couple notes of things that I came to understand despite my best efforts at comprehension of what is already documented and what people told me:

1) You format the USB stick and lable it dataMine.  You do not have to also create a folder on it called dataMine.
2) If the plugin doesn't create a dataMine folder on Vera, you have to create it.  You can do this with PuTTy or SCP.  It's easier to verify its location in SCP as that is Windows based.  When I created this folder in root with PuTTy it didn't work.  When I created it in /<root> (after clicking the up arrow) in SCP, it worked.  I don't know the difference between the two locations.
« Last Edit: January 12, 2014, 03:40:41 pm by Aegis »

Offline C4Vette

  • Full Member
  • ***
  • Posts: 157
  • Karma: +3/-8
Re: New dataMine graphing and logging plugin
« Reply #770 on: January 14, 2014, 05:33:58 am »
I should add a couple notes of things that I came to understand despite my best efforts at comprehension of what is already documented and what people told me:

1) You format the USB stick and lable it dataMine.  You do not have to also create a folder on it called dataMine.
2) If the plugin doesn't create a dataMine folder on Vera, you have to create it.  You can do this with PuTTy or SCP.  It's easier to verify its location in SCP as that is Windows based.  When I created this folder in root with PuTTy it didn't work.  When I created it in /<root> (after clicking the up arrow) in SCP, it worked.  I don't know the difference between the two locations.

note 1: Labeling is optional I discovered.
Note 2: "/" is the top-level of the directory-structure
            "/root" is a subdirectory used as the home-dir for user 'root'.
VeraLite UI7 1.7.649
Fan of: PLEG, DataYours, dutchSmartMeter, MySensors

Offline bore

  • Sr. Newbie
  • *
  • Posts: 49
  • Karma: +0/-0
Re: New dataMine graphing and logging plugin
« Reply #771 on: January 15, 2014, 11:21:40 am »
Hi,
I need a help...
Here we have 2 costs per kWh...
Between 6:00 and 22:00 the cost is 0,09255 EUR and between 22:00 - 6:00 is 0,05803 EUR...
I have an AEON Home Energy Monitor and I have the real consumption in my house in kWh, but I want to know what i need to pay...
Can you help me, please


Offline RichardTSchaefer

  • Community Beta
  • Master Member
  • ******
  • Posts: 10091
  • Karma: +764/-143
Re: New dataMine graphing and logging plugin
« Reply #772 on: January 15, 2014, 12:27:07 pm »
I do not have an AEON home energy monitor.
If there is a variable in the ADVANCED tab of the monitor with cost per KWh then you can do a simple PLEG (or use LUA code if you wish)

PLEG:
DaySchedule:     ON  Sunday-Saturday at 6:00  Off  Interval 16:00:00

Condition:
RATE                 DaySchedule ? 0.99255 : 0.05803

Action:
RATE                 SetVariable   ServiceID, VariableName, DeviceID, Value

You need to use the Advanced Tab of actions  to set the Rate. If they have an action to set the rate ... you should use that.
Otherwise you might also have to force a LUA restart to cause the Monitor to use the new variable value. 
Setting up the action will take a little trial and error to sort out the strategy.  PM me if you decide to proceed here.





Offline bore

  • Sr. Newbie
  • *
  • Posts: 49
  • Karma: +0/-0
Re: New dataMine graphing and logging plugin
« Reply #773 on: January 15, 2014, 01:07:18 pm »
AEON has only kWh and they are from the first day that I insert it...
No variable for the costs...
May be I need to learn PLEG...
10x


Offline RichardTSchaefer

  • Community Beta
  • Master Member
  • ******
  • Posts: 10091
  • Karma: +764/-143
Re: New dataMine graphing and logging plugin
« Reply #774 on: January 15, 2014, 03:19:46 pm »
Actually ... you could have PLEG generate and update a variable that is KWh * Cost ... that would get updated every time the KWh value was updated.

Offline axbl

  • Newbie
  • *
  • Posts: 13
  • Karma: +0/-0
Re: New dataMine graphing and logging plugin
« Reply #775 on: January 20, 2014, 06:20:12 am »
Hi,
is it possible to integrate the stored graphs directly into the UI5 ?

Offline sle118

  • Sr. Newbie
  • *
  • Posts: 39
  • Karma: +4/-0
Re: New dataMine graphing and logging plugin
« Reply #776 on: January 21, 2014, 04:54:45 pm »
I have 2 USB sticks on my USB hub, and after I unplugged a USB to serial adapter for a CM11A X10 device and rebooted, my Vera Lite decided to use the 2nd usb stick as the log storage area.

root@MiOS_35016391:/# mount
rootfs on / type rootfs (rw)
/dev/root on /rom type squashfs (ro,relatime)
proc on /proc type proc (rw,noatime)
sysfs on /sys type sysfs (rw,noatime)
tmpfs on /tmp type tmpfs (rw,nosuid,nodev,noatime)
tmpfs on /dev type tmpfs (rw,noatime,size=512k,mode=755)
devpts on /dev/pts type devpts (rw,noatime,mode=600)
/dev/mtdblock7 on /overlay type jffs2 (rw,noatime)
overlayfs:/overlay on / type overlayfs (rw,noatime,lowerdir=/,upperdir=/overlay)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
/dev/sda1 on /tmp/log/cmh type ext3 (rw,noatime,nodiratime,errors=continue,barrier=0,data=ordered)
/dev/mtdblock8 on /mios type squashfs (ro,relatime)
none on /proc/bus/usb type usbfs (rw,relatime)
/dev/sda1 on /dataMine type ext3 (rw,relatime,errors=continue,barrier=0,data=ordered)

Luckily, the device had already been formatted so I did not loose any of my dataMine data.  The issue, however, is that now the logs as well as the dataMine database are stored in the same USB stick, and I might just loose the data at some point in time when the Vera decides it's time to format it.

I cold swear that /dev/sda1 was the vera log stick and /dev/sdb1 the dataMine stick. Somewhere along the line, though, it seems the OS actually reversed them after a reboot. This happened even if I did not move the sticks in different ports.

Does anyone have an explanation yet?

P.S. am I the only one unable to identify the verification letters to post on this forum in a single attempt??

Offline LightsOn

  • Hero Member
  • *****
  • Posts: 754
  • Karma: +4/-3
Re: New dataMine graphing and logging plugin
« Reply #777 on: January 21, 2014, 05:33:11 pm »
Hi @sle118,

Go back a few posts to see fix. 2 USB sticks has issues as you found.  Soloution is on using ciffs and nas.

Sent from my HTC HD2 using Tapatalk 2


Offline sle118

  • Sr. Newbie
  • *
  • Posts: 39
  • Karma: +4/-0
Re: New dataMine graphing and logging plugin
« Reply #778 on: January 22, 2014, 08:00:26 am »
Go back a few posts to see fix. 2 USB sticks has issues as you found.  Soloution is on using ciffs and nas.

Thanks for the quick reply.

I thought I could just edit fstab to mount by UUIDs rather than by device, but after a quick search I found out that this solution would also eventually fail. 

So basically, as I understand it, the issue is that multiple usb sticks on a hub (I have 2 of the same brand/model) are not always detected in the same order by the OS and this means one stick may appear as /dev/sda at one time, and /dev/sdb at a different time after a reboot or after another device was added to/removed from the hub (for example a USB/Serial adapter).

I'll leave a link to a forum topic that discuss this for others having the same issue
http://forum.micasaverde.com/index.php/topic,16452.0.html

Offline LightsOn

  • Hero Member
  • *****
  • Posts: 754
  • Karma: +4/-3
Re: New dataMine graphing and logging plugin
« Reply #779 on: January 22, 2014, 08:13:14 am »
I think its fair to say that's correct.  Some have had things working for some time and some not but all in all its safe to say that it seems unstable on such a solution.  the ciffs / Nas worjk around appears to work well long term.

good luck :-)