We have moved at community.getvera.com

Author Topic: New dataMine graphing and logging plugin  (Read 287702 times)

Offline AndersH

  • Sr. Newbie
  • *
  • Posts: 40
  • Karma: +0/-0
Re: New dataMine graphing and logging plugin
« Reply #690 on: October 18, 2013, 03:28:33 am »
otterlo,
I may be outside my competence here, but:

your screen capture of the advanced tab and the latest text file do not match, as far as I can see.
From the text file, it seems like you use a manual mounting, i.e. the dataMine will not mount the USB stick automatically.
50   10/17/13 23:10:21.449   luup_log:65: dataMine: Manual mounting to (/tmp/log/cmh/dataMine/) <0x2c0db680>
50   10/17/13 23:10:21.497   luup_log:65: dataMine: ERROR: Unable to open config file for read :: /tmp/log/cmh/dataMine/dataMineConfig.json <0x2c0db680>

AFAIK, your config is somewhat confused. You have to either
- let the Vera logs use the USB stick and then you will be able to share the USB between logs and dtaMine (this is how I set up mine).
- let the dataMine use the USB stick alone, but then you cannot use the /tmp/log/cmh directory, as that will be on a file system that disappears at every Vera reboot. In that case, you remove the manual mounting, and set the dataMine directory to /dataMine as instructed in the manual.
Vera Lite, SRT321 thermostat, Fibaro door sensors, switches and dimmers

Offline Otje

  • Full Member
  • ***
  • Posts: 214
  • Karma: +6/-1
Re: New dataMine graphing and logging plugin
« Reply #691 on: October 18, 2013, 06:13:33 am »
Hello

many thanks for your efforts to help me.

it is also confusing to me, exactly the same. I have disabled the logs to USB in Vera. So my feeling is that the USB is there to be used by datamine only.
I have not done a manual configuration. i have attached the advanced section of the  plugin and i feel that everything is correct.
what i can not see if the USB is correctly mounted.

i hope that the 2 screenshots will help?
VERA Plus, VERA 3 Lite , couple of PIR's, several switch units, wall mounted tablet, Sonos integration with TTS, Temperature sensors, Imperihome, Zwave LED bulbs

Offline AndersH

  • Sr. Newbie
  • *
  • Posts: 40
  • Karma: +0/-0
Re: New dataMine graphing and logging plugin
« Reply #692 on: October 18, 2013, 06:55:07 am »
From the MountLocation and MountType in the lower of your two screenshots, I think that the USB stick is mounted. The type is ext4, which is a linux filesystem. Most likely the USB stick was formatted ext4 when you tried to use it for the Vera logs.

So, I think your USB is mounted. Then, the next question is whether dataMine actually finds the directory /dataMine.

As previously mentioned, it looks from your text file that dataMine looks for data in the /tmp/logs/cmh, which is incorrect with your USB setup.

What I would reccomend, would be to log into the Vera (ssh or similar) and browse around to see if there is anything under /dataMine.
But, as you state that you are no linux expert, does that imply that you never have used linux, and never logged into your Vera previously? In that case, it might be other approaches that are better.

Are you able to start the graphical user interface (page 5 in the user manual) at all?
If so, do you see anything in the configuration tab?
Vera Lite, SRT321 thermostat, Fibaro door sensors, switches and dimmers

Offline Otje

  • Full Member
  • ***
  • Posts: 214
  • Karma: +6/-1
Re: New dataMine graphing and logging plugin
« Reply #693 on: October 18, 2013, 07:21:38 am »
Hello

I formatted the USB stick using my windows pc  on FAT32.
I am not sure if ext 4 is the right entry in this field?

i am not expert on linux but I can connect to Vera using winscp and browse the filestructure.
currently i am not at home and i have to wait until tonight.
i do remember that /datamine is available but i can not judge on the content of that directory.
i will have a look at that.

the GUI works and I can see the configuration tab and see all the devices and parameters. the only things that does not work is that it does not recognise any channel (or variables) so that there is nothing to display in the graph
VERA Plus, VERA 3 Lite , couple of PIR's, several switch units, wall mounted tablet, Sonos integration with TTS, Temperature sensors, Imperihome, Zwave LED bulbs

Offline AndersH

  • Sr. Newbie
  • *
  • Posts: 40
  • Karma: +0/-0
Re: New dataMine graphing and logging plugin
« Reply #694 on: October 18, 2013, 07:46:57 am »
Hello

I formatted the USB stick using my windows pc  on FAT32.
I am not sure if ext 4 is the right entry in this field?

i am not expert on linux but I can connect to Vera using winscp and browse the filestructure.
currently i am not at home and i have to wait until tonight.
i do remember that /datamine is available but i can not judge on the content of that directory.
i will have a look at that.

the GUI works and I can see the configuration tab and see all the devices and parameters. the only things that does not work is that it does not recognise any channel (or variables) so that there is nothing to display in the graph

The USB stick got formatted when you enabled the Vera logging on USB, I presume. But using ext4 should be no problem. The only drawback is that it might be difficult to read the USB stick if you put it into a Windows computer.

Under /dataMine you should see at least one config file with the .json extension, and a directory (I think it's called database). I'm not at home either, so I'm unable to check it out.

What happens if you enable logging (as described on page 16 in the user guide) on one variable that changes rapidly? Are you able to see any graph at all?
The reason for asking: If the dataMine does save all its data outside the USB stick, I presume it woul lose its complete setup at every reboot. But nontheless it should be able to display a graph for data that has changed several times since the reboot.
Vera Lite, SRT321 thermostat, Fibaro door sensors, switches and dimmers

Offline Peter

  • Jr. Member
  • **
  • Posts: 95
  • Karma: +0/-0
Re: New dataMine graphing and logging plugin
« Reply #695 on: October 18, 2013, 08:01:47 am »
How can I see my gaz consumption for a day during a year instead of the increasing total of gaz consumption for a day? Or is this not posible?

Offline DeltaNu1142

  • Hero Member
  • *****
  • Posts: 1153
  • Karma: +40/-67
Re: New dataMine graphing and logging plugin
« Reply #696 on: October 18, 2013, 09:01:26 am »
Clearly...  I'm still doing something wrong.  I decided to revisit this after my first UI5 attempt a few weeks ago.  I'm following the installation PDF.  My 8GB USB drive is labelled "DATAMINE" and formatted in FAT32.  I cleared the "MountLocation" field and populated the other fields as shown in Step 1.

Then I reload Luup a few times, start the dM graphing page, and get the screen shown in Step 2.  I select the UUID, go back to MiOS, reload Luup, and get a notification on the dM graphing page that Vera has restarted and I should restart the dM UI.

I close the dM page, go back to Vera, restart, then click on the dM plugin to view Advanced options.  I see the screen shown in Step 3a.  SetMountUUID has been populated, as has mountLocation.  However, check out disk info (Step 3b)...  6464 total? 6464 used?
Vera 3 @ UI5 | HomeWave | PLEG

Offline DeltaNu1142

  • Hero Member
  • *****
  • Posts: 1153
  • Karma: +40/-67
Re: New dataMine graphing and logging plugin
« Reply #697 on: October 18, 2013, 09:07:01 am »
I've confirmed through WinSCP that the datamine directory is under root.  Not sure where to go from here.

Online status is good/error status is false.  0 channels showing up on the plugin.  I don't know why the debug page shows up as it does, but I know it takes forever to load.
Vera 3 @ UI5 | HomeWave | PLEG

Offline DeltaNu1142

  • Hero Member
  • *****
  • Posts: 1153
  • Karma: +40/-67
Re: New dataMine graphing and logging plugin
« Reply #698 on: October 18, 2013, 09:25:28 am »
Just pulled the USB stick to see what was on it...  see attached.  This was all put on by dM.
Vera 3 @ UI5 | HomeWave | PLEG

Offline AndersH

  • Sr. Newbie
  • *
  • Posts: 40
  • Karma: +0/-0
Re: New dataMine graphing and logging plugin
« Reply #699 on: October 18, 2013, 10:35:29 am »
Just pulled the USB stick to see what was on it...  see attached.  This was all put on by dM.

This is looking much like I would expect on an empty system.
The fact that it shows 0 channels is also normal, provided that you didn't enable logging on any channels yet. I guess that is the problem, that you are unable to access the configuration tab to enable logging?
Vera Lite, SRT321 thermostat, Fibaro door sensors, switches and dimmers

Offline DeltaNu1142

  • Hero Member
  • *****
  • Posts: 1153
  • Karma: +40/-67
Re: New dataMine graphing and logging plugin
« Reply #700 on: October 18, 2013, 10:52:34 am »
...edited...  I think I've got it...

YEP.  My problem was I wasn't drilling down in each device to the points I want to log.  I've now got 6 channels up.  Even though the screens above show my USB UUID, I'm not convinced yet it's logged to USB.  I'll have to pull it out & look later today.

Once I'm convinced this works, I'm likely to move on to logging to Synology NAS.

Thanks!
« Last Edit: October 18, 2013, 11:11:27 am by DeltaNu1142 »
Vera 3 @ UI5 | HomeWave | PLEG

Offline Otje

  • Full Member
  • ***
  • Posts: 214
  • Karma: +6/-1
Re: New dataMine graphing and logging plugin
« Reply #701 on: October 18, 2013, 01:22:05 pm »
Hello

I formatted the USB stick using my windows pc  on FAT32.
I am not sure if ext 4 is the right entry in this field?

i am not expert on linux but I can connect to Vera using winscp and browse the filestructure.
currently i am not at home and i have to wait until tonight.
i do remember that /datamine is available but i can not judge on the content of that directory.
i will have a look at that.

the GUI works and I can see the configuration tab and see all the devices and parameters. the only things that does not work is that it does not recognise any channel (or variables) so that there is nothing to display in the graph

The USB stick got formatted when you enabled the Vera logging on USB, I presume. But using ext4 should be no problem. The only drawback is that it might be difficult to read the USB stick if you put it into a Windows computer.

Under /dataMine you should see at least one config file with the .json extension, and a directory (I think it's called database). I'm not at home either, so I'm unable to check it out.

What happens if you enable logging (as described on page 16 in the user guide) on one variable that changes rapidly? Are you able to see any graph at all?
The reason for asking: If the dataMine does save all its data outside the USB stick, I presume it woul lose its complete setup at every reboot. But nontheless it should be able to display a graph for data that has changed several times since the reboot.


Hi

I think it works for me now as well! checked everything but i believe the trick was to enable logging on the variables..... :-[
It was not until you pointed me to this that I started to understand.

thank you very much and i will try to see how everything works!
VERA Plus, VERA 3 Lite , couple of PIR's, several switch units, wall mounted tablet, Sonos integration with TTS, Temperature sensors, Imperihome, Zwave LED bulbs

Offline Otje

  • Full Member
  • ***
  • Posts: 214
  • Karma: +6/-1
Re: New dataMine graphing and logging plugin
« Reply #702 on: October 20, 2013, 06:10:53 am »
i have another question.

I created a saved graph with reference name "kamertemperatuur", so that I can call this from my favorties in the webexplorer.
I saved a graph to display outside temperatuur with duration for 1 day

when I call the graph on the following way: http://xxx.xxx.x.xx/dm/graph.html?ref=binnentemp, i then see the graph displays nicely but it does not seem to display the recent 2 hours. for example, the time is 12 o clock now and this graph always shows up and to 10 o clock.

Does anybody know why i face this? (hope it is not related to GMT time settings....)
« Last Edit: October 20, 2013, 08:22:30 am by otterlo »
VERA Plus, VERA 3 Lite , couple of PIR's, several switch units, wall mounted tablet, Sonos integration with TTS, Temperature sensors, Imperihome, Zwave LED bulbs

Offline mikee123

  • Hero Member
  • *****
  • Posts: 1521
  • Karma: +18/-11
Re: New dataMine graphing and logging plugin
« Reply #703 on: October 29, 2013, 06:06:24 pm »
This may be another case of a missing 'dataMine directory' - as indicated by "Error reading tmpfile during mountcheck"

http://forum.micasaverde.com/index.php/topic,16975.msg131509.html#msg131509

I did get datamine working by creating a datamine directory in root. I am a little worried that datamine is writing data onto Vera rather than the USB. I have this displayed on the device:

Channels
18
Disk Space Total (kb)
518692
Disk Space Used (kb)
32688
Disk Space Free (kb)
459656
7
dataMine graphing page
dataMine debug page

in the advance settings i have

mountLocation  /dev/sda1
mountType  ext3

does this look right ? I am using a 4Gb usb stick

Offline Otje

  • Full Member
  • ***
  • Posts: 214
  • Karma: +6/-1
Re: New dataMine graphing and logging plugin
« Reply #704 on: October 29, 2013, 07:16:38 pm »
Hi

it looks OK. if you see it in the root it may be the mounted datamine directory which is actually on your usb stick.
just to make sure you could post you Datamine log file so that we can take a look to see?
VERA Plus, VERA 3 Lite , couple of PIR's, several switch units, wall mounted tablet, Sonos integration with TTS, Temperature sensors, Imperihome, Zwave LED bulbs