We have moved at community.getvera.com

Author Topic: New dataMine graphing and logging plugin  (Read 287574 times)

Offline rarev

  • Sr. Newbie
  • *
  • Posts: 43
  • Karma: +0/-0
Re: New dataMine graphing and logging plugin
« Reply #435 on: September 12, 2012, 04:35:27 am »
Sorry for not being clear. What I meant was that the channels seemed to disappear. So I selected 5 channels and the virtual DataMine device showed "5". When I checked a few hours later the counter was 0. A reinstall fixed somehow, so I'm afraid I can't reproduce it.

I had this problem... I think it was that I had mounted the Vera logs to USB... thus it was 'sharing' a mount point with dataMine... about every week or so my channels would go bye-bye. After removing the Vera logs, this stopped and they have not disappeared since. One solution provided was to partition the USB drive and mount to different partitions... but Chris said he would make it so we don't need to us a mount point... I'm waiting for this :)

The problem with the disappearing channels was definitely a problem with the USB mount. So that was my bad. Setting the mount point to /dev/sda in stead of /dev/sda1 on vera3lite. This one is fixed.
« Last Edit: September 12, 2012, 04:50:02 am by rarev »

Offline rarev

  • Sr. Newbie
  • *
  • Posts: 43
  • Karma: +0/-0
Re: New dataMine graphing and logging plugin
« Reply #436 on: September 12, 2012, 06:06:14 am »
Somehow destroyed it this time. There's no way to select a channel...
I uninstalled DataMine. Formatted the USB-drive (1Gb) to FAT32. Installed DataMine again. Plugged in the USB-drive and entered the DataMine MountPoint to /dev/sda1. MountType is EXT4 and I cannot change this. (changing this to VFAT or EXT3, will make no difference, since hitting SAVE in UI5 will reset it to EXT4)

When I open DataMine to select channels nothing happens. The number of selected/logged channels will remain 0. ARRRG!

Offline rarev

  • Sr. Newbie
  • *
  • Posts: 43
  • Karma: +0/-0
Re: New dataMine graphing and logging plugin
« Reply #437 on: September 12, 2012, 07:38:21 am »
It seems that the browser is causing some trouble.
@Aaron By any chance you are using Windows 8 Internet Explorer?

Offline garrettwp

  • Master Member
  • *******
  • Posts: 6371
  • Karma: +227/-128
  • Vera 3, Lite, ISY994
Re: New dataMine graphing and logging plugin
« Reply #438 on: September 12, 2012, 07:41:19 am »
Internet Explorer has been known to cause all sorts of issues with Vera's Interface. It is recommended to use another browser like firefox, chrome, safari, etc.

- Garrett

Offline rarev

  • Sr. Newbie
  • *
  • Posts: 43
  • Karma: +0/-0
Re: New dataMine graphing and logging plugin
« Reply #439 on: September 12, 2012, 10:07:10 am »
Using Chrome solved most issues. But still only exclamation marks on the graphing page

Offline garrettwp

  • Master Member
  • *******
  • Posts: 6371
  • Karma: +227/-128
  • Vera 3, Lite, ISY994
Re: New dataMine graphing and logging plugin
« Reply #440 on: September 12, 2012, 10:43:02 am »
Check the graph data. I have a few exclamation marks but the graph data is current.

- Garrett

Offline rarev

  • Sr. Newbie
  • *
  • Posts: 43
  • Karma: +0/-0
Re: New dataMine graphing and logging plugin
« Reply #441 on: September 12, 2012, 10:53:55 am »
I tried resetting by removing datamine and formatting the USB drive and now I can't even select channels anymore. After setting SetMountPoint, mountLocation and mountType to "/dev/sda1" (2x) and vfat resp. and hitting save in ui5, mountLocation and mountType are empty again. So at the moment DataMine can't even reach my USB drive.  :-[

Offline ipstas

  • Sr. Newbie
  • *
  • Posts: 49
  • Karma: +2/-0
Re: New dataMine graphing and logging plugin
« Reply #442 on: September 12, 2012, 12:58:29 pm »
rarev,

I would suggest to log in through ssh and check out there.
When you insert the stick and run dmesg, it should show you someting like

root@MiOS_30002917:/tmp# dmesg  | grep sda
sd 0:0:0:0: [sda] 1949696 512-byte logical blocks: (998 MB/952 MiB)
sd 0:0:0:0: [sda] Write Protect is off
sd 0:0:0:0: [sda] Mode Sense: 23 00 00 00
sd 0:0:0:0: [sda] Assuming drive cache: write through
sd 0:0:0:0: [sda] Assuming drive cache: write through
 sda: sda1 sda2
sd 0:0:0:0: [sda] Assuming drive cache: write through
sd 0:0:0:0: [sda] Attached SCSI removable disk
EXT3-fs (sda1): using internal journal
EXT3-fs (sda1): mounted filesystem with ordered data mode

So mine has /dev/sda1 and /dev/sda2
At the moment I am using /dev/sda1 for the usb logging and /dev/sda2 for datamine

to be sure /dev/sda2 is mounted to /dataMine you would need to update
vi /etc/init.d/custom-user-startup
mount /dev/sda2 /dataMine &

make sure, the /dev/sda2 have filesystem. You can do it:
mkfs.vfat /dev/sda2
then mount
mount /dev/sda2 /dataMine

I don't recommend the other method of mounting which was mentioned before, /dev/sda is standard in any unix and just easier to do.

Offline Chris

  • Moderator
  • Hero Member
  • *****
  • Posts: 821
  • Karma: +3/-0
Re: New dataMine graphing and logging plugin
« Reply #443 on: September 12, 2012, 06:00:04 pm »
I don't recommend the other method of mounting which was mentioned before, /dev/sda is standard in any unix and just easier to do.

@ipstas - what do you mean by this? You seem to know a little about linux (more than me anyway!) so if you see anything that I'm doing incorrectly, or that could be done better, please let me know.

At the moment, dataMine automatically runs the mount /dev/sda2 /dataMine command (the device being based on config of course).

Thanks
Chris

Offline ipstas

  • Sr. Newbie
  • *
  • Posts: 49
  • Karma: +2/-0
Re: New dataMine graphing and logging plugin
« Reply #444 on: September 12, 2012, 06:14:40 pm »
Chris,

I am from linux side ;)
In http://code.mios.com/trac/mios_datamine it mentioned to mount like that:
mount /dev/scsi/host0/bus0/target0/lun0/part1 /dataMine, it maybe is better to fix it.

I don't know how datamine does mounting, on my unit /dev/sda2 did not have filesystem, probably because I have usb logging and Vera3 formatted the stick itself. So I have created filesystem first and then mounted manually. And just in case created /etc/init.d/ script to mount it at the restart.


Offline mjcohnmd

  • Newbie
  • *
  • Posts: 4
  • Karma: +0/-0
Re: New dataMine graphing and logging plugin
« Reply #445 on: September 12, 2012, 11:40:57 pm »
The thermostats seem to send erroneous outlying data fairly often (usually over 180 deg F, once or so per day per thermostat).  A setting to ignore outlying data (?>50% from mean) for the graphs would be nice. Or a tool for exporting the data to manually edit (or maybe teach me how to get directly at the raw data)

I'll look at adding two options - one to filter "out of range" data (I'll provide the ability to set these ranges). The other is the ability to export the data as CSV - this is something I've been meaning to add anyway... I'll try and add this in the next few weeks (I'm currently a bit busy with work and getting heating into the new extension before winter sets in).

Cheers
Chris

Just go to Home Depot and get a portable heater... you have work to do ;-)

Thank you...I appreciate your efforts.

--JC


Offline Chris

  • Moderator
  • Hero Member
  • *****
  • Posts: 821
  • Karma: +3/-0
Re: New dataMine graphing and logging plugin
« Reply #446 on: September 13, 2012, 04:33:52 pm »
Chris,

I am from linux side ;)
In http://code.mios.com/trac/mios_datamine it mentioned to mount like that:
mount /dev/scsi/host0/bus0/target0/lun0/part1 /dataMine, it maybe is better to fix it.

I don't know how datamine does mounting, on my unit /dev/sda2 did not have filesystem, probably because I have usb logging and Vera3 formatted the stick itself. So I have created filesystem first and then mounted manually. And just in case created /etc/init.d/ script to mount it at the restart.
Thanks - the "original" mount data I found on an OpenWRT forum for mounting USB sticks - it worked, and I didn't know better. The current method simply uses a "mount <mp> <dir>" where mp and dir are set in the configuration. This eliminated the need to mess around with anything in init.d, or custom files or anything. This seems to work ok (I think). These "original" instructions are for the UI4 version - the UI5 version has the functionality to mount the drive in the Lua file.

Cheers
Chris

Offline rarev

  • Sr. Newbie
  • *
  • Posts: 43
  • Karma: +0/-0
Re: New dataMine graphing and logging plugin
« Reply #447 on: September 14, 2012, 04:08:13 am »
I found the problem was in formatting the usb. Still don't know how exactly it should be formatted, but this is what I did and it worked:
  • Enable USB logging
  • USB drive will be formatted by Vera
  • Disable USB logging
  • When disabling, Vera will show the correct mountPoint
  • Install DataMine and enter the mountPoint in Advanced Settings

It just works
« Last Edit: September 14, 2012, 04:10:58 am by rarev »

Offline Chris

  • Moderator
  • Hero Member
  • *****
  • Posts: 821
  • Karma: +3/-0
Re: New dataMine graphing and logging plugin
« Reply #448 on: September 14, 2012, 01:49:21 pm »
I've had no problems using Windows to format a USB - I've done it a number of times on different Vera boxes and never had an issue. It's interesting though that by doing it this way you get the mount point information "for free".

Cheers
Chris

Online parkerc

  • Beta Testers
  • Sr. Hero Member
  • *****
  • Posts: 2479
  • Karma: +35/-48
  • Life Moves Pretty Fast....
Re: New dataMine graphing and logging plugin
« Reply #449 on: September 18, 2012, 06:32:16 am »
not to take anything away from the documentaion provided which is really quite excellent, there seems to just be a gap for those of us on Veralites and also the use of a hub.

Which leads me nicely onto sharing that I am using a Veralite, but with the added complexity of a hub in the middle.

As directed I formatted a new USB memory key and it looks to have create a few things in my /dev/ directory, I can see entries for a sda, sda1 and 1-1.3. Then after a reboot I also now have ttyUSB, ptmx and tyyS0 - (I'm using winscp BTW to look into the directory)

If I ry the guidance above from ipstas

root@MiOS_3XXXXXXXX:~# dmesg | grep sda
sd 0:0:0:0: [sda] 7819264 512-byte logical blocks: (4.00 GB/3.72 GiB)
sd 0:0:0:0: [sda] Write Protect is off
sd 0:0:0:0: [sda] Mode Sense: 03 41 00 00
sd 0:0:0:0: [sda] Assuming drive cache: write through
sd 0:0:0:0: [sda] Assuming drive cache: write through
 sda: sda1
sd 0:0:0:0: [sda] Assuming drive cache: write through
sd 0:0:0:0: [sda] Attached SCSI removable disk
root@MiOS_3XXXXXXXX:~#

So I have put /dev/sda1 into both SetMountPoint and MountLoation of the device, and logging in as root and checking Mount - i seem to have the following

/dev/sda1 on /dataMine type vfat (rw,relatime,fmask=0022,dmask=0022,codepage=cp4                                                                             37,iocharset=iso8859-1,shortname=mixed,errors=remount-ro)

It looks like I can add channels and it records them - but nothing seems to come up on the graphs (6 selected, 6 recording) no data being seen = every graph shows the time across the bottom and runs a line from a -9999989.01 value ?
« Last Edit: September 18, 2012, 09:37:03 am by parkerc »