We have moved at community.getvera.com

Author Topic: How To - Log data to an external CIFS share (e.g. NAS) - No more USB  (Read 51353 times)

Offline parkerc

  • Beta Testers
  • Sr. Hero Member
  • *****
  • Posts: 2479
  • Karma: +35/-48
  • Life Moves Pretty Fast....
I'm extracting the following from another thread, where @Hek & @Chris talked about the ability to logg your data to a remote share rather than to a USB drive.

Please Note : the following requires SSH access and if followed, it will add a non standard module  to your Vera. Therefore only do this with that clear in your mind.

The NAS mount/logging has been working flawlessly for a month now! Seems to be a really good solution to all this data problems we've seen using USB.
Chris has been pretty busy@work lately and it would be good if he would add re-mount functionality before release (in case NAS goes offline for a long time and Vera unmounts the NAS).

Meanwhile you can install the CIFS packages:

Code: [Select]
cd /tmp/
opkg update
wget  http://download.mios.com/firmware/openwrt/ramips/packages/kmod-crypto-md5_2.6.37.1-1_ramips.ipk
opkg install kmod-crypto-md5_2.6.37.1-1_ramips.ipk
wget  http://download.mios.com/firmware/openwrt/ramips/packages/kmod-crypto-hmac_2.6.37.1-1_ramips.ipk
opkg install kmod-crypto-hmac_2.6.37.1-1_ramips.ipk
wget  http://download.mios.com/firmware/openwrt/ramips/packages/kmod-fs-cifs_2.6.37.1-1_ramips.ipk
opkg install kmod-fs-cifs_2.6.37.1-1_ramips.ipk
opkg install cifsmount
rm kmod-crypto-md5_2.6.37.1-1_ramips.ipk kmod-fs-cifs_2.6.37.1-1_ramips.ipk kmod-crypto-hmac_2.6.37.1-1_ramips.ipk

To test mounting manually. Try this:
Code: [Select]
mkdir /nas
mount -t cifs -o user=<username>,pass=<password> //<nas ip>/<share> /nas

Looks like fhis for me:

root@MiOS_3013232:~# df
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/root                 4608      4608         0 100% /rom
tmpfs                    63728       740     62988   1% /tmp
tmpfs                      512         0       512   0% /dev
/dev/mtdblock7           11264      3452      7812  31% /overlay
overlayfs:/overlay       11264      3452      7812  31% /
/dev/sda1               518320     22448    469544   5% /tmp/log/cmh
/dev/mtdblock8            4480      4480         0 100% /mios
//192.168.178.36/nas/HA/dataMineMounted
                     2879621632 1069356416 1810162816  37% /nas
root@MiOS_3013232:~#


And this..

I've been running this for a month or two now, and it's worked fine... The latest version of dataMine has support for this.

Install CIFS as per the info earlier in this thread, and copy all the files from the /dataMine directory on your USB to your NAS. Set the variable SetMountOption to "-t cifs -o user=XXX,pass=YYY", and SetMountPoint to the NAS share (eg "//192.168.2.10/share").
 
dataMine should then mount the NAS on your next startup - in theory it should also unmount your USB - it worked for me, but you should double check this.

The current version won't attempt to remount, so, if you have a power cut, then if Vera starts before your NAS, you may end up with the mounting failing. To resolve this you'd need to press the restart button in UI5. This will be addressed in the next version.

Chris

Once again keep in mind this is early adoption stuff, and I'm thinking @Chris is very busy with other things at the moment, so all we can do is support each other.

I have done the above myself and sadly I'm unable to get it to work successfully, but if you read the full thread where this all started you can see @Hek and others have got it working well. http://forum.micasaverde.com/index.php/topic,14910.0.html

Offline parkerc

  • Beta Testers
  • Sr. Hero Member
  • *****
  • Posts: 2479
  • Karma: +35/-48
  • Life Moves Pretty Fast....
Re: How To - Log data to an external CIFS share (e.g. NAS) - No more USB
« Reply #1 on: January 01, 2014, 01:35:51 pm »
While there are a few updates on the Synology NAS thread. ->
 http://forum.micasaverde.com/index.php/topic,14910.0.html

As there are other NAS or OS that people can use..

If anyone has managed to get it working with an external share / please could someone share (ideally a screenshot ) of the values you have for the mount under Variables on the Advanced tab.

Some help with mounting can be found here -> http://wiki.openwrt.org/doc/howto/cifs.client
« Last Edit: January 01, 2014, 03:06:50 pm by parkerc »

Offline akbooer

  • Moderator
  • Master Member
  • *****
  • Posts: 6387
  • Karma: +292/-70
  • "Less is more"
Re: How To - Log data to an external CIFS share (e.g. NAS) - No more USB
« Reply #2 on: January 01, 2014, 01:54:18 pm »
If anyone has managed to get it working with an external share / please could someone share (ideally a screenshot ) of the values you have for the mount under Variables on the Advanced tab.

Snapshot attached with main items.  The only other thing missing is the SetMountOptions, which are:
Code: [Select]
-t cifs -o user=XXX,pass=YYY,nounix,noserverino

What I've done in this case is simply to transfer my USB which I was using to a slot on the NAS (an Apple Time Capsule).  I had earlier backed up the database onto the NAS itself and tried that but it didn't work... in hindsight, I believe that this was because I had nested the dataMine/ structure one folder too deep.  If you have a problem getting it working for you, do check the the mount point takes you to the right place from Vera (you can manually mount to check this).

3x Vera Lite-UI5/Edge-UI7, 25x Fibaro, 23x TKB, 9x MiniMote, 2x NorthQ Power, 2x Netatmo, 1x Foscam FI9831P, 9x Philips Hue,
Razberry, MySensors Arduino, HomeWave, AltUI, AltHue, DataYours, Grafana, openLuup, ZWay, ZeroBrane Studio.

Offline parkerc

  • Beta Testers
  • Sr. Hero Member
  • *****
  • Posts: 2479
  • Karma: +35/-48
  • Life Moves Pretty Fast....
Re: How To - Log data to an external CIFS share (e.g. NAS) - No more USB
« Reply #3 on: January 01, 2014, 03:05:59 pm »
Hi @akbooer

Thanks for sharing. I can't even mount a share now so I'm struggling to do anything at the mement.  However what I did find interesting was the screenshot you shared as it is different to mine?

Offline akbooer

  • Moderator
  • Master Member
  • *****
  • Posts: 6387
  • Karma: +292/-70
  • "Less is more"
Re: How To - Log data to an external CIFS share (e.g. NAS) - No more USB
« Reply #4 on: January 01, 2014, 03:23:34 pm »
So what's in the logs, and on the debug page?
3x Vera Lite-UI5/Edge-UI7, 25x Fibaro, 23x TKB, 9x MiniMote, 2x NorthQ Power, 2x Netatmo, 1x Foscam FI9831P, 9x Philips Hue,
Razberry, MySensors Arduino, HomeWave, AltUI, AltHue, DataYours, Grafana, openLuup, ZWay, ZeroBrane Studio.

Offline parkerc

  • Beta Testers
  • Sr. Hero Member
  • *****
  • Posts: 2479
  • Karma: +35/-48
  • Life Moves Pretty Fast....
Re: How To - Log data to an external CIFS share (e.g. NAS) - No more USB
« Reply #5 on: January 01, 2014, 03:44:02 pm »
I played with it today and this is the result.

Quote
Filesystem                Size      Used Available Use% Mounted on
/dev/root                      4.5M      4.5M         0          100% /rom
tmpfs                          30.5M      1.0M     29.5M          3% /tmp
tmpfs                        512.0K          0      512.0K          0% /dev
/dev/mtdblock7           11.0M      4.0M       7.0M         36% /overlay
overlayfs:/overlay        11.0M      4.0M       7.0M         36% /
/dev/sda1                     1.8G     35.7M       1.7G          2% /tmp/log/cmh
/dev/mtdblock8             4.4M      4.4M         0          100% /mios
//192.168.1.101/backup
                                  11.0M      4.0M      7.0M          36% /
//192.168.1.101/backup
                                  11.0M      4.0M      7.0M          36% /
//192.168.1.101/backup
                                  11.0M      4.0M      7.0M           36% /

Whatever I did earlier has created 3 mounts, which are wrong as the NAS has gigabytes free

When I try a direct mount from the command line I now get the following..

root@MiOS_123456789 :~# mount -t cifs -o user=username,pass=password,nounix,noserverino //192.168.1.101/backup /nas
mount error: can not change directory into mount target /nas
mount: mounting //192.168.1.101/backup on /nas failed: No such file or directory

But I can see that share is there.

UPDATE - I may have cracked it !! Not exactly sure what I've done but let's see if it holds..
« Last Edit: January 01, 2014, 05:56:27 pm by parkerc »

Offline parkerc

  • Beta Testers
  • Sr. Hero Member
  • *****
  • Posts: 2479
  • Karma: +35/-48
  • Life Moves Pretty Fast....
Re: How To - Log data to an external CIFS share (e.g. NAS) - No more USB
« Reply #6 on: January 01, 2014, 06:10:15 pm »
OK, i think i've got it to work.

My goal was to have the data stored in the a folder called dataMine on a share called backup on a NAS located at 192.168.1.101 but for some reason I could not get it to work, however if I can think back i have managed to get the following to work..

First i copied all of the files over that were on the original USB key to the desitination share on the NAS

Then I logged into you Vera via SSH and then at the command line, type ..

1) mkdir /nas
2) mount -t cifs -o user=username,pass=password,nounix,noserverino //192.168.1.101/backup /nas

My understanding of the above is that i've created a directory called 'nas' and mapped/mounted it so it now is pointed to the share on my NAS

-----

Then on the Advanced tab of dataMine plugin i completed the following fields

SetDataDirectory  = /nas/
SetMountPoint = //192.168.1.101/backup
SetMountOptions = -t cifs -o user=username,pass=password,nounix,noserverino 

-----

Now to be honest, things like putting a / after the share when mounting it would cause it to fail, so I'm not sure exactly why it works and also why I can only get it to work/mount at the base share /backup/ and not at the folder i want /backup/dataMine

I'll keep an eye on it and just hope it holds - because if I can avoid USB keys - I'll be happy :-)

Offline akbooer

  • Moderator
  • Master Member
  • *****
  • Posts: 6387
  • Karma: +292/-70
  • "Less is more"
Re: How To - Log data to an external CIFS share (e.g. NAS) - No more USB
« Reply #7 on: January 01, 2014, 06:26:55 pm »
The mount you did when logged in over ssh will not survive a reboot.  However, it should not be necessary.  The root directory you have created as the mount point will always be there.  The dataMine mount information is what will be in action every Luup restart. 
3x Vera Lite-UI5/Edge-UI7, 25x Fibaro, 23x TKB, 9x MiniMote, 2x NorthQ Power, 2x Netatmo, 1x Foscam FI9831P, 9x Philips Hue,
Razberry, MySensors Arduino, HomeWave, AltUI, AltHue, DataYours, Grafana, openLuup, ZWay, ZeroBrane Studio.

Offline parkerc

  • Beta Testers
  • Sr. Hero Member
  • *****
  • Posts: 2479
  • Karma: +35/-48
  • Life Moves Pretty Fast....
Re: How To - Log data to an external CIFS share (e.g. NAS) - No more USB
« Reply #8 on: January 02, 2014, 03:03:44 am »
The mount you did when logged in over ssh will not survive a reboot.  However, it should not be necessary.  The root directory you have created as the mount point will always be there.  The dataMine mount information is what will be in action every Luup restart. 

While the mount is holding now, considering what is tried last night, it seems like (for me) that was the only way I could get it to work. Just having the setMountPoint and SetMountOptions alone did not seem to work. I'll keep an eye on it.

I want it to point to a specific directory on the share and not the base of the share itself (it's now full of text fils), are you aware if any reasons why it might not like the next layer/folder down?

Can I ask what you see via SSH when you run df -h  ?

Offline akbooer

  • Moderator
  • Master Member
  • *****
  • Posts: 6387
  • Karma: +292/-70
  • "Less is more"
Re: How To - Log data to an external CIFS share (e.g. NAS) - No more USB
« Reply #9 on: January 02, 2014, 05:00:33 am »
Code: [Select]
# df -h
Filesystem                Size      Used Available Use% Mounted on
/dev/root                 4.5M      4.5M         0 100% /rom
tmpfs                    30.5M      6.8M     23.7M  22% /tmp
tmpfs                   512.0K         0    512.0K   0% /dev
/dev/mtdblock7           11.0M      2.9M      8.1M  27% /overlay
overlayfs:/overlay       11.0M      2.9M      8.1M  27% /
/dev/mtdblock8            4.4M      4.4M         0 100% /mios
//172.16.42.100/DATAMINE
                        977.3M     24.4M    953.0M   2% /dataMine
3x Vera Lite-UI5/Edge-UI7, 25x Fibaro, 23x TKB, 9x MiniMote, 2x NorthQ Power, 2x Netatmo, 1x Foscam FI9831P, 9x Philips Hue,
Razberry, MySensors Arduino, HomeWave, AltUI, AltHue, DataYours, Grafana, openLuup, ZWay, ZeroBrane Studio.

Offline konradwalsh

  • Hero Member
  • *****
  • Posts: 566
  • Karma: +19/-6
Re: How To - Log data to an external CIFS share (e.g. NAS) - No more USB
« Reply #10 on: January 02, 2014, 10:43:41 am »
The mount you did when logged in over ssh will not survive a reboot.  However, it should not be necessary.  The root directory you have created as the mount point will always be there.  The dataMine mount information is what will be in action every Luup restart. 

While the mount is holding now, considering what is tried last night, it seems like (for me) that was the only way I could get it to work. Just having the setMountPoint and SetMountOptions alone did not seem to work. I'll keep an eye on it.

I want it to point to a specific directory on the share and not the base of the share itself (it's now full of text fils), are you aware if any reasons why it might not like the next layer/folder down?

Can I ask what you see via SSH when you run df -h  ?

Could you post a screenshot of your Advanced tab so I can compare where I might have misstepped

Offline konradwalsh

  • Hero Member
  • *****
  • Posts: 566
  • Karma: +19/-6
Re: How To - Log data to an external CIFS share (e.g. NAS) - No more USB
« Reply #11 on: January 02, 2014, 11:13:10 am »
I cannot seem to get this working either..

the share is on a windows server

Code: [Select]
-1-===========================================================================
09 01/02/14 16:11:17.575 JobHandler_LuaUPnP::Run device 171 dataMine graphing and logging room 0 type urn:schemas-cd-jackson-com:device:DataMine:1 id  parent 0/0xe16ae0 upnp: 0 <0x2b683000>
50 01/02/14 16:11:44.459 luup_log:171: dataMine: Initialising dataMine System (0.980) __LEAK__ this:65536 start:892928 to 0x17f0000 <0x2bfb7680>
06 01/02/14 16:11:44.460 Device_Variable::m_szValue_set device: 171 service: urn:cd-jackson-com:serviceId:DataMine1 variable: errorStatus was: true now: false #hooks: 0 upnp: 0 v:(nil)/NONE duplicate:0 <0x2bfb7680>
06 01/02/14 16:11:44.461 Device_Variable::m_szValue_set device: 171 service: urn:cd-jackson-com:serviceId:DataMine1 variable: ChannelCnt was: 0 now: 0 #hooks: 0 upnp: 0 v:0xfd77a8/NONE duplicate:1 <0x2bfb7680>
06 01/02/14 16:11:44.461 Device_Variable::m_szValue_set device: 171 service: urn:cd-jackson-com:serviceId:DataMine1 variable: ChannelRec was: 0 now: 0 #hooks: 0 upnp: 0 v:(nil)/NONE duplicate:1 <0x2bfb7680>
50 01/02/14 16:11:44.499 luup_log:171: dataMine: Error reading tmpfile during mountcheck <0x2bfb7680>
06 01/02/14 16:11:44.500 Device_Variable::m_szValue_set device: 171 service: urn:cd-jackson-com:serviceId:DataMine1 variable: mountLocation was:  now:  #hooks: 0 upnp: 0 v:(nil)/NONE duplicate:1 <0x2bfb7680>
06 01/02/14 16:11:44.500 Device_Variable::m_szValue_set device: 171 service: urn:cd-jackson-com:serviceId:DataMine1 variable: mountType was:  now:  #hooks: 0 upnp: 0 v:(nil)/NONE duplicate:1 <0x2bfb7680>
50 01/02/14 16:11:44.523 luup_log:171: dataMine: Mounting dataMine storage (//192.168.2.1/vera_data) to (/dataMine/) <0x2bfb7680>
50 01/02/14 16:11:44.586 luup_log:171: dataMine: Error reading tmpfile during mountcheck <0x2bfb7680>
06 01/02/14 16:11:44.586 Device_Variable::m_szValue_set device: 171 service: urn:cd-jackson-com:serviceId:DataMine1 variable: mountLocation was:  now:  #hooks: 0 upnp: 0 v:(nil)/NONE duplicate:1 <0x2bfb7680>
06 01/02/14 16:11:44.587 Device_Variable::m_szValue_set device: 171 service: urn:cd-jackson-com:serviceId:DataMine1 variable: mountType was:  now:  #hooks: 0 upnp: 0 v:(nil)/NONE duplicate:1 <0x2bfb7680>
50 01/02/14 16:11:44.609 luup_log:171: dataMine: Mount point error: :://192.168.2.1/vera_data <0x2bfb7680>
06 01/02/14 16:11:44.609 Device_Variable::m_szValue_set device: 171 service: urn:cd-jackson-com:serviceId:DataMine1 variable: errorStatus was: false now: true #hooks: 0 upnp: 0 v:(nil)/NONE duplicate:0 <0x2bfb7680>
50 01/02/14 16:11:44.625 luup_log:171: dataMine: ERROR: Startup state is 'uninitialised'! <0x2bfb7680>
06 01/02/14 16:11:44.625 Device_Variable::m_szValue_set device: 171 service: urn:cd-jackson-com:serviceId:DataMine1 variable: ChannelCnt was: 0 now: 0 #hooks: 0 upnp: 0 v:0xfd77a8/NONE duplicate:1 <0x2bfb7680>
06 01/02/14 16:11:44.625 Device_Variable::m_szValue_set device: 171 service: urn:cd-jackson-com:serviceId:DataMine1 variable: ChannelRec was: 0 now: 0 #hooks: 0 upnp: 0 v:(nil)/NONE duplicate:1 <0x2bfb7680>
50 01/02/14 16:11:44.626 luup_log:171: dataMine: Reinitialising configuration structure <0x2bfb7680>
50 01/02/14 16:11:44.627 luup_log:171: dataMine: Startup complete <0x2bfb7680>
50 01/02/14 16:11:59.093 luup_log:171: dataMine: Delete backups complete - No files found <0x319b7680>
50 01/02/14 16:11:59.116 luup_log:171: dataMine: Error reading tmpfile during spacecheck <0x319b7680>
-2-===========================================================================
-3-===========================================================================
rootfs on / type rootfs (rw)
/dev/root on /rom type squashfs (ro,relatime)
proc on /proc type proc (rw,noatime)
sysfs on /sys type sysfs (rw,noatime)
tmpfs on /tmp type tmpfs (rw,nosuid,nodev,noatime)
tmpfs on /dev type tmpfs (rw,noatime,size=512k,mode=755)
devpts on /dev/pts type devpts (rw,noatime,mode=600)
/dev/mtdblock7 on /overlay type jffs2 (rw,noatime)
overlayfs:/overlay on / type overlayfs (rw,noatime,lowerdir=/,upperdir=/overlay)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
/dev/mtdblock8 on /mios type squashfs (ro,relatime)
none on /proc/bus/usb type usbfs (rw,relatime)
-4-===========================================================================

Disk /dev/mtdblock0: 0 MB, 524288 bytes
255 heads, 63 sectors/track, 0 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/mtdblock0 doesn't contain a valid partition table
-5-===========================================================================
SetDataDirectory-/dataMine/
SetMountUUID    -
SetMountPoint   -//192.168.2.1/vera_data
SetManualMount  -0
-6-===========================================================================
{"dbVersion":2,"Events":{"count":0,"last":0},"Version":"0.980","Variables":[],"Graphs":[],"guiConfig":[],"nextId":1}
-7-===========================================================================

Offline Sender

  • Hero Member
  • *****
  • Posts: 973
  • Karma: +552/-467
  • Sr. manager of my Vera
Re: How To - Log data to an external CIFS share (e.g. NAS) - No more USB
« Reply #12 on: January 02, 2014, 01:14:46 pm »
Is there any chance that this can be written out for dummies?
The whole package? What to do from start till end with a system with default datamine running?
In clear steps? Off course I can try to find out myself but chance of failure if fairly big then. Thanks in advance!
Good firmware is more important than good karma. (and this costed me lots of Karma)

Offline parkerc

  • Beta Testers
  • Sr. Hero Member
  • *****
  • Posts: 2479
  • Karma: +35/-48
  • Life Moves Pretty Fast....
Re: How To - Log data to an external CIFS share (e.g. NAS) - No more USB
« Reply #13 on: January 02, 2014, 03:00:31 pm »
For me, I think the first thing to do so you have confidence it works - is to manually mount the target share to check that ...

a) it mounts correctly (you'll see error if not and these will help you troubleshoot
b) you can access it and see the files and the permissions of those files

What I did was to log into my Vera via SSH and then at the command line, typed ..

1) mkdir /nas
2) mount -t cifs -o user=username,pass=password,nounix,noserverino //192.168.1.101/backup /nas

This created a directory on Vera called /nas/ which maps to my NAS and I could access/see all the files.

Note : copy the contents of your current DataMine USB key over to the share, if you have watched variables already in your datamineconfig then they show in the UI once the mount is successful.


Offline konradwalsh

  • Hero Member
  • *****
  • Posts: 566
  • Karma: +19/-6
Re: How To - Log data to an external CIFS share (e.g. NAS) - No more USB
« Reply #14 on: January 03, 2014, 04:33:53 am »
I can manually mount with that command to the folder.

before I run your command, its like this:
Code: [Select]
root@MiOS_3*****5:~# df
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/root                 4608      4608         0 100% /rom
tmpfs                    31240      5464     25776  17% /tmp
tmpfs                      512         0       512   0% /dev
/dev/mtdblock7           11264      5712      5552  51% /overlay
overlayfs:/overlay       11264      5712      5552  51% /
/dev/mtdblock8            4480      4480         0 100% /mios

after, its like this:

Code: [Select]
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/root                 4608      4608         0 100% /rom
tmpfs                    31240      5484     25756  18% /tmp
tmpfs                      512         0       512   0% /dev
/dev/mtdblock7           11264      5712      5552  51% /overlay
overlayfs:/overlay       11264      5712      5552  51% /
/dev/mtdblock8            4480      4480         0 100% /mios
//192.168.2.1/vera_data
                     3906885628 3193756608 713129020  82% /nas

But on reboot it doesnt get created.