We have moved at community.getvera.com

Author Topic: 1.1.1338 beta available  (Read 28555 times)

Offline Henk

  • Hero Member
  • *****
  • Posts: 820
  • Karma: +3/-0
Re: 1.1.1338 problem with NetworkMonitor.log
« Reply #30 on: August 18, 2011, 05:57:14 am »
@ridgecat

No harm intended, i was honestly impressed.
We see all sorts of newbies, most have trouble even setting up Vera out of the box.

Its nice and refreshing to see someone come around and support the forum with some in depth knowledge and usefull background info.

There have been issues with logging for even before i joined the forum (im in here for a few months now (unlike guys like @guessed and @JOD who were here from scratch) and still learning myself. So .. from my side... very usefull contribution.

I hope @MCV picks up on it finally and looks into the issues with logging, log-rotations and Vera rebooting for some users.

- Henk
To Henk: Thanks for the kind words. It has nothing to do with the number of posts; I've just been developing system-level software since 1964 (now retired).
« Last Edit: August 18, 2011, 06:32:37 am by Henk »
| Vera2 @ UI4 1.1.1350 / 3.20 | Vera Lite @ UI5 | Vera 3 @ UI5 | 2x Merten  504519 | 1x Duewi  064374 | 1x Everspring SM103 doorbell mod |1 Y-cam IP cam | various LUUP plugins |

Offline garrettwp

  • Master Member
  • *******
  • Posts: 6371
  • Karma: +227/-128
  • Vera 3, Lite, ISY994
Re: 1.1.1338 beta available
« Reply #31 on: August 18, 2011, 07:05:47 am »
It has now been 5 hours since I cleared the NetworkMonitor.log and it has grown over 10MB in that time frame. It looks like there is some kind of error in the networkmonitor script!

- Garrett

Offline JOD

  • Hero Member
  • *****
  • Posts: 1973
  • Karma: +4/-0
Re: 1.1.1338 beta available
« Reply #32 on: August 18, 2011, 09:08:52 am »
Anyone else have verbose turned on?
Good point, verbose was off for my system.

I disabled logging last night and this morning /etc/cmh/cmh.conf = 83bytes.
[7;36m2011-08-18 07:36:19 - NetworkMonitor Terminated with Exit Code: 0[1;00m

JOD.
I'm sorry, my responses are limited. You must ask the right questions.

Offline jackpot

  • Full Member
  • ***
  • Posts: 166
  • Karma: +1/-1
Re: 1.1.1338 beta available
« Reply #33 on: August 18, 2011, 10:23:36 am »
I can not seem to turn off Verbose Logging on my system. Every time time I un check the box it automaticly
checks it again ! Logging to USB is un checked. What the heck am I doing wrong ?
Vera 2 with .1338

Any ideas ?l
Vera 3....1.5.622

Offline jackpot

  • Full Member
  • ***
  • Posts: 166
  • Karma: +1/-1
Re: 1.1.1338 beta available
« Reply #34 on: August 18, 2011, 10:53:53 am »
I found out what was wrong. I had disabled the logging in /ect/cmh/cmh.conf . Put it back to factory default and now I can disable verbose logging. Maybe this will help someone else.
I guess I over reacted.
Vera 3....1.5.622

Offline Henk

  • Hero Member
  • *****
  • Posts: 820
  • Karma: +3/-0
Re: 1.1.1338 beta available
« Reply #35 on: August 18, 2011, 10:57:05 am »
Maybe, but verbose logging is not "ON"  by default...
I informed @MCV to take a loot at the thread and see if it has leads to go on...

- Henk
I found out what was wrong. I had disabled the logging in /ect/cmh/cmh.conf . Put it back to factory default and now I can disable verbose logging. Maybe this will help someone else.
I guess I over reacted.
| Vera2 @ UI4 1.1.1350 / 3.20 | Vera Lite @ UI5 | Vera 3 @ UI5 | 2x Merten  504519 | 1x Duewi  064374 | 1x Everspring SM103 doorbell mod |1 Y-cam IP cam | various LUUP plugins |

Offline jackpot

  • Full Member
  • ***
  • Posts: 166
  • Karma: +1/-1
Re: 1.1.1338 beta available
« Reply #36 on: August 18, 2011, 11:11:09 am »
Thanks........ FYI - I had Log Levels set at 1 only. Like I said I put them back to original and all is fine now.
Vera 3....1.5.622

Offline micasaverde

  • Hero Member
  • *****
  • Posts: 1666
  • Karma: +15/-1
Re: 1.1.1338 beta available
« Reply #37 on: August 18, 2011, 02:59:29 pm »
NetworkMonitor should log the contents of /etc/cmh only when it fails to establish an internet connection.  This is because there was a report of problems occurring when Vera doesn't have internet and upon inspection some files in /etc/cmh/ were missing, which the user insists he didn't delete.  So, some time ago, we added this extra logging so that when it happened again we'd be able to catch it.

This shouldn't be a problem, though.  The only effect should be that logs are rotated more often, however, it will rotate logs up to once a minute if necessary, so even if the logs are growing as described, they should never grow so fast that it brings down the system.

Offline micasaverde

  • Hero Member
  • *****
  • Posts: 1666
  • Karma: +15/-1
Re: 1.1.1338 beta available
« Reply #38 on: August 18, 2011, 03:27:04 pm »
BTW, yes, NetworkMonitor's log is a total mish-mash of stuff, because that's what NetworkMonitor is too.  It originally started as a daemon that would restart firewall rules when the network IP changed and handle light blinking.  But it's grown to be something that has all sorts of functions checking all sorts of things, like CPU load, loopback tests on the port forward (that's the lu_alive you see) and restarting the tunnels, checking that the wi-fi is up, checking for lockups or crashes in certain kernel modules like the ntp clock, resetting the date if it skewed, rebooting if resources are too low, etc.

Every function logs stuff so that if a customer says: "At 8:32 the system stopped responding" we can go into NetwokMonitor's logs and see everything going on at 8:32.  USB logging helps becuase that way the logs persist across reboots, so if, at 8:31 CPU load was high, and the unit crashed at 8:32 and at 8:35 the user cycled power, the logs from 8:31 will still be there and we can see what was hogging the CPU.  Without USB logging the logs are stored in a ram drive that's lost at each boot so unless the logs had rotated after 8:31 and before 8:32, when the user rebooted the logs would have been lost and we wouldn't be able to see what was going on at 8:31.

Hope this helps...

Offline garrettwp

  • Master Member
  • *******
  • Posts: 6371
  • Karma: +227/-128
  • Vera 3, Lite, ISY994
Re: 1.1.1338 beta available
« Reply #39 on: August 18, 2011, 06:11:50 pm »
I do not know why my log is so large. I have not deleted any files on my vera. Is there a way I can check to see what is going on? I looked at the log and it says:

02      08/18/11 18:10:37.247   CheckFreeTunnels file gone response 0

I do not know why it says that as again, I have not deleted any files on the unit.

- Garrett


Offline ridgecat

  • Newbie
  • *
  • Posts: 12
  • Karma: +0/-0
Re: 1.1.1338 beta available
« Reply #40 on: August 18, 2011, 11:27:06 pm »
NetworkMonitor should log the contents of /etc/cmh only when it fails to establish an internet connection.  This is because there was a report of problems occurring when Vera doesn't have internet and upon inspection some files in /etc/cmh/ were missing, which the user insists he didn't delete.  So, some time ago, we added this extra logging so that when it happened again we'd be able to catch it.

This shouldn't be a problem, though.  The only effect should be that logs are rotated more often, however, it will rotate logs up to once a minute if necessary, so even if the logs are growing as described, they should never grow so fast that it brings down the system.

This may be true, but as I mentioned in an earlier post in this thread, I am getting frequent (every 5 seconds for bursts that may last for several minutes) errors saying Check Free Tunnels file gone response 0. Each of these single line errors is followed by a dump of /etc/cmh. I am running through a wireless router with Vera as an access point (MIOS is a bridge) and I have a very reliable and fast internet connection. Vera has been configured like this since I first got it and that has been a long time.

And maybe errors and system failures "shouldn't" happen, but they do. The failure to investigate what is at best strange software behavior because failures "shouldn't" happen isn't reassuring.
BTW, my logging is not verbose.

Offline oTi@

  • Community Beta
  • Master Member
  • ******
  • Posts: 4041
  • Karma: +32/-6
  • UI what ?!
Re: 1.1.1338 beta available
« Reply #41 on: August 19, 2011, 08:21:30 am »
@ridgecat,
I looked into it a bit and have sent a PM to @micasaverde; maybe we get some more details. However, I think it's just redundant logging, and does not necessarily explain the lockup you describe. The logs would fill up quicker than seems necessary, causing log rotations to happen sooner, but it's all bounded in size (I think 10x 20MB before compression when logging to USB).
Dezwaved at the moment...

Offline garrettwp

  • Master Member
  • *******
  • Posts: 6371
  • Karma: +227/-128
  • Vera 3, Lite, ISY994
Re: 1.1.1338 beta available
« Reply #42 on: August 19, 2011, 09:06:03 am »
OTI@ contacted my and asked to see if my "techsupport" was disabled. He asked that I try and turn it on and see if the logging stops. I did what he requested and the logging stopped dead in it's tracks. I feel this should be listed as a bug as there is no need to have tech support enabled if none is being requested.

- Garrett

Offline micasaverde

  • Hero Member
  • *****
  • Posts: 1666
  • Karma: +15/-1
Re: 1.1.1338 beta available
« Reply #43 on: August 19, 2011, 12:13:40 pm »
The logging of the /etc/cmh/ was added on February 21.  1.1.1245 was released on April 20.  This logging was in 1245 too.  NetworkMonitor didn't change from 1245 to 1338.

I checked the commit logs and the reason it was added was that a user kept reporting that he would enable the tech support tunnels (which writes the ra_* file in /etc/cmh) but that the files would just disappear all on their own.  So the point of the logging was to have a continual snapshot of /etc/cmh/ to see when files were getting removed, and which files were removed, and what was going on at that time.

However, it's been 6 months and we haven't gotten any more reports of files in /etc/cmh/ disappearing all on their own.  So it's probably overkill to keep this logging in place and I'll remove it.

The only tough decision is if this means we stop the 1338 release, do a new build, and start the whole in-house testing, private beta test, public beta test, release cycle all over again, which will delay getting the fix to 1245 out for another week or two and will delay the UI5 beta since we'll to pull the testers off UI5.  Or we continue with the plan to release 1338 as-is since this logging issue is not new and was there in the last release anyway.

Offline Henk

  • Hero Member
  • *****
  • Posts: 820
  • Karma: +3/-0
Re: 1.1.1338 beta available
« Reply #44 on: August 19, 2011, 12:23:14 pm »
@micasaverde,

my suggestionwould be:

1. Keep 1338 intact and on schedule.
2. Document (using this thread and or release notes) the issue, including a workaround if there is one.
3. Try to fix it in the NEXT UI5 build as im sure the initial build will probably not make it to final.

That way the issue is captured, possible problems stay contained within a percentage of users and most importantly, no release gets delayed!

Hope others can chip in and add or comment to this proposal.

Best,

- Henk

The logging of the /etc/cmh/ was added on February 21.  1.1.1245 was released on April 20.  This logging was in 1245 too.  NetworkMonitor didn't change from 1245 to 1338.

I checked the commit logs and the reason it was added was that a user kept reporting that he would enable the tech support tunnels (which writes the ra_* file in /etc/cmh) but that the files would just disappear all on their own.  So the point of the logging was to have a continual snapshot of /etc/cmh/ to see when files were getting removed, and which files were removed, and what was going on at that time.

However, it's been 6 months and we haven't gotten any more reports of files in /etc/cmh/ disappearing all on their own.  So it's probably overkill to keep this logging in place and I'll remove it.

The only tough decision is if this means we stop the 1338 release, do a new build, and start the whole in-house testing, private beta test, public beta test, release cycle all over again, which will delay getting the fix to 1245 out for another week or two and will delay the UI5 beta since we'll to pull the testers off UI5.  Or we continue with the plan to release 1338 as-is since this logging issue is not new and was there in the last release anyway.
| Vera2 @ UI4 1.1.1350 / 3.20 | Vera Lite @ UI5 | Vera 3 @ UI5 | 2x Merten  504519 | 1x Duewi  064374 | 1x Everspring SM103 doorbell mod |1 Y-cam IP cam | various LUUP plugins |