We have moved at community.getvera.com

Author Topic: Best route for dataMine Weather LUA Programming  (Read 2557 times)

Offline Z-Waver

  • Master Member
  • *******
  • Posts: 4437
  • Karma: +249/-120
Best route for dataMine Weather LUA Programming
« on: January 19, 2014, 12:48:35 pm »
I have a weather station from which I would like to incorporate the outside temperature and possibly a couple of other variables(cloud cover/humidity) into Vera for use by dataMine and perhaps some PLEG conditions.

Current weather station data is stored in the from of a 1.5KB JSON file that is HTTP accessible. This station also uploads to Weather Underground.

I realize that in terms of ease of implementation, the Weather Underground plugin is the most obvious choice for doing this. However, I am mildly concerned about plugin load and the need for Vera to perform WAN calls versus LAN calls.

This question is a request for comment on what would be the least heavy implementation method(CPU/Memory) for this task. Should I just go ahead with the Weather Underground plugin and be done with it, or would my needs be better served by using LUA to pull the JSON file and parse that into something?

Fair Warning: Recommendations for using the LUA route will be followed with how-to questions.

Offline akbooer

  • Beta Testers
  • Master Member
  • *****
  • Posts: 6387
  • Karma: +290/-70
  • "Less is more"
Re: Best route for dataMine Weather LUA Programming
« Reply #1 on: January 19, 2014, 01:06:29 pm »
For such a small file, I should hardly think it matters which way you go?  Both will need a good JSON decoder.  I pull a much larger file down from the Netatmo servers every 10 minutes and that seems to work well, but if it's on the system already, then why not go that route?
3x Vera Lite-UI5/Edge-UI7, 25x Fibaro, 23x TKB, 9x MiniMote, 2x NorthQ Power, 2x Netatmo, 1x Foscam FI9831P, 9x Philips Hue,
Razberry, MySensors Arduino, HomeWave, AltUI, AltHue, DataYours, Grafana, openLuup, ZWay, ZeroBrane Studio.

Offline RichardTSchaefer

  • Community Beta
  • Master Member
  • ******
  • Posts: 10091
  • Karma: +764/-143
Re: Best route for dataMine Weather LUA Programming
« Reply #2 on: January 19, 2014, 01:11:41 pm »
The Weather Underground Plugin will cause memory allocation because it's a plugin ... It has a stack and a LUA context.
It polls at a regular interval ... I forget what the default poll interval is ...
There is no memory savings in doing an http request from LAN vs WAN.
The implementation details of parsing json vs XML is probably not worth comparing for something that happens so infrequently.

So by doing this in LUA ... presumably from a timer in the Startup Window ... the only thing you would save is the STACK space for another plugin and another LUA context.  I am pretty sure the  Startup window uses the same stack space and lua context as the Scene controller.

The benefits of using the Weather Plugin are as follows:
   1) If the PWS goes down ,,, it will return the Weather of the next closest location.
   2) I can use PLEG to sense that the WU isn't getting updated from my PWS ... this is my most typical failure mode. Couple times a year.

Offline JeepGoesBeep

  • Sr. Newbie
  • *
  • Posts: 23
  • Karma: +0/-0
Re: Best route for dataMine Weather LUA Programming
« Reply #3 on: January 19, 2014, 01:42:16 pm »

I use WUHU on a pc to upload my data to Weather Underground and I wrote a script to send the data to Vera so this might not work for you but here is the strategy I chose:

I installed the Weather Underground plugin...since it is so popular third party plugins support it so I didnt see a need to reinvent the wheel.

I made a custom implementation file and uploaded it to Vera and told the parent weather device to use it instead of the one installed by the plugin...I called it I_WUIWeatherLocal.xml as recommended by someone here when I asked about this a while back. The only change the custom one has is that I commented out the updating of the variables I wanted to update myself so it still works just like it did before just with fewer updated values.

I setup the plugin to point to my weather station on Weather Underground by using the pws:STATIONID method in the location but I set the polling to 0 so that Vera would never poll for info.

On the PC running WUHU i made an action list for WUHU to run my vbscript and passed it the variables for temp, wind speed, gusts, etc.

Then the vbscript sends that info to Vera using http gets...I also made sure I changed the last update time of the weather device using a unix time stamp from my script so that anything that looked at that would know the time it was updated as well.

The script then tells the weather device on Vera to poll weather underground for the forecasted highs and lows for the day as well as the current conditions (clear/cloudy/overcast etc) using an http request to Vera telling it to fire the poll function of the plugin. That way it still polls, but only when I update on my side.

The script then sets the polling period on Vera back to 0 just in case it gets set by someone on accident.

This gives me the best of both worlds, I have my temps, wind speeds etc updated locally by my weather software and I also get the forecasted highs and lows from WU so I know whether I need to wear a jacket for the day :)

Offline Z-Waver

  • Master Member
  • *******
  • Posts: 4437
  • Karma: +249/-120
Re: Best route for dataMine Weather LUA Programming
« Reply #4 on: January 19, 2014, 03:45:37 pm »
Thanks to you all. You've all given me great ideas.

@RichardTSchaefer - How do you use PLEG to determine if WU is being updated or not. That outage scenario happens to me several times more frequently than you.

@JeepGoesBeep - This sounds like a really good idea, the best of both worlds as you said. But, I'm unclear on how you send Vera an XML file with HTTP gets. Also, where on Vera do you put the XML file? Would you mind sharing your script, or at least the pseudo code for it, as I would be implementing this on Linux which does not do VBScript.

Offline RichardTSchaefer

  • Community Beta
  • Master Member
  • ******
  • Posts: 10091
  • Karma: +764/-143
Re: Best route for dataMine Weather LUA Programming
« Reply #5 on: January 19, 2014, 04:20:42 pm »
In PLEG I have one device property  input:

WeatherStation    Bound to the LocationUsedText  property of the WU plugin
   
And two conditions:

WSProblems         WeatherStation ne 'Smithwick, Marble Falls, Texas'
WSOK                  Not WSProblems

The actions send alerts via Vera Alerts. I use a tone that repeats until acknowledged for the WSProblems action.

Offline Z-Waver

  • Master Member
  • *******
  • Posts: 4437
  • Karma: +249/-120
Re: Best route for dataMine Weather LUA Programming
« Reply #6 on: January 19, 2014, 04:48:30 pm »
Thanks @RichardTSchaefer! I anticipated a temperature value comparison or similar, rather than simply an absence of the station.

Offline JeepGoesBeep

  • Sr. Newbie
  • *
  • Posts: 23
  • Karma: +0/-0
Re: Best route for dataMine Weather LUA Programming
« Reply #7 on: January 19, 2014, 04:56:28 pm »

Absolutely, keep in mind I havent implemented any sanity checking on the values yet, and I dont check to see if I get a 200 response back from Vera after sending the http...I havent gotten that far into it yet. Doing this in bash should be a simple matter of using wget or curl after you've setup the URL using your own device numbers. Also keep in mind, I'm no programmer so it's kind of dirty, but it works.

The implementation file I am attaching is simply uploaded from Apps/Develop Apps/Luup Files/choose the file and click go. Then go into the weather device (the parent) and edit the value for impl_file to I_WUIWeatherLocal.xml...set your poll value to 0, your script will take care of all the polling and setting values. You can see I commented out a few lines ( the ones that have the --  in front of *changed*) I had to rename the veraweather.vbs to veraweather.txt to upload here. There are no other changes to that file from the original other than the commenting out of those lines. All of the values get changed from the veraweather script

If you have any questions just holler...I've been using this for a day or so and it's working perfect so far...

Offline JeepGoesBeep

  • Sr. Newbie
  • *
  • Posts: 23
  • Karma: +0/-0
Re: Best route for dataMine Weather LUA Programming
« Reply #8 on: January 19, 2014, 04:59:32 pm »

One thing I forgot to add, you might have to do the math for the wind directions (N, NNW, SSW etc)..luckily WUHU had a value for that that I could send on the commandline

Offline RichardTSchaefer

  • Community Beta
  • Master Member
  • ******
  • Posts: 10091
  • Karma: +764/-143
Re: Best route for dataMine Weather LUA Programming
« Reply #9 on: January 19, 2014, 05:47:27 pm »
Quote
I anticipated a temperature value comparison or similar, rather than simply an absence of the station.
I always get good temp data from WU ... It's usually from my station. But when my station is acting up WU returns data from a nearby PWS.

I do not think WU will report data that is older than 1 hour.

Offline Z-Waver

  • Master Member
  • *******
  • Posts: 4437
  • Karma: +249/-120
Re: Best route for dataMine Weather LUA Programming
« Reply #10 on: January 19, 2014, 05:51:10 pm »
Thanks very much. This will get get me right where I want to be.

I'll be able to use local weather data for scene logic even if the internet or Weather Underground  connection is out.

Using Richard's PLEG I'll have Vera alert me when Weather Underground connectivity is interrupted(WU's email alert is too slow and often goes unnoticed for even longer.)


Offline JeepGoesBeep

  • Sr. Newbie
  • *
  • Posts: 23
  • Karma: +0/-0
Re: Best route for dataMine Weather LUA Programming
« Reply #11 on: January 19, 2014, 07:30:20 pm »
Be sure to set your polling to something sensible....WU frowns on pulling too many requests on the same API id in one day. Once every ten minutes should be more than adequate to keep current temps...it's what I have mine set at so there are only 6 requests in an hour so only 144 in a 24 hour period...well under the 500 (i think) that they allow in a day.