Author Topic: Virtual Rain Sensor  (Read 30228 times)

Offline BulldogLowell

  • Hero Member
  • *****
  • Posts: 1571
  • Karma: +190/-85
Re: Virtual Rain Sensor
« Reply #30 on: May 12, 2014, 12:39:35 pm »
Would it make sense to add the past/current/forecast tempterature to the formula to indicate if watering is needed?

it took me a while,but finally got what you meant,  In SoFla where (HOT therefore RAIN) is the norm, I was in the wrong paradigm.   Extended hot weather without rain... may need to add a watering cycle that otherwise you may not do.

you could use PLEG to do that for you; eg. if it records a sensor's temperature over 90F for the past three days, and today is Wednesday, water today instead of thursday.


Offline Freddan101

  • Sr. Member
  • ****
  • Posts: 347
  • Karma: +12/-2
Re: Virtual Rain Sensor
« Reply #31 on: May 12, 2014, 02:21:38 pm »
Yes, PLEG would work for that. I see a scenario where it hasn't rained for 3 days but max temp has only been 10C. Probably watering is not needed.

Offline blakem

  • Sr. Newbie
  • *
  • Posts: 47
  • Karma: +7/-0
Re: Virtual Rain Sensor
« Reply #32 on: May 12, 2014, 08:29:05 pm »
Would it make sense to add the past/current/forecast tempterature to the formula to indicate if watering is needed?

Yes it would, but it is unfortunately not that simple.  Equations for calculating evaporation are functions of not only temperature, but wind, humidity and solar radiation.  The agriculture sciences have come up with some nice guides on all of these calculations and I started off studying them before making this plugin.  In the end I figured I would just keep it simple by only limiting the number of days that I would look at and the moving window would sort of solve the evaporation problem.  While it is not perfect I feel it does do an "ok" job at creating a method in the majority of cases to limit irrigation during periods of rain.

In the end you need to know not only how much precipitation falls, but also how much runs off before it is absorbed in the ground.  Also some water pools up on the ground and evaporates at a higher rate than what gets absorbed below the surface.  Another loss is transpiration of the grass itself, but most group evaporation and transpiration together and call it evapotranspiration.

In the end I might look at adding an option to calculate evapotranspiration so it will essentially reduce the total precipitation over time, which might possibly drop below the threshold.  That way you can set a longer period of time and it will better model natural evaporation like a real physical rain sensor.  It also could work the opposite way and evaporate more quickly in hot weather which would also be desirable.

Offline AgileHumor

  • Hero Member
  • *****
  • Posts: 984
  • Karma: +51/-27
  • KISS
Re: Virtual Rain Sensor
« Reply #33 on: May 12, 2014, 08:34:18 pm »
This plugin is amazing.  Thanks BlakeM!
WMC Leviton:18xVPE06,8xVRS15,3xVRP03-W,2xVRR15,4xVRCS4,2xVRCS2,VP00R,8xVRS15 Aeon:5xDSC06106,4xDSC24,4xDSC25,12xDSB29,2xDSC11,4xDSB54,DSB05,3xDSA22,DSA38,2xDSA03202B,DSB09104,HEM Other:3xYale,12xHSM100v3,7xSP103,45604,WDHA-12,SSA2USR,EVLCD1T,6xWWA02A,7xIPC-HFW2100,URTSI,Hue,Russound,OpenSprinker

Offline blakem

  • Sr. Newbie
  • *
  • Posts: 47
  • Karma: +7/-0
Re: Virtual Rain Sensor
« Reply #34 on: May 12, 2014, 08:49:22 pm »
Awesome plugin and thanks for forecast fast fix with v.2.5!

Few suggestions on future development:
- display values in Dashboard plugin interface: total, past & forecast, last pull time
I had thought about that before, but has not been a priority yet.  I went ahead and added the total precipitation to the dashboard and will put it in the next release, but I am not sure there is room to put really any more info without crowding it up.

- pull and display all available days, not just threshold days
Not sure exactly what you mean, but I assume you mean display the data for 7 days past and future but only compare to the number set on the settings tab.  It could be done, but it would be a big change.  Also there is a separate request for each day in the past.  With the option to select the number it does give the option of how many requests to generate.  Not sure everyone will want it to create that many.

- separate parameter for each day (.., d-2,d-1,d0,d1,..)
I tried to keep the number of device state variables used to a minimum, but if there is a need to export the data in that form then it could be possible.  Right now with a little lua/luup code you can get each individual day from the CSV string.

- probability for each forecast day
I don't think this would be very helpful.  I used the QPF since it gives the magnitude of the precipitation, but the PoP has no magnitude.  It just states the probability that greater than 0.01" of rain will fall within the predicted period.  So it basically tells the certainty, but lacks the amount which is what in this application I think you would want to know.  I had thought about multiplying the QPF and the PoP together to give the forecast a weighted value of certainty but I don't know if that would even make any sense to do that.  It was just a thought on how to have a more optimistic approach.

- multiple stations/plug-ins for averaging/failover between two/several stations
This can already sort of be done now by entering your location by lat long or zip code.  It will choose the nearest station online.  It is an interesting idea on the possibility of extrapolation by triangulation of multiple stations nearby.  It would definitely increase the complexity but should be possible.

Offline thebgrian

  • Newbie
  • *
  • Posts: 7
  • Karma: +0/-0
Re: Virtual Rain Sensor
« Reply #35 on: May 13, 2014, 11:51:01 pm »
Hi, blakem
Thank you for the great plugin. It's a must for automation of sprinkler systems.
I have a data problem- the first day on which I installed the plugin is showing for Accumulated Precipitation Tue, 5/13 -999.00, Mon, 5/12    T Sun, 5/11 T,  for Forecasted Precipitation - Tue, 5/13 0.27   Wed, 5/14 0, for Total Precipitation -998.73.
It will take a big flood to get me over the limit of 0.1 I have set :)
Is there  way to fix the -999.00 precipitation I have showing for Tue, 5/13?

Offline theal

  • Jr. Member
  • **
  • Posts: 51
  • Karma: +0/-3
Re: Virtual Rain Sensor
« Reply #36 on: May 14, 2014, 01:43:15 pm »
BlakeM,

I went ahead and added the total precipitation to the dashboard and will put it in the next release, but I am not sure there is room to put really any more info without crowding it up.

Thank you!

there is a separate request for each day in the past.

How about display ALL future days since it's one call?
I found it extremely useful to have a quick view of week-ahead for my activity schedule.
I see a lot of uses for your plugin beyond just irrigation controller.


I tried to keep the number of device state variables used to a minimum, but if there is a need to export the data in that form then it could be possible.  Right now with a little lua/luup code you can get each individual day from the CSV string.

My major goal (along with water conservation) is to avoid run-offs by drying soil prior and after large rainfall, so I would need to look at future and past days separately.  Also it would be nice to have ability for a single day threshold alarm.
I have not master lua/luup code yet. Do you have an example how to get each individual day from the CSV string? Is it 'ForecastedPrecip[d#]?
Perhaps you can share your irrigation control logic.

Btw, how does Today's forecast reported? Is it for total 24 hours or for remaining of the day?

I don't think this would be very helpful.  I used the QPF since it gives the magnitude of the precipitation, but the PoP has no magnitude.  It just states the probability that greater than 0.01" of rain will fall within the predicted period.  So it basically tells the certainty, but lacks the amount which is what in this application I think you would want to know.  I had thought about multiplying the QPF and the PoP together to give the forecast a weighted value of certainty but I don't know if that would even make any sense to do that.  It was just a thought on how to have a more optimistic approach.

Thanks for clarification. I thought of multiplying the QPF and the PoP too. I still think probability would be helpful, for example, if POP under some threshold, I can ignore QPF for that day.  If not for irrigation, it would be helpful for activity planing.

This can already sort of be done now by entering your location by lat long or zip code.  It will choose the nearest station online. 

Some stations may be missing past rainfall or may not report forecast.  Does station selection account for this?
I do see big discrepancy between my three station in 5 mile radius so averaging would be good.

Thanks again for great work!

Offline blakem

  • Sr. Newbie
  • *
  • Posts: 47
  • Karma: +7/-0
Re: Virtual Rain Sensor
« Reply #37 on: May 14, 2014, 04:09:50 pm »
Hi, blakem
Thank you for the great plugin. It's a must for automation of sprinkler systems.
I have a data problem- the first day on which I installed the plugin is showing for Accumulated Precipitation Tue, 5/13 -999.00, Mon, 5/12    T Sun, 5/11 T,  for Forecasted Precipitation - Tue, 5/13 0.27   Wed, 5/14 0, for Total Precipitation -998.73.
It will take a big flood to get me over the limit of 0.1 I have set :)
Is there  way to fix the -999.00 precipitation I have showing for Tue, 5/13?

Thank you for reporting the problem.  I have just fixed it so today's precipitation value will show "T" for trace amount when the value is negative.  It should be in the next release.  It was working on past days but not the current day.  Unfortunately that only fixes part of the problem so that you won't see a negative total precipitation. 

The other problem is I think that weather station might need service unless there really has been trace amounts of rain for the last several days.  Go to the "Data" tab on the plugin and click on the station name in the bottom right corner to look at what values it is showing on Weather Underground.  On that webpage you can also click the "change station" link at the top of the webpage to display other nearby station names.  You can manually enter the 4 letter airport code or a personal weather station name starting with "pws:" for the location to force the plugin to use a different station.

BTW which station name is displayed on the data tab so I can investigate it just incase there is something else going on.

Offline thebgrian

  • Newbie
  • *
  • Posts: 7
  • Karma: +0/-0
Re: Virtual Rain Sensor
« Reply #38 on: May 14, 2014, 05:41:48 pm »
Thank you, Blakem

You are right, the station that I got based on the zip code ( KTXTHECO9 ) is not reporting precipitation data. After changing the station I'm getting more acceptable readings.

Offline blakem

  • Sr. Newbie
  • *
  • Posts: 47
  • Karma: +7/-0
Re: Virtual Rain Sensor
« Reply #39 on: May 15, 2014, 08:06:55 pm »
Do you have an example how to get each individual day from the CSV string? Is it 'ForecastedPrecip[d#]?

Here is an example that will create a table in lua containing the daily precipitation values.  This is just one example and I am sure there are many ways to accomplish the same thing.
Code: [Select]
function csvToTable(csvString)
   local t={}
   local i=1

   for str in string.gmatch(csvString, "([^, ]+)") do
      t[i] = str
      i = i + 1
   end

   return t
end

--Assume device id of Virtual Rain Sensor is 99
local deviceId = 99

--Get past precipitation as CSV string
local pastPrecipCSV = luup.variable_get("urn:upnp-org:serviceId:VRainSensor","PastPrecip",deviceId)

--Convert CSV string to a lua table
local pastPrecipTable = csvToTable(pastPrecipCSV)

--Number of elements in table
local tableSize = #pastPrecipTable

for day = 1, tableSize do
   --Get past precip for each day
   --pastPrecipTable[day]
end

Btw, how does Today's forecast reported? Is it for total 24 hours or for remaining of the day?
Well before the API changed, it would forecast QPF for a full 24 hour period.  That made it very simple so I could just take the max value of the accumulated precipitation and forecast precipitation for the current day since the time periods overlapped exactly.  Now the forecasted precipitation is only for the time remaining in the day so it is just added to the total.  I don't think this is quite as good but it is what I got.  I actually found I could opt not to use this new best forecast and use the old method, but I am sure the new model is more accurate so it will probably be better in the way of accuracy.

Thanks for clarification. I thought of multiplying the QPF and the PoP too. I still think probability would be helpful, for example, if POP under some threshold, I can ignore QPF for that day.  If not for irrigation, it would be helpful for activity planing.
I have thought quite a bit about how to improve the rain detected decision for forecasted precipitation, but after you posted this I looked some more and found something that does make a little sense.  As stated before I don't think the PoP alone is what you are interested in and the QPF alone only gives the total potential without any certainty.  With a little math you can determine the PQPF(Probabilistic Quantitative Precipitation Forecasting) from both the PoP and QPF.  This combined approach will give the probability that a rain total of a given amount will occur.  This way you could make the decision it will rain when the probability that 0.25in or greater of rain will fall is over a 50% chance.  You would still need these two values of the precipitation amount to exceed and percentage threshold to trigger rain condition, but this is the only other way that makes sense to me.  For right now though I think I will leave it the way it is since it is quite simple how it works now.  If I add this logic in the future I will make it an option to use the probabilistic approach or simply add the QPF to the total just like it does right now.

An article describing this type of analysis.
http://www.srh.noaa.gov/tsa/?n=pqpf_explaination

Some stations may be missing past rainfall or may not report forecast.  Does station selection account for this?
I do see big discrepancy between my three station in 5 mile radius so averaging would be good.
No it does not.  The API actually decides the closest station.  It may be possible to add logic to check for this in the future, but I would have to know what to be checking.  All I can say is pick the best one you think is the most reliable.  In my location I have several weather stations within a few miles and I do see differences, but it from the type of isolated storms that move through here.  I have seen nearly an inch on one station and just a few hundredths on another only 2 miles away.  I verified with radar data that the other site was right on the edge of the line of storms as it moved through so that is one reason this plugin will not always be 100% accurate unless the weather station is in your backyard.  I personally have not seen any trend that one station is more biased, at least not for the amount of precipitation, but I am sure it is possible.  I have seen that some stations do become unavailable from time to time. 

In the case that a station was offline in the past then it will display "N/A" for that day.  If there was rain that day it will just not be included in the total.  So it may show no rain when there really was rain.  From a water conservation point of view it might be a little wasteful, but not critical in my view.  My worry is a case where it detects rain and when there is not sufficient rain causing it to suspend irrigation when it shouldn't.

Offline blakem

  • Sr. Newbie
  • *
  • Posts: 47
  • Karma: +7/-0
Future Development
« Reply #40 on: May 15, 2014, 08:44:26 pm »
Right now I am currently looking into adding an evapotranspiration estimate model to the plugin.  Thank you to Freddan101 for raising up the issue.

While researching it looks like a lot of people are using evapotranspiration estimates to determine the water needs of their crops, plants and trees.  Ideally you would take the total amount of rain and irrigation water in and balance the equation with the water needs determined by the evapotranspiration estimate.  In the case of this plugin there is no way to balance the water in and water out equation completely since it does not know how much water is added by irrigation.  What it can do is subtract away the precipitation lost due to evaporation and used by the plants.

So when it is humid and cool the evapotranspiration amount will be less so it will take longer for the total precipitation to fall below the threshold allowing more days between irrigation.  Likewise when it is hot and dry the evapotranspiration amount will be greater so it will quickly subtract away from the total and drop below the threshold sooner so the irrigation can resume before it dries up.

The evapotranspiration estimate will also be applied to the forecast precipitation as well.  If you think about time will elapse before that precipitation will fall but your plants will still be using water until it falls.  By subtracting the amount of evapotranspiration for each day in the future it will balance the water in and water out equation.

The user will simply enter the crop coefficient which will adjust the amount of evapotranspiration for their grass or plants.  There are plenty of coefficients already determined for most species and growing stages.  For instance most grasses have a coefficient of 0.80 or 80% of the reference crop evapotranspiration.  In theory the days in the past should be able to be set higher since the evapotranspiration model should be resetting the sensor and not the past number of days limit.

For more information on the concepts and calculations read the FAO paper 56
http://www.fao.org/docrep/x0490e/x0490e00.htm

Offline theal

  • Jr. Member
  • **
  • Posts: 51
  • Karma: +0/-3
Re: Virtual Rain Sensor
« Reply #41 on: May 15, 2014, 11:50:18 pm »

Here is an example that will create a table in lua ....

Thank you

Offline sidmsmith

  • Newbie
  • *
  • Posts: 6
  • Karma: +0/-0
Re: Virtual Rain Sensor
« Reply #42 on: May 18, 2014, 11:28:35 am »
This is a great app and Blake has done a great job responding to PMs as well.  Does anyone have an issue with updating the threshold value?  I am trying to change it from the default of .13 to .2 and it keeps reverting back to .13.  I have tried using the Save and Update button, modifying straight in the Advanced tab, and a few other combinations of the two.  Any suggestions?

Offline blakem

  • Sr. Newbie
  • *
  • Posts: 47
  • Karma: +7/-0
Re: Virtual Rain Sensor
« Reply #43 on: May 18, 2014, 02:21:16 pm »
This is a great app and Blake has done a great job responding to PMs as well.  Does anyone have an issue with updating the threshold value?  I am trying to change it from the default of .13 to .2 and it keeps reverting back to .13.  I have tried using the Save and Update button, modifying straight in the Advanced tab, and a few other combinations of the two.  Any suggestions?

Two things you can try is first make sure you enter "0.20" and not just ".2".  It needs the leading zero.  The second is close and reopen your browser.  I have still had issues where my internet explorer seems to become unresponsive to the "Save & Update" button.  I added an extra parameter to the URL request to avoid caching but it does not appear to be that simple.  I have noticed other plugins that have the same problem like the Variable Container plugins "Save" button.  Please let me know if either one of these does not fix your problem.

Offline Rich1983

  • Full Member
  • ***
  • Posts: 202
  • Karma: +1/-0
Re: Virtual Rain Sensor
« Reply #44 on: May 25, 2014, 10:22:56 am »
Hey

Good looking app. I apologise if this has been covered but is it possible to see the prediction in % for the day.

For example: make Lua code to sent prowl alert or something if prediction was above 70% rain.


Thank you

Rich


Sent from my iPad using Tapatalk