Author Topic: Virtual Rain Sensor  (Read 30229 times)

Offline blakem

  • Sr. Newbie
  • *
  • Posts: 47
  • Karma: +7/-0
Virtual Rain Sensor
« on: April 10, 2014, 08:16:48 pm »
Description
The Virtual Rain Sensor is a device that activates or trips when it has rained or will rain.  It does this by retrieving the past and future precipitation(rain, snow, sleet & hail) total for each day over a defined window of time and comparing the total to an adjustable threshold.  When the threshold is exceeded the device will change state to "tripped" indicating that rain is detected.  The intended application of this device is for suspending an automatic irrigation system when it has or will rain.  The threshold value ensures that the irrigation will not be bypassed for instances of light rain and aims to maintain a minimum amount of moisture at the given location.  For forecasted precipitation this device compares the Quantitative Precipitation Forecast(QPF) or the user may choose the Probability of Precipitation(PoP).  A free account with the Weather Underground API is required for this plugin to operate. If you do not already have a key which is used for the weather plugin then follow instructions below.

Requirements\Limitations
  • Must have free developer key from Weather Underground API(see instructions below)
  • Not all locations will have precipitation available or a weather station nearby

Version History
  • 1.0 Initial Release
  • 2.0 Added settings tab to UI, combined API requests on update, and improved update scheduling.
  • 2.1 Committed correct icons
  • 2.2 Fixed stale 'lastUpdate' value(Fixed problem where multiple updates could be requested on the UI in 1 minute exceeding the limit)
  • 2.4 Fixed to allow personal weather stations as location. Current date synced to weather location NOT Vera system time.  Dates displayed on UI are also synced with location and not local system time of web browser.  (Any location can be entered in the world regardless of time zone and the UI can be displayed on a system with a different time zone and it will still show the correct dates)
  • 2.5 Fixed non-available forecast precipitation after the API changed.  Corrected browser caching problem on UI.  Improved error reporting so it defaults to an un-tripped state on a failed update.
  • 2.6 Corrected so negative precipitation values from personal weather stations will not create a negative total, but will show as "T" or "N/A".  Printed total precipitation on the dashboard.
  • 2.7 Added probability of precipitation device state variable for information only and not for decision making.
  • 2.8 Improved the cleanup and release of allocated memory.
  • 3.0 Added variable threshold calculated by evapotranspiration.  Added option for PoP or QPF as forecasted rain trigger. Added trigger for when the current temperature is too low.
  • 3.1 Corrected problem with UI7 not displaying settings tab.


Weather Underground API Key Sign Up
You must obtain a free key for the Weather Underground API before you can use this plugin.  If you already have a key setup with the Weather plugin then you may use that key for both plugins.
  • Go to http://www.wunderground.com/weather/api/d/login.html.
  • Under "Create Your Free Account!", enter your email and choose a password(used only during setup).
  • Pick an arbitrary handle(not used but required) and then click "Sign Up". A confirmation email is sent.
  • Open the confirmation email from the email address you used to sign up.
  • Click the link in the email.
  • On the webpage that loads, under "Already have a wunderground.com account?", enter your email and password you picked then click "Login".
  • On the webpage that loads, click "Explore My Options".
  • On the webpage that loads, click "Purchase Key".  Note the cost of $0 USD per month.
  • On the webpage that loads, fill out the form and click "Purchase Key".
  • On the webpage that loads, your API key is located in the field "Key ID", save this key for later.


Installation
http://apps.mios.com/plugin.php?id=6366


API Requests
The Weather Underground API developer key only allows 500 calls per day and no more than 10 calls in one minute.  Use these equations below to calculate the number of requests this plugin will generate.  Note that it is not possible to exceed either one of these conditions with this plugin alone.
Maximum API requests in 1 min: = 1 + (daysInPast - 1)
Minimum API requests in 1 day: = ( 24 hours / hoursBetweenUpdate ) + daysInPast - 1

Icon State
Grey Raindrop: No rain detected, "Not tripped", total precipitation below threshold
Blue Raindrop: Rain detected, "Tripped", total precipitation above threshold


Variables
urn:upnp-org:serviceId:VRainSensor
   Inputs
   Key: Weather Underground API key
   Location: Site to monitor precipitation, formats may be "lat,long" or 5-digit zip code
   MetricUnits(0 or 1): Sets metric or english units, "1" for metric units and "0" for english units
   DaysInPast(0-8): Number of days in the past, including today, to add the accumulated precipitation in the total
   DaysInFuture(0-8): Number of days in the future, including today, to add the forecasted precipitation in the total
   Threshold: Value to determine tripped state from precipitation total, in units of inches or millimeters
   HoursBetweenUpdate(1-12): Number of hours until the next update will occur
   LandscapeCoefficient(0-1.2): Evapotranspiration crop coefficient sets the water demand of plants in landscape (0=Use fixed threshold)
   LowTempCutoff: Temperature threshold in Celsius or Fahrenheit that the "Tripped" state occurs when the current temperature falls below
   PoPThreshold(0-100): Percentage of PoP that will trigger the "Tripped" state (0=Use QPF instead of PoP)

   Outputs
   PrecipitationTotal: Total amount of precipitation over the defined past and future days
   ForecastedPrecip: CSV(Comma Separated Value) string of the QPF(Quantitative Precipitation Forecast) for each day
   PastPrecip: CSV string of the precipitation total for each day in the past
   ProbabilityOfPrecip: CSV string of the probability of precipitation for each day
   TodaysDate: Current date in format 'MM/DD/YYYY'
   DataSource: Weather Underground Station id used to get accumulated precipitation(past and present)
   PastEvapotranspiration: CSV string of the evapotranspiration for each day in the past
   FutureEvapotranspiration: CSV string of the evapotranspiration for each day in the future

urn:micasaverde-com:serviceId:SecuritySensor1
   Armed: Armed state of the device, "1" for armed and "0" for bypassed
   Tripped: Tripped state of the device, "1" for precip total above threshold and "0" for precip total below threshold
   LastTrip: Unix epoch time when "Tripped" last transitioned from "0" to "1"

urn:micasaverde-com:serviceId:HaDevice1
   LastUpdate: Unix epoch time when data and tripped state last updated


Triggers
Since version 3.0 there are 3 different trigger options. These are OR logic so it only requires one condition to be true for the sensor to trip to a "1" state and all conditions must be false for it to reset to "0" state.

  1. Total Precipitation Threshold
    1a. Fixed amount - A constant threshold that does not change over time
    1b. Evapotranspiration - Threshold is continually set equal to the total evapotranspiration for the given period. The Evapotranspiration landscape coefficient allows for adjusting to various types of crops/plants that have different amount of water demands.
  2. Forecast Method - There are two modes; When QPF is selected the QPF is added to the total precipitation and PoP is ignored.  When a PoP threshold is selected then QPF is ignored and the forecast is triggered by only PoP.
  3. Low Temp Cutoff - Threshold where the sensor will trip to a "1" state when the current temperature drops below the threshold.


Settings
The following table shows the relationships between the length of the window(number of days in the past and future) and the level of the threshold.

Short Window, Low Threshold:  Sensitive to small amounts of rain in one day(quick to set, quick to reset)
Short Window, High Threshold: Sensitive to large amounts of rain in one day(quick to set, quick to reset)
Long Window, Low Threshold:   Sensitive to small amounts of rain in one day(quick to set, slow to reset)
Long Window, High Threshold:  Sensitive to small amounts of rain over several days(slow to set, quick to reset) or a large amount of rain in one day(quick to set, slow to reset)

One caution I would add about choosing settings is the potential for the sensor to remain tripped when there is rain potential that is over the threshold reoccurring for several days, but there is actually no or little rainfall.  Logic could be added to escape this condition, but for now I would just like to see if it is really necessary.  Setting a high enough threshold and a shorter forecast window should minimize the condition from occurring.


Usage In Scene
Add a trigger and choose the Virtual Rain Sensor device

A sensor is tripped
   Device is tripped: fired when total precipitation above threshold
   Device is not tripped: fired when total precipitation below threshold

An armed sensor is tripped
   Device armed is tripped: fired when armed and total precipitation above threshold
   Device armed is not tripped: fired when armed and total precipitation below threshold


Usage In Luup Code
Here are some Luup code examples to display possible uses.  In each example the device number is assumed to be 99 and will need to be changed to your device number.  These are just examples so other parameters may need to be tweaked to fit your needs.

Input Examples
Code: [Select]
--Arm
luup.call_action("urn:micasaverde-com:serviceId:SecuritySensor1", "SetArmed", {newArmedValue="1"}, 99)

--Bypass
luup.call_action("urn:micasaverde-com:serviceId:SecuritySensor1", "SetArmed", {newArmedValue="0"}, 99)

--Force an update of the forecast
luup.call_action("urn:upnp-org:serviceId:VRainSensor", "Update", {}, 99)

--Change to english units
luup.call_action("urn:upnp-org:serviceId:VRainSensor", "SetMetric", {newMetricValue="0"}, 99)

--Change to metric units
luup.call_action("urn:upnp-org:serviceId:VRainSensor", "SetMetric", {newMetricValue="1"}, 99)

--Change the weather underground API key to use "abcd1234"
luup.call_action("urn:upnp-org:serviceId:VRainSensor", "SetKey", {newKeyValue="abcd1234"}, 99)

--Change location to use zipcode "12345"
luup.call_action("urn:upnp-org:serviceId:VRainSensor", "SetLocation", {newLocationValue="12345"}, 99)

--Change the number of days to look in the past(including today) to 5
luup.call_action("urn:upnp-org:serviceId:VRainSensor", "SetDaysInPast", {newDaysInPastValue="5"}, 99)

--Change the number of days to look in the future(including today) to 4
luup.call_action("urn:upnp-org:serviceId:VRainSensor", "SetDaysInFuture", {newDaysInFutureValue="4"}, 99)

--Change the total precipitation threshold to 0.56
luup.call_action("urn:upnp-org:serviceId:VRainSensor", "SetThreshold", {newThresholdValue="0.56"}, 99)

--Change the update interval to 6 hours
luup.call_action("urn:upnp-org:serviceId:VRainSensor", "SetHoursBetweenUpdate", {newHoursBetweenUpdateValue="6"}, 99)

--Change the landscape coefficient to 0.6 (will also change to variable threshold)
luup.call_action("urn:upnp-org:serviceId:VRainSensor", "SetLandscapeCoefficient", {newLandscapeCoefficientValue="0.6"}, 99)

--Change the landscape coefficient to 0 (will also change to fixed threshold)
luup.call_action("urn:upnp-org:serviceId:VRainSensor", "SetLandscapeCoefficient", {newLandscapeCoefficientValue="0"}, 99)

--Change the Probability of precipitation threshold to 60% (will stop using QPF for forecasted rain)
luup.call_action("urn:upnp-org:serviceId:VRainSensor", "SetPoPThreshold", {newPoPThresholdValue="60"}, 99)

--Change the Probability of precipitation threshold to 0% (will start using QPF for forecasted rain)
luup.call_action("urn:upnp-org:serviceId:VRainSensor", "SetPoPThreshold", {newPoPThresholdValue="0"}, 99)

--Change the low temperature cutoff to -10 degrees
luup.call_action("urn:upnp-org:serviceId:VRainSensor", "SetLowTempCutoff", {newLowTempCutoffValue="-10"}, 99)


Output Examples
1. Make decision based on rain or no rain detected
Code: [Select]
--Force update(this step optional)
luup.call_action("urn:upnp-org:serviceId:VRainSensor", "Update", {}, 99)

--Get current state
local tripped = luup.variable_get("urn:micasaverde-com:serviceId:SecuritySensor1","Tripped", 99)

if tripped == "0" then
  --Do not tripped logic here
  --Precipitation not detected, total precipitation below threshold
end

if tripped == "1" then
  --Do tripped logic here
  --Precipitation detected, total precipitation above threshold
end

2. Retrieve total precipitaion
Code: [Select]
--Get total precipitation in past and future
local precipitationTotal = luup.variable_get("urn:upnp-org:serviceId:VRainSensor","PrecipitationTotal", 99)

3. Create a lua table containing the precipitation for each day in the past
Code: [Select]
--Convert CSV string to a lua table(
function csvToTable(csvString)
   local t={}
   local i=1

   for str in string.gmatch(csvString, "([^, ]+)") do
      t[i] = str
      i = i + 1
   end

   return t
end

--Assume device id of Virtual Rain Sensor is 99
local deviceId = 99

--Get past precipitation as CSV string
local pastPrecipCSV = luup.variable_get("urn:upnp-org:serviceId:VRainSensor","PastPrecip",deviceId)

--Convert CSV string to a lua table
local pastPrecipTable = csvToTable(pastPrecipCSV)

--Number of elements in table
local tableSize = #pastPrecipTable

for day = 1, tableSize do
   --Get past precip for each day
   local precipForOneDay = pastPrecipTable[day]
end
« Last Edit: May 21, 2015, 10:20:10 pm by blakem »

Offline RexBeckett

  • Master Member
  • *******
  • Posts: 3888
  • Karma: +477/-10
Re: Virtual Rain Sensor
« Reply #1 on: April 11, 2014, 04:00:23 am »
That is a really comprehensive and well-written set of instructions. Good work, @blakem.

You may want to add instructions for manually installing the icon files while you wait for it to arrive in the App store.
 

Offline sjolshagen

  • Sr. Member
  • ****
  • Posts: 416
  • Karma: +5/-0
    • Wicked Strong Fitness
Re: Virtual Rain Sensor
« Reply #2 on: April 11, 2014, 11:55:39 am »
Possibly a stupid question, but what happens for locations that have frozen precipitation (snow, sleet, ice)?
Vera3/UI5 @ v1.5.622

Offline RichardTSchaefer

  • Master Member
  • *******
  • Posts: 9572
  • Karma: +729/-136
    • RTS Services Plugins
Re: Virtual Rain Sensor
« Reply #3 on: April 11, 2014, 02:25:56 pm »
You might want to discuss what happens with people that use the Weather Underground plugin ...

Can you obtain 2 keys per user ? One for each app ?

You might also want to comment on using the Same API key for the Weather Underground plugin ... or better yet ... search for it and grab it's key if it's already installed.
How often do you Poll ... If they use the same key ... the total needs to be within the Fee App Limit specs.




Offline blakem

  • Sr. Newbie
  • *
  • Posts: 47
  • Karma: +7/-0
Re: Virtual Rain Sensor
« Reply #4 on: April 11, 2014, 09:02:54 pm »
@sjolshagen, that is a good question and in short yes it does include the melted equivalent.  The fields I am getting from the API are of the precipitation totals for the given day.  The definition of precipitation is of all the forms of fallen water as you described so I believed it did include the melted snow equivalent.  I just pulled up some data from days earlier this year and sure enough every day there is snow, there is about 10 to 20 times less precipitation which is about the melted equivalent so I am confident that yes snow and sleet are included in the calculation.

@RichardTSchaefer, I really appreciate your reply.  I should have mentioned more about the potential conflicts.  I limited the amount of days in the past to avoid any issue but if you poll every 30 seconds or less with the weather plugin(2 calls or more per minute) and set the Virtual Rain Sensor to the maximum 8 days and do a reload, then you will create 11 calls in one minute which is over the maximum of 10 calls in one minute.  I am going to work on my code some to reduce my calls by 1 by combining the current day precipitation update and forecast update into one call which is possible by the API.  It is too bad you cannot access the same feature(history) more than once in a single call, but at least I can do the forecast and history together.

It would be possible to search for the weather plugin and copy the key on install to make it simpler but I was not sure that was a good idea or not.  What do you think?

Also it is possible to signup for a second API key with another email.  That is what I did when creating the instructions.

Offline blakem

  • Sr. Newbie
  • *
  • Posts: 47
  • Karma: +7/-0
Re: Virtual Rain Sensor
« Reply #5 on: April 24, 2014, 04:13:43 pm »
Version 2.1 is now available for download in the OP.  A settings tab was added which simplifies the initial setup along with changes to the way the system calls the updates periodically.  Enjoy!

Offline BulldogLowell

  • Hero Member
  • *****
  • Posts: 1571
  • Karma: +190/-85
Re: Virtual Rain Sensor
« Reply #6 on: April 24, 2014, 09:25:51 pm »
Nice plugin.

Has anyone else reported (last version) the device report Lua errors?

I'm running this on a test vera lite with only a few other apps installed, UI5.


Offline blakem

  • Sr. Newbie
  • *
  • Posts: 47
  • Karma: +7/-0
Re: Virtual Rain Sensor
« Reply #7 on: April 24, 2014, 10:53:01 pm »
@Bulldoglowell, I designed it so if there are 5 consecutive communication failures it will set the device into failure.  The idea was to notify the user that it is not updating.  In the latest version 2.1 I write errors to the task status bar on the UI so that feature is not really that important now.  If it becomes a nuisance it could be removed easily.  In any case if the communication issue resolves itself then the plugin will continue to function again even though it shows a device failure.  A reload will clear the failure flag.

Also worth noting there were some quirks with version 1.0 that are now resolved with version 2.1 that could have caused the device failure to report needlessly.  It should be a nonissue now but please let me know of any problems you might have with the latest.

Offline BulldogLowell

  • Hero Member
  • *****
  • Posts: 1571
  • Karma: +190/-85
Re: Virtual Rain Sensor
« Reply #8 on: April 24, 2014, 11:09:12 pm »
Right then, thanks for the quick response. I'll update it when I get the chance and let you know.

Cheers.

Offline Faber Optime

  • Newbie
  • *
  • Posts: 8
  • Karma: +0/-0
Re: Virtual Rain Sensor
« Reply #9 on: April 25, 2014, 11:20:56 am »
Top idea, kudos and appreciation for the work!  :D
Unfortunately I'm not able to get it working though. :(
On all settings for historic precipitation other than None, I'm running into error at every reload:
VRainSensor : Error!, precip section not found in history data.
Device is not showing any rain drop icon (never has, despite uploading the (27bytes, really?) "icon" (.png, but unreadable) files to directory as specified in OP.
If I change historic setting to None, runs without error, but despite that, still haven't managed to see it working.

Offline Sender

  • Hero Member
  • *****
  • Posts: 757
  • Karma: +532/-439
  • Sr. manager of my Vera
Re: Virtual Rain Sensor
« Reply #10 on: April 25, 2014, 01:39:14 pm »
Hi,

installed plugin succesfully. Created account and generated API key. But the plugin wont save my configuration and API key.

Vera keeps saying: VRainSensor : Enter API Key on Device Settings Tab

Do I need to save in a special way?

EDIT: There is a "hidden" button you can go there with "TAB"... its on the configure page...
« Last Edit: April 25, 2014, 01:53:35 pm by SanderL »
Good firmware is more important than good karma. (and this costed me lots of Karma)

Offline blakem

  • Sr. Newbie
  • *
  • Posts: 47
  • Karma: +7/-0
Re: Virtual Rain Sensor
« Reply #11 on: April 25, 2014, 03:55:32 pm »
Top idea, kudos and appreciation for the work!  :D
Unfortunately I'm not able to get it working though. :(
On all settings for historic precipitation other than None, I'm running into error at every reload:
VRainSensor : Error!, precip section not found in history data.
Device is not showing any rain drop icon (never has, despite uploading the (27bytes, really?) "icon" (.png, but unreadable) files to directory as specified in OP.
If I change historic setting to None, runs without error, but despite that, still haven't managed to see it working.
Thank you for your reply and sorry for being on the front end of testing. :) I believe what is happening is a conflict between your system time zone and the time zone of the location you selected.  Is your Vera system time set properly for your time zone(shown in top right corner)? What are you using as your location?  You can private message it to me if you don't want to post it.  For now you are limited to those two time zones being the same.  In short the plugin is requesting precipitation data on a day that does not exist yet.

For the icons, if you manually uploaded them in the "Luup files" section that will not work.  Honestly the best thing to do now since the plugin got approved is go into the app store and install the plugin.  Then the icons will be loaded in the right place.

EDIT: Ok looking at it closer it looks like some locations do not have precipitation available since there are no weather stations reporting precipitation nearby.  I have found some locations outside the united states where there is no precipitation so maybe that is actually your case.  If you post your location I can look it up.
Hi,

installed plugin succesfully. Created account and generated API key. But the plugin wont save my configuration and API key.

Vera keeps saying: VRainSensor : Enter API Key on Device Settings Tab

Do I need to save in a special way?

EDIT: There is a "hidden" button you can go there with "TAB"... its on the configure page...
Glad you got it to work and thank you for letting me know about the button not showing properly.  What browser are you using by the way? Are you using Vera UI5?  I think this will be a simple change to the CSS in the HTML to fix.  Also the settings tab should look like the image attached to the first post.
« Last Edit: April 25, 2014, 10:18:02 pm by blakem »

Offline Sender

  • Hero Member
  • *****
  • Posts: 757
  • Karma: +532/-439
  • Sr. manager of my Vera
Re: Virtual Rain Sensor
« Reply #12 on: April 25, 2014, 04:03:52 pm »
Ui5 and chrome
Good firmware is more important than good karma. (and this costed me lots of Karma)

Offline blakem

  • Sr. Newbie
  • *
  • Posts: 47
  • Karma: +7/-0
Re: Virtual Rain Sensor
« Reply #13 on: April 25, 2014, 05:35:42 pm »
Ui5 and chrome
Thanks, I installed Chrome and duplicated the problem.  It will be fixed in the next release.  I just removed the style so it will just look like a plain button now, but hey you can see it. ;)

Offline Sender

  • Hero Member
  • *****
  • Posts: 757
  • Karma: +532/-439
  • Sr. manager of my Vera
Re: Virtual Rain Sensor
« Reply #14 on: April 26, 2014, 01:39:23 am »
Thanx!
Good firmware is more important than good karma. (and this costed me lots of Karma)