We have moved at community.getvera.com

Author Topic: Energy/Temperature Publication using SmartEnergyGroups.com (SEG)  (Read 13956 times)

Offline ccclapp

  • Sr. Newbie
  • *
  • Posts: 44
  • Karma: +1/-0
Re: Energy/Temperature Publication using SmartEnergyGroups.com (SEG)
« Reply #45 on: March 24, 2015, 07:00:40 pm »
I think you're correct.  GEM can nor report all channels directly to ISY but ISY doesn't report ANYTHING to SEG.

Also, most relevant to this thread is the fact that on any device ONLY GUESSED'S SCRIPT HANDLES PLUGS.  For me that's a key missing link...
Any chance you can/would take a stab at adding the polling/frequency edit to his script as he described in response to your comment about the impact of the lack of it?  I wish I could, but can't script :-(

Offline scyto

  • Jr. Member
  • **
  • Posts: 88
  • Karma: +1/-0
Re: Energy/Temperature Publication using SmartEnergyGroups.com (SEG)
« Reply #46 on: March 25, 2015, 03:47:39 pm »
Any chance you can/would take a stab at adding the polling/frequency edit to his script as he described in response to your comment about the impact of the lack of it?  I wish I could, but can't script :-(

Me neither, I can read one and make educated guess, actually figuring out where the braces go and what should be between them - no chance :-)  - unless I go pick up a book on lua scripting (my tip, download notepad++ on windows - makes reading these files a breeze)

Offline celbrandt

  • Newbie
  • *
  • Posts: 3
  • Karma: +0/-0
Re: Energy/Temperature Publication using SmartEnergyGroups.com (SEG)
« Reply #47 on: April 15, 2015, 07:04:08 am »
I have now changed the script to send KWH data to SEG every 30 minutes. I have added a new parameter to each device I want to report to SEG, and modified the script to be used in a scene instead of under "startup-lua". This way I can easyli adjust how often values should be send to SEG, and if a device is asleep for several hours and hence has not updated the KWH for a long time nothing will be send to SEG untill the KWH is updated by the device.

The reason I need the extra paramenter is because SEG needs the delta KWH hence I need to know the prevous send value in order to calucalte the delta.

I have created a new paramenter for each device with the same serviceId as the original KWH (under the advanced tab->new variable"

then created a new new scene running every 30 min doing nothing but running this LUA script ( you can of course keep the exisiting script in start up lua for watts devices that should report watts in "real time" to SEG)

local http = require('socket.http')
http.TIMEOUT = 5
 
local SEG_SITE = "15e38bcc3f0d3d7"
local SEG_URL = "http://api.smartenergygroups.com/api_sites/stream"
local CALC_SID="urn:micasaverde-com:serviceId:EnergyMetering1"
 
SEG_DEVICES = {
   {node='vera', stream='e_beggekwh',   deviceId=3, serviceId='urn:micasaverde-com:serviceId:EnergyMetering1', serviceVar="KWH"},
   {node='vera', stream='e_northqt15',   deviceId=7, serviceId='urn:micasaverde-com:serviceId:EnergyMetering1', serviceVar="KWH"},
}

local i=1
while SEG_DEVICES do
    local v=SEG_DEVICES
    local oldCalc1 = luup.variable_get(CALC_SID, "KWHold", v.deviceId )
    local newCalc1 = luup.variable_get(CALC_SID, "KWHnew", v.deviceId)
    local oldCalc=tonumber(oldCalc1)
    local newCalc=tonumber(newCalc1)
    if (newCalc>oldCalc) then 
      luup.variable_set(CALC_SID, "KWHold", newCalc, v.deviceId)
      segData = string.format("(site %s (node %s ? (%s %s)))",  SEG_SITE, v.node, v.stream, newCalc-oldCalc)
      http.request(SEG_URL, segData)
    end
    i=i+1
 end


I have only had the veralite for 1 month and this is my first script written i LUA, (and the first script I have programmed for the last 3 years), hence please bear with me if it could have been done smarter, but I had to reuse as much as possible from guessed original script :-)
« Last Edit: April 15, 2015, 11:03:32 am by celbrandt »

Offline gerardosamara

  • Full Member
  • ***
  • Posts: 200
  • Karma: +5/-2
Re: Energy/Temperature Publication using SmartEnergyGroups.com (SEG)
« Reply #48 on: April 16, 2015, 04:28:14 am »
Great stuff ... I have setup 3 streams ( temperature , energy watts and Kwh ) and they are displayed correctly on the SEG site ( Thanks Guessed)

But i noticed that there are some disconnections of the streams as per alert received from the SEG site below .Is it a known issue ?

Recent Device Events

@ your device dsb05 is missing, last known value was: 21.0 C 7 hours ago

@ your device dsb05 is back! with value: 21.0 C  9 hours ago

@  your device dsb05 is missing, last known value was: 22.0 C 4 hours ago

@ your device dsb05 is back! with value: 22.0 C  17 hours ago

« Last Edit: April 17, 2015, 03:18:59 am by bottrader »
VeraLite+VeraEdge/EnOcean version US en UI7.1.17 + Vera Mobile UI7 V7.15.17.40 + Imperihome pro V2.8.3
Wall switch : Leviton VRS15 , Smart switch : Aeotec DSC24 , Motion sensor : Aeotec 4in1 DSB05 + Everspring HSP02 , Cameras Dlink 742L + Foscam 8910 + Foscam 9853EP

Offline guessed

  • Community Beta
  • Master Member
  • ******
  • Posts: 5301
  • Karma: +92/-22
  • Release compat is not a bolted-on afterthought
Re: Energy/Temperature Publication using SmartEnergyGroups.com (SEG)
« Reply #49 on: April 16, 2015, 02:03:35 pm »
The two cases where I've seen SEG Report like that correlate to the client(s) "reporting too fast" and "reporting too slow".

In my case, I had a few other things reporting across the same entity and/or there was a volatile value in the list of stuff I sent over to SEG.

With energy data, unless it's some sort of daily-summary, I'd expect you'd get a reasonable level of volatility/change, so it's possible that it's too fast and SEG is throttling.

Quickest way to find out, esp if there's a [relatively] constant pattern, is to look at the Tools/Trace output (that's a SEG Menu/Tool) and see if your device is reporting (or not) during these periods.

For reference, my on-schedule Energy-based samples are sent to SEG ~5 minutes, with data collections every 90s.  My on-change openHAB transmissions are based upon change of the source data (in my case Nest-openHAB, polling every minute, and Weather, polling every 30 minutes)

Offline gerardosamara

  • Full Member
  • ***
  • Posts: 200
  • Karma: +5/-2
Re: Energy/Temperature Publication using SmartEnergyGroups.com (SEG)
« Reply #50 on: April 16, 2015, 06:01:50 pm »
OK @Guessed,

The decision to have on or off status of a stream is based on the missing period parameter in the API details ( 5000 Seconds )

As per the API trace , it is confirmed that the missing requests from Vera are the cause of these events due to no change reported to SEG within the API missing period of 5000 Seconds :

- The wall plug is always making energy report to SEG within the 5000 s period of time
- The 4in1sensor (DSB)  does not report temperature report  sometimes to SEG within the 5000 s period of time.

I was able to force a temperature report by changing the location of the DSB05 multisensor and to have then the status back on line for this device.

Conclusion : The root cause is sometimes too slow data , but this is consistent with the behavior of the devices which sends report only in case of change of value ( temperature or watts/kwh ).

« Last Edit: April 16, 2015, 06:03:42 pm by bottrader »
VeraLite+VeraEdge/EnOcean version US en UI7.1.17 + Vera Mobile UI7 V7.15.17.40 + Imperihome pro V2.8.3
Wall switch : Leviton VRS15 , Smart switch : Aeotec DSC24 , Motion sensor : Aeotec 4in1 DSB05 + Everspring HSP02 , Cameras Dlink 742L + Foscam 8910 + Foscam 9853EP

Offline guessed

  • Community Beta
  • Master Member
  • ******
  • Posts: 5301
  • Karma: +92/-22
  • Release compat is not a bolted-on afterthought
Re: Energy/Temperature Publication using SmartEnergyGroups.com (SEG)
« Reply #51 on: April 16, 2015, 07:11:06 pm »
Some posts back, I pushed what was an almost-complete mod to the existing code to make it push data periodically. 

It only needed to be wrapped in a function declaration, and scheduled/re-scheduled via luup.call_timer() and it would provide both "periodic" as well as the existing "change-only" push to SEG.

That's one option. 

The next is to try the Scene-implemented version posted above, with the Scene acting as the timer. 

The third it to include "something" in the SEG list that does change often enough... Outdoor Temp, or anything really... it doesn't have to be data that SEG actually graphs  ;)

Offline ccclapp

  • Sr. Newbie
  • *
  • Posts: 44
  • Karma: +1/-0
Re: Energy/Temperature Publication using SmartEnergyGroups.com (SEG)
« Reply #52 on: April 16, 2015, 07:22:06 pm »
I have now changed the script to send KWH data to SEG every 30 minutes. I have added a new parameter to each device I want to report to SEG, and modified the script to be used in a scene instead of under "startup-lua". This way I can easyli adjust how often values should be send to SEG, and if a device is asleep for several hours and hence has not updated the KWH for a long time nothing will be send to SEG untill the KWH is updated by the device.

The reason I need the extra paramenter is because SEG needs the delta KWH hence I need to know the prevous send value in order to calucalte the delta.

I have created a new paramenter for each device with the same serviceId as the original KWH (under the advanced tab->new variable"

then created a new new scene running every 30 min doing nothing but running this LUA script ( you can of course keep the exisiting script in start up lua for watts devices that should report watts in "real time" to SEG)



Hi  celbrandt

I'm traveling for a bit and can't try your script, but wanted to thank you for your effort in adding periodic poling/reporting to guessed' script.

Is it working well?

Thanks again I look forward to this!

Offline celbrandt

  • Newbie
  • *
  • Posts: 3
  • Karma: +0/-0
Re: Energy/Temperature Publication using SmartEnergyGroups.com (SEG)
« Reply #53 on: April 19, 2015, 03:26:23 pm »
Hi CCCLAP

my script seems to Work flawlessly.  The last three days, KwH consumption for the last hours has been reported for 3 different stream every hour without any glitches.

But we still owe it all to Guessed. Without any LUA experience I could not have done it without all the hard work done initially by Guessed !




Offline bhunt99

  • Newbie
  • *
  • Posts: 5
  • Karma: +0/-0
Re: Energy/Temperature Publication using SmartEnergyGroups.com (SEG)
« Reply #54 on: May 01, 2015, 10:33:31 am »
hoping someone can help me troubleshoot this. I initially set up the original script to send energy and power data from an aeon labs smart switch hooked up to my washing machine. In SEG I created the dummy device, did the discovery, and it discovered the two streams and connected them to my device. Great so far.
I then tried to add streams for 2 home energy monitors. 2 monitors - power and energy for each. I'm not having any luck picking them up in SEG. When I do an api trace I almost never see anything at all in there, but the 2 streams for my washing machine device do regularly update. I tried to manually create another device thinking maybe I need to do that first, but I get a message about 'permalink already used'.

Here's my script
Code: [Select]
local http = require('socket.http')
http.TIMEOUT = 5
 
local SEG_SITE = "478c91a17fca017"
local SEG_URL = "http://api.smartenergygroups.com/api_sites/stream"
local segLog = function (text) luup.log('SEG Logger: ' .. (text or "empty")) end
 
SEG_DEVICES = {
    {node='vera', stream='p_washing_machine',   deviceId=19, serviceId='urn:micasaverde-com:serviceId:EnergyMetering1', serviceVar="Watts"},
    {node='vera', stream='e_washing_machine', deviceId=19, serviceId='urn:micasaverde-com:serviceId:EnergyMetering1', serviceVar="KWH"},
    {node='vera', stream='p_hem1', deviceId=28, serviceId='urn:micasaverde-com:serviceId:EnergyMetering1', serviceVar="Watts"},
    {node='vera', stream='e_hem1', deviceId=28, serviceId='urn:micasaverde-com:serviceId:EnergyMetering1', serviceVar="KWH"},
    {node='vera', stream='p_hem2', deviceId=37, serviceId='urn:micasaverde-com:serviceId:EnergyMetering1', serviceVar="Watts"},
    {node='vera', stream='e_hem2', deviceId=37, serviceId='urn:micasaverde-com:serviceId:EnergyMetering1', serviceVar="KWH"}
  }
 
local function findStream(deviceId, serviceId, serviceVar)
  -- TODO: Change from a simple scan to something that'll support more entries efficiently.
  for k, v in pairs(SEG_DEVICES) do
    if (v.deviceId == deviceId and
        v.serviceId == serviceId and
        v.serviceVar == serviceVar) then
      return v.node, v.stream
    end
  end
 
  return nil, nil
end
 
local function initWatch()
  for k, v in pairs(SEG_DEVICES) do
    luup.variable_watch('segWatch', v.serviceId, v.serviceVar, v.deviceId)
  end
end
 
function segWatch(deviceId, serviceId, serviceVar, oldValue, newValue)
  local nodeName, streamName = findStream(deviceId, serviceId, serviceVar)
 
  if (nodeName == nil or streamName == nil) then
    segLog(string.format("Node not found for %s,%s,%s", deviceId, serviceId, serviceVar))
    return
  end
 
  -- TODO: Encoding
  segData = string.format("(site %s (node %s ? (%s %s)))", SEG_SITE, nodeName, streamName, newValue)
  segLog(segData)
 
  result, status = http.request(SEG_URL, segData)
end
 
initWatch()

Offline bhunt99

  • Newbie
  • *
  • Posts: 5
  • Karma: +0/-0
Re: Energy/Temperature Publication using SmartEnergyGroups.com (SEG)
« Reply #55 on: May 01, 2015, 02:13:01 pm »
I think I have it figured out now and I was just thinking of things on the SEG side in wrong way. I was expecting a device in SEG for each device in Vera. Now I set up 1 device in SEG called Vera w/ multiple streams coming from the different physical devices I have and everything seems to be working.

Is that the correct way to set it up on the SEG side?
Site = Vera
Device = Vera
Stream 1, 2, 3, 4, 5 etc. = p_hem1, e_hem1, p_hem2, e_hem2, p_washing_machine, etc.

Offline gerardosamara

  • Full Member
  • ***
  • Posts: 200
  • Karma: +5/-2
Re: Energy/Temperature Publication using SmartEnergyGroups.com (SEG)
« Reply #56 on: October 07, 2015, 10:11:24 am »
Hum , I received a mail from SG this morning ,


Smart Energy Groups has been free for the majority of our members now for a long time now, and we need your support in order to continue.

This means that Smart Energy Groups is transitioning to a paid service in order for it to continue.

When you log in next, you will be prompted to select from our Community or Premium plans. This will set you up with an account and all the other necessary things, like your credit card details etc.


EDIT : After checking the SEG site , the free subscription still exists but with only 3 streams
« Last Edit: February 02, 2016, 09:35:21 pm by gerardosamara »
VeraLite+VeraEdge/EnOcean version US en UI7.1.17 + Vera Mobile UI7 V7.15.17.40 + Imperihome pro V2.8.3
Wall switch : Leviton VRS15 , Smart switch : Aeotec DSC24 , Motion sensor : Aeotec 4in1 DSB05 + Everspring HSP02 , Cameras Dlink 742L + Foscam 8910 + Foscam 9853EP