We have moved at community.getvera.com

Author Topic: Brultech ECM-1240 Energy Monitor  (Read 52163 times)

Offline smilligan

  • Full Member
  • ***
  • Posts: 108
  • Karma: +0/-0
Re: Brultech ECM-1240 Energy Monitor
« Reply #30 on: June 24, 2011, 10:05:58 am »
Guessed,

It would be nice if we could use VERA not only as realtime view of power consumption, as well as automatic pass up to MIOS, but as a gateway to other richer web services.

Specifically: It is great to see realtime info in vera, and it would be the icing on the cake if you could simply "pass through" the raw PnP string with HTTP parameters (such as is available on Etherbee) to allow us to send the information further up the food chain to other web services.

In our case we want the option to log to "check-it.ca" or "my1240.com" or our own java web service for data accumulation.

What are your thoughts, comments?

Thanks
Sean

Offline guessed

  • Community Beta
  • Master Member
  • ******
  • Posts: 5301
  • Karma: +92/-22
  • Release compat is not a bolted-on afterthought
Re: Brultech ECM-1240 Energy Monitor
« Reply #31 on: June 28, 2011, 02:55:17 am »
That's how I was originally going to handle the Google integration.  Basically reconfigure the Etherbee to call Vera, on one of it's UPnP calls that, in turn, would effectively call an Action on my plugin.... And then have the plugin call Google / my1240 / etc...

I decided against that for a few reasons:
A) it puts Vera in the middle of every transaction
These extra 'hops' add a certain degree of flexibility, but at the cost of lowered resilience on the overall path to the data being delivered, and I have an architectural pref for things to be autonomous (and keep working if the HA controller goes down, for example)

B) it requires reconfiguration of the Etherbee
Which adds manual configuration steps using a Windows-only tool.  This makes it harder for people to just get going, simply.  I have most of the code to decode the raw buffers, which is the default for the '1240 so it would be the cleanest route for most users....


Given Google is no longer going forward with PowerMeter, I might as well reconfigure my Etherbee back to it's default configuration.

Still wrapping my head about how to make it interface cleanly with Vera given multiple 1240's on a single channel.  I have ideas, ones that will work with Vera's notion of 'upfront' Device creation, but folks won't be able to readily 'know' which MiOS device relates to which ECM unit...

Offline smilligan

  • Full Member
  • ***
  • Posts: 108
  • Karma: +0/-0
Re: Brultech ECM-1240 Energy Monitor
« Reply #32 on: June 28, 2011, 08:54:23 am »
Guessed,

Quote
I decided against that for a few reasons:
A) it puts Vera in the middle of every transaction
These extra 'hops' add a certain degree of flexibility, but at the cost of lowered resilience on the overall path to the data being delivered, and I have an architectural pref for things to be autonomous (and keep working if the HA controller goes down, for example)

But it seems you must plan for some mechanism to allow "unaltered" ECM data to get beyond the VERA platform.  There are many  people developing web services around the ECM protocol.  Our company is an example.   

Also, there is always a weak link in every chain, and yes VERA is the weak link by adding another HOP.  However, the beauty of ECM message set is that it supports Watt Seconds in each GET, so regardless of howmany messages are dropped, i can always calculate energy usage between messages.

I guess the question is:  Do you intend to "stop" the message at VERA?  If so, that pretty much rules us out as we must be able to get the messages to more powerful reporting services, and be forced to configure ECM to point to existing web services where power reporting is richer and has more momentum.   Maybe start another "fork"? or??  let me know your thoughts.

One final note:  You can simply add a field to the main brultech device that allows the user to specify a "prepended" string to the PnP string, and simply PASS this up the WAN interface if the field is populated (length > 0).   Then is available if needed, or not.

Quote
B) it requires reconfiguration of the Etherbee
Which adds manual configuration steps using a Windows-only tool.  This makes it harder for people to just get going, simply.  I have most of the code to decode the raw buffers, which is the default for the '1240 so it would be the cleanest route for most users....
Given Google is no longer going forward with PowerMeter, I might as well reconfigure my Etherbee back to it's default configuration.

I did not know Google pulled the plug (no pun intended) on powermeter.  Nonetheless there are other services that have been around for sometime that can be used as well..  Google Power Meter is not the only service..

Quote
Still wrapping my head about how to make it interface cleanly with Vera given multiple 1240's on a single channel.  I have ideas, ones that will work with Vera's notion of 'upfront' Device creation, but folks won't be able to readily 'know' which MiOS device relates to which ECM unit...

hmmm..  ECM used to "print" serial numbers on the 1240s.   These serial numbers were also used in the binary, ascii and PnP message sets.  Not sure why they stopped putting Serial #s on outside of unit.  At any rate you could always TELL the user to ADD one ECM at a time to the Etherbee network.  Once it is added, they can label the device in VERA.   Furthermore, you could have somehere in the Plugin primary device the actual serial number that was presented as the PnP data string and instruct the user to "brother label" it to the ECM.   And then start on next, etc..   Or maybe get Brultech to start labelling the ECMs again with Serial #s..  But BOTH of the above assume you have a field in the VERA device that shows the "scraped" serial number as derived from GET.   The above is just some thoughts, and i may be WAY off base..    Nonetheless, I am very interested in seeing/testing what you develop once you decide the best route for addressing this next phase of your plugin.

In Closing: Your plugin is great!!!  You should be very proud of your work.  I wish I were as altruistic as you, but unfortunately, my father's capitalism runs deep in my blood.

Kind Regards

Offline guessed

  • Community Beta
  • Master Member
  • ******
  • Posts: 5301
  • Karma: +92/-22
  • Release compat is not a bolted-on afterthought
Re: Brultech ECM-1240 Energy Monitor
« Reply #33 on: June 28, 2011, 11:56:26 am »
It's not that it can't be done, it's just that it's not my first priority.  The best way to do what you're describing is to outline, and implement, a common UPnP Service that can "send stuff" to remote Power Collection entities.

If this can be standardized, and made neutral of the specific "format" that the data must be sent over the wire (eg, not a URL-like ECM string), then ALL of the Energy Monitoring plugins could call it.

In turn, there would be multiple implementations of this Service, to handle sending stuff to My1240.com, Check-it.ca, etc).

If you build that Service/Device, and it's vendor neutral, then I'm sure you'll get all of the Power measurement Plugins connecting to it.... meaning you'd not only cover the ECM-1240, but likely the TED5000, and the CurrentCost EnviR...

It's probably the way that Vera's own EMonitoring should be done, so we can "unplug" it if folks wanted to (along with plugins for Notification that could be plugged)



BTW, they still write the Serial# on each device, but that's not the problem.  The problem is that the data will come in some unpredictable ordering, and I'll have to "allocate out" Children to each ECM as it does.

It'll be first come, first served... at least the first time around.

So the first ECM would get Child devices 1..7, the next would get 8..14, etc, and the user would have to indicate that they have 14 children (as they do today).  The problem is that with ECM-1 and ECM-2, you'd never know which one "got" 1..7 and which got 8..14 (etc) - at least not without opening up the Child and looking at it's properties (where I can store the Serial#)

Not really a big deal, but overall I preference making it simpler for the user (and not having them configure Serial#'s and the like)

Offline futzle

  • Beta Testers
  • Master Member
  • *****
  • Posts: 3260
  • Karma: +192/-9
Re: Brultech ECM-1240 Energy Monitor
« Reply #34 on: June 28, 2011, 05:20:15 pm »
The best way to do what you're describing is to outline, and implement, a common UPnP Service that can "send stuff" to remote Power Collection entities.

If this can be standardized, and made neutral of the specific "format" that the data must be sent over the wire (eg, not a URL-like ECM string), then ALL of the Energy Monitoring plugins could call it.

This is exactly what I recommend.  I have a script that bridges Vera to the free sysadmin monitoring software Nagios.  It polls a device's "Watts" variable every minute (this period is configurable) and plops the result into my network status along with ping times and UPS statuses and disk usages.

It's all done with curl and xsltproc, and the fact that the energy monitor is a CurrentCost EnviR isn't known to Nagios.  Vera abstracts that away.

Anyone who wants the script can PM me.

Offline smilligan

  • Full Member
  • ***
  • Posts: 108
  • Karma: +0/-0
Re: Brultech ECM-1240 Energy Monitor
« Reply #35 on: June 30, 2011, 01:28:05 pm »
Quote
It's not that it can't be done, it's just that it's not my first priority.  The best way to do what you're describing is to outline, and implement, a common UPnP Service that can "send stuff" to remote Power Collection entities.

I think one of the problems that might be experienced with this approach is that everyone's logging requirements might be different relative to the specific power monitoring device.  Not that the uPnP forwarding service cannot format the info, but rather the raw info as captured by plugin might "drop" important info that it in turn makes available to the uPnP service.

For example, in the case of brultech, they have a nice piece of info called WattSeconds for each channel.   From any logging/reporting service this is probably the most crucial.

My guess is that in your current Brultech plugin you discard that in the stream and just show current wattage of last message set.

So developing uPnP service to "forward info" as gathered by your plugin might not work as expected.


Quote
If this can be standardized, and made neutral of the specific "format" that the data must be sent over the wire (eg, not a URL-like ECM string), then ALL of the Energy Monitoring plugins could call it.    In turn, there would be multiple implementations of this Service, to handle sending stuff to My1240.com, Check-it.ca, etc).

I see your point, and this makes perfect sense in theory.


Quote
If you build that Service/Device, and it's vendor neutral, then I'm sure you'll get all of the Power measurement Plugins connecting to it.... meaning you'd not only cover the ECM-1240, but likely the TED5000, and the CurrentCost EnviR...

I like the way you are thinking, and in theory it would be great to have this uPnP service for each monitoring device and service logging combination.   This would allow services (like ours) to monitor record various monitoring hardware via VERA.  And it would be great..   I wonder that even if you can get everyone to "play nicely" you would find that you have similar data from various monitoring devices that would allow for single point of logging/reporting.

Quote
BTW, they still write the Serial# on each device, but that's not the problem.  The problem is that the data will come in some unpredictable ordering, and I'll have to "allocate out" Children to each ECM as it does.  It'll be first come, first served... at least the first time around.  So the first ECM would get Child devices 1..7, the next would get 8..14, etc, and the user would have to indicate that they have 14 children (as they do today).  The problem is that with ECM-1 and ECM-2, you'd never know which one "got" 1..7 and which got 8..14 (etc) - at least not without opening up the Child and looking at it's properties (where I can store the Serial#)

Yes, this is the way i would suspect it needs to be done.  It may not be plug and play, but it will work and gives you some level of control.  Especially from a support perspective when you need to "swap" equipment.

Quote
Not really a big deal, but overall I preference making it simpler for the user (and not having them configure Serial#'s and the like)

Wish i could offer up a suggestion to provide this level of Plug/Play operation, but right now my mind is drawing a blank.

Nonetheless, once you have something ready that supports multiple ECMS via Etherbee, let me know.   Have "guinea pig" will travel.

PS: We continue to be impressed with your plugin.   Thanks for your hard work.

Kind Regards

Offline guessed

  • Community Beta
  • Master Member
  • ******
  • Posts: 5301
  • Karma: +92/-22
  • Release compat is not a bolted-on afterthought
Re: Brultech ECM-1240 Energy Monitor
« Reply #36 on: July 18, 2011, 10:06:16 am »
Support added for multiple ECM-1240's on a single EtherBee or Mux'd Serial port.  Changes pushed to the latest over at the home, along with full source code history:

    http://code.mios.com/trac/mios_brultech-power-monitor

Offline shifter775

  • Sr. Newbie
  • *
  • Posts: 20
  • Karma: +0/-0
Re: Brultech ECM-1240 Energy Monitor
« Reply #37 on: July 27, 2011, 11:33:33 am »
Looks like they stopped selling the Plug & Play packages. Maybe a result of Google PowerMeter going away?

Does anyone know how hard it is to configure a standard package to work with this plugin?

I don't have MS Windows around, but I'm comfortable configuring things via a serial port. The downloads I can see on the Brultech website look like they are for windows only. Couldn't find a manual either.

Any pointers on getting this to work now that Plug & Play seems to be gone are much appreciated.

TIA

Offline guessed

  • Community Beta
  • Master Member
  • ******
  • Posts: 5301
  • Karma: +92/-22
  • Release compat is not a bolted-on afterthought
Re: Brultech ECM-1240 Energy Monitor
« Reply #38 on: July 27, 2011, 12:56:33 pm »
All of Brultech's current [re]configuration tools are Windows-based. 

If you had/borrow a Windows machine, you could reconfigure the ECM-1240 to emit the ASCII format that I'm using, it's just a few radio button options and you'd have it.... even when GPM goes away.  The reconfig is really simple (if you have a box to run their tools)

I had a number of HW Devices that all did the same thing (X-CTU for XBee's, WIZNet for SR110's, and the Brultech stuff... which partially uses the former) so I broke down and got myself a Netbook.  It made things a little easier than running VirtualBox (or whatever) just to get a Windows machine on my Mac.


Anyhow, I have most of the code "inside" the Plugin for handling the Binary format for the non-PnP models, but I've never reconfigured mine over to the "default".  With GPM going away, it's probably a good time to complete that support.

Give me two weekends and I'll complete (and test) the code so it can work without reconfiguration on the Binary-format models.

Offline smilligan

  • Full Member
  • ***
  • Posts: 108
  • Karma: +0/-0
Brultech w/ EtherX and two ECMs stops refreshing
« Reply #39 on: August 03, 2011, 11:43:53 am »
Guessed,
Hopeful you can help me out with a slight problem.

We have latest vera, with your latest brultech plugin (i think it is latest, as i am not sure what changes you may have made in last few weeks)

Configuration:
1) Vera
2) EtherX Gateway
3) Two ECM1240s connected to EtherX gateway

The brultech plugin seems to work ok for some period of time (not sure how long), but then stops refreshing data.   I have found that simply hitting "save" button on VERA web interface seems to wake up the plulgin and i start getting data realtime..  at least for a short period.  After which point the web UI plugin stops updating and i must click on "Save" again.

I am not 100% positive that the "save" fixes the problem or it is merely coincidental, but it seems to have solved teh problem the last 4 times it has occured.

Any ideas would be greatly appreciated.

PS: we have a USB drive connected to VERA for USB logging and not sure if this new addition might be related to problem.   We added USB logging about the same time we noted this strange behavior.

Thanks
Sean

Offline guessed

  • Community Beta
  • Master Member
  • ******
  • Posts: 5301
  • Karma: +92/-22
  • Release compat is not a bolted-on afterthought
Re: Brultech ECM-1240 Energy Monitor
« Reply #40 on: August 03, 2011, 11:37:49 pm »
Sean,
When it occurs next, can you email me a copy of your complete log file?  I'm guessing I'll need the whole file in order to debug it and capture the right messages from the Plugin itself.  I suspect the last-event occurrence data has probably disappeared from your logs by now.

You already have my Email address, so you don't need to post that content here (in case it's sensitive in any way).

Offline guessed

  • Community Beta
  • Master Member
  • ******
  • Posts: 5301
  • Karma: +92/-22
  • Release compat is not a bolted-on afterthought
Re: Brultech ECM-1240 Energy Monitor
« Reply #41 on: August 06, 2011, 01:19:00 am »
Sean,
I've uploaded a new copy, which includes some new files, and changed the installation instructions at:
    http://code.mios.com/trac/mios_brultech-power-monitor/wiki/Installation-UI4

there are a bunch of new files that now need to be uploaded, so you might as well upload all 9x of them.  They'll overlay any existing config since I haven't changed any names, just added a few, and a little refactoring of the code to get ready to support native/raw format.

It adds the poll() method we discussed, that executes every 5 minutes, to attempt to "reset" the connection in the power outage situation you'd experienced.  Let me know how that goes, since it's a bit of an experiment.  If it works-around the issue successfully, I'll open a Ticket to get MCV to fix that particular situation with [TCP-Based] IO dropouts.


As a potentially related issue, if you have two ECM units, and they "start" at the same time (after a power failure) then they'll routinely transmit data "at the same time".  This causes data collisions, so you'll want to see each ECM to a slightly different data publication/update frequency.

eg. 50s for the first, 51s for the second, 52s for the third, and so on...

Offline mckervey

  • Newbie
  • *
  • Posts: 5
  • Karma: +0/-0
Re: Brultech ECM-1240 Energy Monitor
« Reply #42 on: August 28, 2011, 08:13:42 pm »
So close, but I need help.

I'm not seeing values in the web interface (see attachment).

I do see data in the LuaUPnP.log

 08/28/11 20:10:28.619   luup_log:25: Brultech PowerMeter: debug: processIncomingText:: Buffer=sec=190566&v=1253&c1w=5179&c2w=2734&wsa1=624668649&wsa2=317966239&wsap1=16&wsap2=29941&A1w=&A1ws=38425565&A2w=85&A2ws=4721477&A3w=286&A3ws=42990710&A4w=434&A4ws=28007620&A5w=337&A5ws=23881848&dev=34180&id=3&Resp=, Length=212 <0x3c10>
50      08/28/11 20:10:28.620   luup_log:25: Brultech PowerMeter: debug: Skipping buffer=sec=190566&v=1253&c1w=5179&c2w=2734&wsa1=624668649&wsa2=317966239&wsap1=16&wsap2=29941&A1w=&A1ws=38425565&A2w=85&A2ws=4721477&A3w=286&A3ws=42990710&A4w=434&A4ws=28007620&A5w=337&A5ws=23881848&dev=34180&id=3&Resp= <0x3c10>
50   

I'm using a wiznet serial to ethernet converter and have the Brultech in ASCI mode obviously.

Any ideas?

Offline guessed

  • Community Beta
  • Master Member
  • ******
  • Posts: 5301
  • Karma: +92/-22
  • Release compat is not a bolted-on afterthought
Re: Brultech ECM-1240 Energy Monitor
« Reply #43 on: August 28, 2011, 08:59:41 pm »
You are nearly there, there's a prefix missing:

50   08/28/11 17:49:56.769   luup_log:234: Brultech PowerMeter: debug: processIncomingText:: Buffer=GET /usr/uuuuuuu/dev.php?sec=4044681&v=1216&c1w=&c2w=&wsa1=386751&wsa2=1129950&wsap1=119384&wsap2=400521&A1w=&A1ws=89628&A2w=&A2ws=164811&A3w=&A3ws=&A4w=&A4ws=47655&A5w=&A5ws=347427&dev=xxxxx&id=2&Resp= HTTP/1.0, Length=210 <0x4c12>


as mine is still uploading to Brultech's my1240.com service (for now)

I hunt for a pattern like:
    local result = s:match("^GET /.+%?(.-) HTTP/%d%.%d$")

so it doesn't really matter what that leading part of the string looks like, but it does need that general form.

There should be a spot to put that into the configuration, probably near where you configured it over to ASCII format.

Offline mckervey

  • Newbie
  • *
  • Posts: 5
  • Karma: +0/-0
Re: Brultech ECM-1240 Energy Monitor
« Reply #44 on: August 29, 2011, 11:07:59 am »
I played around trying to configure the ECM-IA app (set url info menu option) to output with the text you mentioned, but no avail.

Instead I tried forcing it by changing a function in L_BrultechMeter1.lua (I know this isn't preferrable, I just want to get it working  ::))

function processIncomingText(s)
    debug(string.format("processIncomingText:: Buffer=GET /usr/uuuuuuu/dev.php?%s, Length=%s", s, tostring(s:len())))

    local result = s:match("^GET /.+%?(.-) HTTP/%d%.%d$")

    if (result ~= nil) then
        decodeBufferText(result)
    else
        debug("Skipping buffer=GET /usr/uuuuuuu/dev.php?" .. s)
    end
end

The output in the log file looks correct, but it's still not showing any data in the UI4 gui :(

08/29/11 10:57:59.980   luup_log:58: Brultech PowerMeter: debug: processIncomingText:: Buffer=GET /usr/uuuuuuu/dev.php?sec=243774&v=1253&c1w=644&c2w=&wsa1=785931537&wsa2=394009123&wsap1=16&wsap2=32849&A1w=&A1ws=46067518&A2w=10&A2ws=6486579&A3w=138&A3ws=54418949&A4w=22&A4ws=32387183&A5w=25&A5ws=30193765&dev=34180&id=3&Resp=, Length=205 <0x3c10>
50      08/29/11 10:57:59.981   luup_log:58: Brultech PowerMeter: debug: Skipping buffer=GET /usr/uuuuuuu/dev.php?sec=243774&v=1253&c1w=644&c2w=&wsa1=785931537&wsa2=394009123&wsap1=16&wsap2=32849&A1w=&A1ws=46067518&A2w=10&A2ws=6486579&A3w=138&A3ws=54418949&A4w=22&A4ws=32387183&A5w=25&A5ws=30193765&dev=34180&id=3&Resp=