We have moved at community.getvera.com

Author Topic: HTTP callback  (Read 296 times)

Offline Buxton

  • Full Member
  • ***
  • Posts: 209
  • Karma: +12/-0
HTTP callback
« on: May 02, 2018, 07:08:12 pm »
Apologies in advance if this topic has been discussed somewhere, but I couldn't find anything definitive in the forum archives.  Or for that matter if the answer is self-evident. 
I have a device that emits an xml message stream in response to receiving an HTTP get.  The return xml schema varies by message content--there are approximately 4 different schemas that I can see, and messages are broadcast approximately every 3 seconds after the initial wget triggers the server response.  Does openLuup have a way to set-up a callback similar to the UDP style callback?  And secondly, I read the source code notes in xml.lua about the basic internal xml parser, but wasn't sure what the boundaries were for its use, or the syntax for calling it. 

The functionality I need is fairly straight forward.  I need to start the message service via a wget which would activate the openLuup listener, parse the xml response (and reissue the wget if the connection drops), scan, and then act on the data. I know a listener can be done programmatically as many of the plugins do exactly that, but would much prefer a simpler call directly to openLuup to handle the listening. ;D

Any insights are greatly appreciated. 

Offline akbooer

  • Moderator
  • Master Member
  • *****
  • Posts: 6387
  • Karma: +292/-70
  • "Less is more"
Re: HTTP callback
« Reply #1 on: May 03, 2018, 01:38:38 am »
Apologies in advance if this topic has been discussed somewhere, but I couldn't find anything definitive in the forum archives.  Or for that matter if the answer is self-evident. 

No need to apologise.  This is definitely a new topic!

Quote
I have a device that emits an xml message stream in response to receiving an HTTP get.  The return xml schema varies by message content--there are approximately 4 different schemas that I can see, and messages are broadcast approximately every 3 seconds after the initial wget triggers the server response. 

That's pretty unusual behaviour, I'd say?  I've never seen a device do anything quite like that before.

Quote
Does openLuup have a way to set-up a callback similar to the UDP style callback? 

Yes.  There's at least three possible alternatives to explore:
  • create a separate HTTP server on another port (this should be easier than it sounds... in the end, just a one-liner, although the existing server code may need just a few tweaks.)
  • open a vanilla TCP server (also a one-liner) and handle the HTTP protocol and XML schemas directly
  • use the standard luup.io module functionality and write a plugin with <incoming> tags
Which way to go would very much depend on the detailed behaviour of your device: what it puts in the headers; how it handles the connection; ...

Is there a detailed description of that?

Quote
And secondly, I read the source code notes in xml.lua about the basic internal xml parser, but wasn't sure what the boundaries were for its use, or the syntax for calling it. 

In one of those curious coincidences, in the latest release I have just removed the separate XML file, for a number of reasons.  In particular, the caveat at the start of that module remains pertinent:

Code: [Select]
-- general xml reader: this is just good enough to read device and implementation .xml files
-- doesn't cope with XML attributes or empty elements: <tag />
--
-- TODO: proper XML parser rather than nasty hack?

You can still access the XML module, it's now a sub-module of the loader, since that was the only user of its functionality, and I've currently hidden the decode() function, but it's trivial to reinstate that.

Code: [Select]
local xml = require "openLuup.loader" .xml

What does the XML sent by the device actually look like?  (I hate XML... JSON is so much easier)

Quote
The functionality I need is fairly straight forward.  I need to start the message service via a wget which would activate the openLuup listener, parse the xml response (and reissue the wget if the connection drops), scan, and then act on the data.

As above, it really depends on the details of the device and it's handling of the connection.  I'm not yet convinced that what you ask is actually "straight forward."

Quote
I know a listener can be done programmatically as many of the plugins do exactly that, but would much prefer a simpler call directly to openLuup to handle the listening. ;D

Wise choice.  The Vera/MiOS model for this type of device I/O is deeply flawed.  Not least, the ability (or lack thereof) to reopen a connection when it drops.  So that's my option #3 above, out of the running.

Quote
Any insights are greatly appreciated.

Well, there they are, FWIW.  Your thoughts, now?
3x Vera Lite-UI5/Edge-UI7, 25x Fibaro, 23x TKB, 9x MiniMote, 2x NorthQ Power, 2x Netatmo, 1x Foscam FI9831P, 9x Philips Hue,
Razberry, MySensors Arduino, HomeWave, AltUI, AltHue, DataYours, Grafana, openLuup, ZWay, ZeroBrane Studio.

Offline Buxton

  • Full Member
  • ***
  • Posts: 209
  • Karma: +12/-0
Re: HTTP callback
« Reply #2 on: May 03, 2018, 07:30:28 pm »
OK, the device is an ACTi NVR.  The call to start the message broadcast is http://[login]:[PW]@[IP]:[Port]/Media/message.  There are no other parameters.  Typical big gulp of data below....

Typical XML is
Code: [Select]
<?xml version="1.0" encoding="UTF-8"?><Message type="Event" Time="1524876477.100" Bias="480" DaylightBias="60"><Event type="Status" src="6" id="DI0" number="0" active="off"/><Event type="Status" src="6" id="DI1" number="1" active="off"/><Event type="Status" src="6" id="DI2" number="2" active="off"/><Event type="Status" src="6" id="DI3" number="3" active="off"/><Event type="Status" src="6" id="DI4" number="4" active="off"/><Event type="Status" src="6" id="DI5" number="5" active="off"/><Event type="Status" src="6" id="DI6" number="6" active="off"/><Event type="Status" src="6" id="DI7" number="7" active="off"/><Event type="Status" src="6" id="DI8" number="8" active="off"/><Event type="Status" src="6" id="DI9" number="9" active="off"/><Event type="Status" src="6" id="DI10" number="10" active="off"/><Event type="Status" src="6" id="DI11" number="11" active="off"/><Event type="Status" src="6" id="DI12" number="12" active="off"/><Event type="Status" src="6" id="DI13" number="13" active="off"/><Event type="Status" src="6" id="DI14" number="14" active="off"/><Event type="Status" src="6" id="DI15" number="15" active="off"/><Event type="Status" src="6" id="DI16" number="16" active="off"/><Event type="Status" src="6" id="Motion0" number="0" active="off"/><Event type="Status" src="6" id="Motion1" number="1" active="off"/><Event type="Status" src="6" id="Motion2" number="2" active="off"/><Event type="Status" src="6" id="Motion3" number="3" active="off"/><Event type="Status" src="6" id="Motion4" number="4" active="off"/><Event type="Status" src="6" id="Motion5" number="5" active="off"/><Event type="Status" src="6" id="Motion6" number="6" active="off"/><Event type="Status" src="6" id="Motion7" number="7" active="off"/><Event type="Status" src="6" id="Motion8" number="8" active="off"/><Event type="Status" src="6" id="Motion9" number="9" active="off"/><Event type="Status" src="6" id="Motion10" number="10" active="off"/><Event type="Status" src="6" id="Motion11" number="11" active="off"/><Event type="Status" src="6" id="Motion12" number="12" active="off"/><Event type="Status" src="6" id="Motion13" number="13" active="off"/><Event type="Status" src="6" id="Motion14" number="14" active="off"/><Event type="Status" src="6" id="Motion15" number="15" active="off"/><Event type="Status" id="VideoRecovery1" number="1" active="on" src="6"/></Message>
Followed by
Code: [Select]
--2A575808
Content-Type: text/xml
Content-Length: 121

<Message type="Event" Time="1524876473.850" Bias="480" DaylightBias="-60"><Event type="Status" id="LiveCheck"/></Message>

and
Code: [Select]
--2A575808
Content-Type: text/xml
Content-Length: 165

<Message Time="1524876483.7" Bias="480" DaylightBias="-60" type="Event">
<Event type="Status" id="RecordStatus" src="4" value="1,0,0" active="1,0,0"/>
</Message>

The "--2A575808" is a reference to the NVR serial no.

Below describes the message format:
 
Code: [Select]
Multiple parts format :
--(Boundary)CRLF
Content-Type : media type of media data CRLF
Content-Length : length of media data CRLF
CRLF
<<>>CRLF
 
Sample :
GET /media/message HTTP/1.1
Accept: text/html, application/xhtml+xml, */*
Accept-Language: zh-TW
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko
Accept-Encoding: gzip, deflate
Host: 172.16.18.13
Authorization: Basic YWRtaW46MTIzNDU2
Connection: Keep-Alive
DNT: 1
 
HTTP/1.1 200 OK
Cache-Control: no-cache
Connection: Keep-Alive
Pragma: no-cache
Content-Type: multipart/x-mixed-replace; boundary=506CB008
x-sessioncookie: 28385470
Access-Control-Allow-Origin: *

Given that there is little if any client/server interaction here, it seems like the best way to go is via luup.io, though without knowing the internal strengths of each method, I'm fairly unqualified to make that call.  :) As a basic approach, I would choose the method that requires the least amount of overhead processing while maintaining a reliable connection as there is no need for any program modification of the call to the NVR.  A parameter to set a "timeout" before a retry would be nice, though of course that can be handled in the listener call.
Thx for looking at this.