The Vera Community forums have moved!

Advanced => Programming => Plugins & Plugin Development => Topic started by: Roy S on May 12, 2011, 05:56:05 am

Title: SQ Blaster Plugin, http requests too fast ?
Post by: Roy S on May 12, 2011, 05:56:05 am
I'm using guessed's SQBlaster plugin and I built my own plugin on top to control my TV though an SQ Blaster puck.
The problem is that when I send two consecutive commands programmatically ( ie 2 calls to luup.call_action("urn:micasaverde-com:serviceId:IrTransmitter1","SendProntoCode",... ) , the second is not executed: SQ Blaster returns the following error code (request out of sequence)


<execCmndResponse>
<status messageNum="111" reasonCode="29" />
</execCmndResponse>

I suspect that the http requests are being called in 2 different threads  ???.

Is there any way to serialize these calls or have the plugin function sleep for a few milliseconds after each call ?

Any ideas are appreciated.

Thanks.
Title: Re: SQ Blaster Plugin, http requests too fast ?
Post by: guessed on May 12, 2011, 07:45:02 am
Ping @SquareconnectMat and find the puck behavior in this situation.  The plugin makes a HTTP call when sending stuff, and blocks for the response, so there shouldn't be a timing problem (or we'd have the same issue in other plugins also)

He'll likely want a copy of the specific requests being made, and the sequencing from the MiOS logs in order to see how he handles them at the Puck end.

I'm assuming that both of your calls are being made, sequentially, from within a single device (2x calls from a single device)
Title: Re: SQ Blaster Plugin, http requests too fast ?
Post by: Ap15e on May 12, 2011, 07:46:12 am
You could try specifying your SQBlaster commands on the 'Advanced' tab of a scene and insert delays - or try using luup.sleep() ...
Title: Re: SQ Blaster Plugin, http requests too fast ?
Post by: guessed on May 12, 2011, 07:54:57 am
Delays should not be necessary, and should be a hack of last recourse.  If we work on fixing the core issue, then everyone will benefit.

It's likely that something else is amiss if these errors are being reported and Mat is keen to get that stuff sorted out in his overall solution stack.

PS: Mat will want to know what Firmware version your running also, in case it's been addressed/changed in more recent firmwaes.  This information is available in the SQBlaster plugin dialog.
Title: Re: SQ Blaster Plugin, http requests too fast ?
Post by: guessed on May 12, 2011, 08:09:04 am
@Roy S, just noticed that you're not at a level where you can PM yet, so sent Mat a PM on you're behalf.  Hopefully he'll be able to respond in thread with some technical reasoning for the behavior you're seeing.  You can post FWare details here in the meanwhile.
Title: Re: SQ Blaster Plugin, http requests too fast ?
Post by: Roy S on May 12, 2011, 10:40:37 am
Thank you guessed,

My SQBlaster firmware version is A069 (FirmwareDate Dec 26 2010).

I had opened a supported ticket with Squareconnect and they sent me their system integrators guide (that's how I knew that messagenum=111 is a sequence out of request error). I thought this could be a problem faced by you guys.

Please find the part of the log file showing the error below


08      05/12/11 17:35:53.017   JobHandler_LuaUPnP::HandleActionRequest device: 28 service: urn:micasaverde-com:serviceId:NumericEntry1 action: 2 <0x24808>
08      05/12/11 17:35:53.018   JobHandler_LuaUPnP::HandleActionRequest argument serviceId=urn:micasaverde-com:serviceId:NumericEntry1 <0x24808>
08      05/12/11 17:35:53.018   JobHandler_LuaUPnP::HandleActionRequest argument action=2 <0x24808>
08      05/12/11 17:35:53.019   JobHandler_LuaUPnP::HandleActionRequest argument DeviceNum=28 <0x24808>
08      05/12/11 17:35:53.019   JobHandler_LuaUPnP::HandleActionRequest argument _=1305210952850 <0x24808>
50      05/12/11 17:35:53.023   luup_log:28: I_domotixtv: executing: 2 <0x400>
08      05/12/11 17:35:53.024   JobHandler_LuaUPnP::HandleActionRequest device: 26 service: urn:micasaverde-com:serviceId:IrTransmitter1 action: SendProntoCode <0x400>
08      05/12/11 17:35:53.024   JobHandler_LuaUPnP::HandleActionRequest argument ProntoCode=P277e 6695 95eb e3d3 ad38 f532 f1f3 9c97 679d 61a8 6311 41c5 1590 98eb 4368 f934 24d0 3633 16dd a03c 2b0a 73d6 f87b 980c 679d 61a8 6311 41c5 1590 98eb 4368 f934 24d0 3633 16dd a03c 2b0a 73d6 f87b 980c 1af5 bf7b 9b34 0332 9 <0x400> c96e f45f 129c 2830 63a6 d00d 89c0 fb32 5f9e c0c5 effa 4ffc df55 31fa d3dd 5a6b 89ec 76b8 6dbf 1305 277d f42e 7a95 91bc 15fb ee91
08      05/12/11 17:35:53.035   JobHandler_LuaUPnP::HandleActionRequest device: 28 service: urn:micasaverde-com:serviceId:NumericEntry1 action: 3 <0x24808>
08      05/12/11 17:35:53.035   JobHandler_LuaUPnP::HandleActionRequest argument serviceId=urn:micasaverde-com:serviceId:NumericEntry1 <0x24808>
08      05/12/11 17:35:53.038   JobHandler_LuaUPnP::HandleActionRequest argument action=3 <0x24808>
08      05/12/11 17:35:53.038   JobHandler_LuaUPnP::HandleActionRequest argument DeviceNum=28 <0x24808>
08      05/12/11 17:35:53.039   JobHandler_LuaUPnP::HandleActionRequest argument _=1305210952851 <0x24808>
50      05/12/11 17:35:53.085   luup_log:26: Web request returned status=200 response=<?xml version="1.0" encoding="utf-8"?>
<execCmndResponse>
<status messageNum="0" reasonCode="0" />
</execCmndResponse> in 0ms <0x400>

06      05/12/11 17:35:53.087   UserData::m_luc_HAG_Variables_set variable: DataVersionStatus now: 192995192 subs: 0 <0x400>
04      05/12/11 17:35:53.089   <Job ID="576" Name="" Created="11-05-12 17:35:53" Started="11-05-12 17:35:53" Completed="11-05-12 17:35:53" Duration="0.67345000" Runtime="0.66056000" Status="Successful" LastNote=""/> <0x400>
50      05/12/11 17:35:53.091   luup_log:28: I_domotixtv: executing: 3 <0x400>
08      05/12/11 17:35:53.092   JobHandler_LuaUPnP::HandleActionRequest device: 26 service: urn:micasaverde-com:serviceId:IrTransmitter1 action: SendProntoCode <0x400>
08      05/12/11 17:35:53.093   JobHandler_LuaUPnP::HandleActionRequest argument ProntoCode=P277e 6695 95eb e3d3 ad38 f532 f1f3 9c97 679d 61a8 6311 41c5 1590 98eb 4368 f934 24d0 3633 16dd a03c 2b0a 73d6 f87b 980c 679d 61a8 6311 41c5 1590 98eb 4368 f934 129c 2830 63a6 d00d 89c0 fb32 5f9e c0c5 679d 61a8 6311 41c5 1 <0x400> 4368 f934 24d0 3633 16dd a03c 2b0a 73d6 f87b 980c ffb9 404f c3bd 30b6 283b ba0c f73d e93e 6dbf 1305 277d f42e 7a95 91bc 15fb ee91
50      05/12/11 17:35:53.145   luup_log:26: Web request returned status=200 response=<?xml version="1.0" encoding="utf-8"?>
<execCmndResponse>
<status messageNum="111" reasonCode="29" />

Title: Re: SQ Blaster Plugin, http requests too fast ?
Post by: SquareConnectMat on May 13, 2011, 08:03:31 pm
Hi Guys,

Sorry for the slightly tardy response here on the forum. For some reason I am not getting notified by email when PM's are sent or topics I am watching get updated!

What is happening here is that when you issue an HTTP command to the blaster, it starts blasting and returns immediately. The blaster continues to blast. It will not accept another start command while it is blasting. Hence the 111 error. It may also be a problem in press and hold mode, if the sequencing is out (which can happen when there are a lot of errors and retries going on under the hood.

Ther is a long and complicated reason for this behaviour - mostly to do with ensuring that the blaster behaves well when there are errors on the network and commands need to be near 'real time'.

If you are using the press and hold form of the commands, the 'repeat' command IS accepted if it follows the rules as described in the document.

Basically, if you get this response... back off for 250ms and then try again.

Mat


Title: Re: SQ Blaster Plugin, http requests too fast ?
Post by: guessed on May 13, 2011, 10:38:03 pm
@mat,
Thanks for the official response.  Any API guidance on hw many retries should be executed under this scheme - say, based upon the code lengths and likely IR transmission times you've [historically] seen.
Title: Re: SQ Blaster Plugin, http requests too fast ?
Post by: Roy S on May 14, 2011, 04:17:13 am
Thanks Guys,

Based on Mat's reply, I was able to solve the problem by having the function sleep for 250ms after each http command.
luup.sleep(250) was giving errors (something to do with Vera firmware 1.1.1047 maybe ?) so I added

socket.select(nil, nil, 0.25)

at the end of the SendProntoCode function code.
I know this is not the perfect solution as there's no error management but it's working  in my case.
Title: Re: SQ Blaster Plugin, http requests too fast ?
Post by: Ap15e on May 14, 2011, 05:41:15 am
luup.sleep() was introduced around 1.1.1155, see http://forum.micasaverde.com/index.php?topic=5244.0.
Title: Re: SQ Blaster Plugin, http requests too fast ?
Post by: SquareConnectMat on May 14, 2011, 11:54:33 pm
@guessed

Well.... First, the next version of the SQ API will provide tools to be notified when a blast is finished We are in the process of designing this. Your input is always welcome on that. We will also be enabling in-blaster execution of a 'macro' or stream of commands.. so the blaster can manage when a blaster is ready for the next command for you. Very useful for the channel entry and other simple macros. But as always, this is some time away... towards the end of the year, most likely.

Most IR commands last less than 250ms - that is the time at which it starts to become less than 'instantaneous'. There are a lot that are less than 10ms - especially volume up/down etc... but they often require a multiple burst to be recognized by the equipment.

The real issue is when will the target equipment accept the next command. Anyone with one of the older Blu-ray player's know that issue...

Unfortunately the only real way of determining this is to test the exact timing/repeats and sequence for your specific set of equipment. The macro function in SQ Remote can be useful to find this out and experiment. Then you could put these values into the Lua script...


Mat

Title: Re: SQ Blaster Plugin, http requests too fast ?
Post by: SquareConnectMat on May 15, 2011, 12:05:32 am
@mat,
Thanks for the official response.  Any API guidance on hw many retries should be executed under this scheme - say, based upon the code lengths and likely IR transmission times you've [historically] seen.

In terms of number of retries... I would say if after 4 retries without success I would say something was wrong... most likely another client was trying to blast at the same time.

If two or more Users are trying to battle the same blaster, while one is blasting, the other just gets rejected until the blaster is clear. pure chance who gets it next.

The only state we keep track of is which toggle code was sent for a specific device/command, so we can send the next toggle (mostly correctly) and when in press and hold, which sequence number we are currently holding for. On time outs (after 300ms) we will go into 'accept next command' mode. (if that makes any sense).

I am not sure what the correct behavior for an automatic script in a controller like Vera should be.. I am tempted to say, if the command MUST be executed, back off a second and try again. But that may lead to surprising things happening in your environment.

Title: Re: SQ Blaster Plugin, http requests too fast ?
Post by: Roy S on May 15, 2011, 06:22:50 am
@mat,

Since a new version of the APIs will be released, can't you guys implement some kind of queuing mecanism within the SQ Blaster so that commands are stored and blasted serially?

(I have no idea what kind of memory a puck has in it and if this is feasible... Just adding my 2cents)

Roy.
Title: Re: SQ Blaster Plugin, http requests too fast ?
Post by: guessed on May 15, 2011, 11:41:45 am
@Roy S,
I'm going to mod the SQBlaster plugin itself to do the retries, based upon Mat's comments/data above. 

In Lua, it's trivial to work out if a method is available or not, so I'll add logic to work use luup.sleep, or fallback to your method (for earlier MiOS revs)

I'll need you to test that properly since mine are all newer Betas.

Anyhow, if you're calling my code, you'll pickup the mods without anymore code (etc) once I've readied them.



In past discussions with Mat, there were reasons for not "queuing" commands in the Puck.  Some of these related to what can happen if you have a rogue client sending too many "critical" commands (like  VolumeUp) leading to all sorts of bad situations.
Title: Re: SQ Blaster Plugin, http requests too fast ?
Post by: Roy S on May 15, 2011, 12:54:13 pm
Got it @guessed,

I'll be waiting for your new plugin version. Meanwhile, can someone change my security privileges so I can PM people ? (Anything required from my side ? )

Cheers,
Roy.
Title: Re: SQ Blaster Plugin, http requests too fast ?
Post by: guessed on May 15, 2011, 02:41:59 pm
@Roy S,
You get that automatically once you reach 25 posts, or "Sr Newbie" status.  It was a restriction that I put in to stop the Spammers mis-using the Forums (as they're prone to do for a variety of reasons).

Unfortunately, the spammers create a high percentage of the accounts on this system, and have been using them for link promotion (amongst other things) so some restrictions were put in place on Newbie accounts to stem the flow of SPAM.
Title: Re: SQ Blaster Plugin, http requests too fast ?
Post by: guessed on May 15, 2011, 08:23:07 pm
@Roy S,
I've modified the code to do 4x attempts at sending, using 250ms gaps (per Mat's suggestions).  You can pull a modified version of the Plugin's implementation file from this location in source control:

    http://code.mios.com/trac/mios_sqblaster/export/34/trunk/I_SQBlaster1.xml

Can you try that out, and let me know if it addresses the problem you were seeing? 


If so, I'll tag the latest round of files, and publish them formally.

It should work on older Vera versions as it's using luup.sleep (if available) and falls back to socket.sleep if luup.sleep isn't available in MiOS (to avoid forcing MiOS upgrades to the Beta releases)

The changes are simple, but it took a while to test all the combinations.  Anyone wanting to see what was changed can go here for full details:

    http://code.mios.com/trac/mios_sqblaster/changeset/33/trunk/I_SQBlaster1.xml
Title: Re: SQ Blaster Plugin, http requests too fast ?
Post by: Roy S on May 16, 2011, 04:08:00 am
Thanks @guessed,

Plugin tested successfully. The plugin retries to send to code until it succeeds.  It is usually successful on the 2nd or 3rd
attempt.

But I noticed another thing while testing: In my plugin, I send IR commands as jobs (as below).
Sometimes, when sending multiple commands, they are not executed in the right order.
I had tried to use <run></run> instead of jobs, but vera would crash and need a restart.


<serviceId>urn:micasaverde-com:serviceId:DiscretePower1</serviceId>
<name>Off</name>
<job>
   sendCommand("PowerOff")  -- Local function
</job>


Any ideas ?
Title: Re: SQ Blaster Plugin, http requests too fast ?
Post by: guessed on May 16, 2011, 11:27:52 am
I'd have to see the full code you're using, so that the impl of the sendCommand was exposed also, including a code-block showing multiple calls in a single job/run block. 

Generally speaking, the order of execution of different job blocks requests is indeterminate.  If you need a single job block to run a list of items serially, I'm assuming you're either using multiple sendCommand calls, or you're invoking a scene with the steps all listed out in it.

Can you provide a little more detail?
Title: Re: SQ Blaster Plugin, http requests too fast ?
Post by: Roy S on May 16, 2011, 04:44:18 pm

I've attached the implementation file of my TV plugin.

In short, I store the IR pronto codes in a text file, that is loaded into an array at startup.
Since the file name is a variable, I can have multiple IR Devices using the same plugin.

A custom interface calls the different actions, either manually or programmatically.

Now the problem is that when actions are being called, the sendCommand function is executed through a job. I can understand that jobs may not be executed serially when they target multiple devices (ie turning off all lights), but the jobs here are all related to the same device, so logically the order of execution should be kept.
Title: Re: SQ Blaster Plugin, http requests too fast ?
Post by: guessed on May 17, 2011, 12:07:50 am
Thanks for publishing. 

I've only ever done the IR stuff using <run> blocks with Lua or code-generate <ir> blocks with embedded Pronto codes.  These seem to run fine, in single-tests as well as in composite (scenes) scenarios.... at least I haven't had any trouble with them (yet)

Quote
but the jobs here are all related to the same device, so logically the order of execution should be kept.

The job stuff hasn't got a clear definition for what happens within a single device, or across devices, when it's called rapidly.  Personally, I'd like to be able to have multiple concurrent jobs in a single Device (for concurrent execution) so I can see both "needs" here.

<run> has generally been a safe bet, given it's guaranteed ordering, but with the downside of blocking the Control Point during execution.

You mentioned previously that your <run> blocks were crashing.  Can you explain what you're seeing when you switch from <job> to <run>?


Side-notes...
Looking at your code, you might be interested in:
    http://code.mios.com/trac/mios_sqblaster/browser/trunk/I_SQBlasterController1.xml

It's a code-generator that converts source IR files (this case in SQBlaster format) on the fly into MiOS format D_*.xml/I_*.xml files, and then instantiates them as child devices.  It's not fully functional just yet, but might give you some code snippets to do the file loading/parsing (using string.match() with multiple captures, for example) and some stuff about handling the config files in a "standard" location reachable from the UI so folks can upload new ones easily.

Unfortunately the D_ and I_ files have a "fixed", compile time, format implementing one or more Services.  The UI tends to show "all" actions exposed by any of the Service(s) you implement... which can be annoying when you're implementing everything from TV's to Amp's, through to Blu-Ray players... they all implement loads of Services and you end up implementing them all "just in case".

The code generator above was done to try and avoid that headache - albeit at the loss of runtime flexability (and code size 8) )
Title: Re: SQ Blaster Plugin, http requests too fast ?
Post by: Ap15e on May 17, 2011, 04:06:43 am
@Roy S

Thanks for sharing your code. Would you mind posting your complete plugin?

You might find LIRC2Luup4SBS (a lirc file to Luup device compiler) interesting (even if you don't own the HW required):
http://forum.micasaverde.com/index.php?topic=5709.0
Title: Re: SQ Blaster Plugin, http requests too fast ?
Post by: Roy S on May 17, 2011, 04:47:28 am
@guessed,

When I replace <job> with <run> blocks, and I execute a few actions, Lua terminates with the following error:

LuaUPnP Terminated with Exit Code: 245

From then on, the only option is to do a full reboot. That's why I switched to job blocks.
I will try to think of a solution to eliminate the order problem (maybe add more delays  ??? ...).

Meanwhile, thanks for the tip about SQBlasterController. I might use it as I want to develop a user friendly web interface to generates the IR Code files used by my plugin (Will use the SQ Blaster IR learning API ).

@Ap15e,

I've attached the plugin files. I'm not sure they can be of any use to anyone at the moment, but they can definitely be tweaked. I've also attached an IR Pronto Codes file (needs to be unzipped) as an example (Samsung Plasma TV model no PS50C430A1XTW).