Author Topic: HEX answer over TCP and analysis  (Read 1195 times)

Offline halo

  • Jr. Member
  • **
  • Posts: 56
  • Karma: +0/-0
HEX answer over TCP and analysis
« on: December 29, 2017, 05:26:40 am »
Dear experts,

I am struggling few weeks already to properly decode and analyse TCP response from my 3rd party box.

I can sent proper request and I  am getting proper answer but I do not know how to analyse it and use it later on.

For example:

I am getting the response as in attached file and I do not know how to analyse it and decode.

Currently I am having the code as below, but "Variable3" it is showing NULL value.

How to put into the Variable3 the string from TCP answer "FEFE09000000007DB4FE0D".
 

local hex = {
  "FE", "FE", "09", "D7", "EB", "FE", "0D"
}
local binary = "";
for i, v in ipairs(hex) do
  binary = binary .. string.char(tonumber(v, 16))
end
local socket = require("socket")
tcp = assert(socket.connect("192.168.1.248", 7094))
tcp:send(binary .. "\r\n")
tcp:settimeout(15)
reply = tcp:receive()
luup.sleep(50)
tcp:close()
luup.variable_set("urn:upnp-org:serviceId:VContainer1", "Variable3",reply,231)






Offline akbooer

  • Master Member
  • *******
  • Posts: 6120
  • Karma: +274/-69
  • "Less is more"
Re: HEX answer over TCP and analysis
« Reply #1 on: December 29, 2017, 06:04:52 am »
What status response do you get from the tcp receive() function?
Is there anything diagnostic in the log?
3x Vera Lite-UI5/Edge-UI7, 25x Fibaro, 23x TKB, 9x MiniMote, 2x NorthQ Power, 2x Netatmo, 1x Foscam FI9831P, 9x Philips Hue,
Razberry, MySensors Arduino, HomeWave, AltUI, AltHue, DataYours, Grafana, openLuup, ZWay, ZeroBrane Studio.

Offline halo

  • Jr. Member
  • **
  • Posts: 56
  • Karma: +0/-0
Re: HEX answer over TCP and analysis
« Reply #2 on: December 29, 2017, 11:30:28 am »
After adding the luup.log (reply) into my code, I am getting info as attached.

50   12/29/17 17:26:57.835   luup_log:0: (null) <0x72b6a520>
01   12/29/17 17:26:57.835   luup_variable_set interface 0x133db38 no Variable3/(null) <0x72b6a520>


Offline akbooer

  • Master Member
  • *******
  • Posts: 6120
  • Karma: +274/-69
  • "Less is more"
Re: HEX answer over TCP and analysis
« Reply #3 on: December 29, 2017, 02:48:26 pm »
Looks like the receive has failed or timed out.  It has a second status return parameter.  That would give more info.
3x Vera Lite-UI5/Edge-UI7, 25x Fibaro, 23x TKB, 9x MiniMote, 2x NorthQ Power, 2x Netatmo, 1x Foscam FI9831P, 9x Philips Hue,
Razberry, MySensors Arduino, HomeWave, AltUI, AltHue, DataYours, Grafana, openLuup, ZWay, ZeroBrane Studio.

Offline halo

  • Jr. Member
  • **
  • Posts: 56
  • Karma: +0/-0
Re: HEX answer over TCP and analysis
« Reply #4 on: December 29, 2017, 03:04:06 pm »
Any suggestions? Different logs query or put to settimout different value. Is there any different logs to search?

Offline halo

  • Jr. Member
  • **
  • Posts: 56
  • Karma: +0/-0
Re: HEX answer over TCP and analysis
« Reply #5 on: December 29, 2017, 03:05:34 pm »
Is the script itself correct? Should I modify somehow the "reply" variable before receive function?

Offline akbooer

  • Master Member
  • *******
  • Posts: 6120
  • Karma: +274/-69
  • "Less is more"
Re: HEX answer over TCP and analysis
« Reply #6 on: December 29, 2017, 05:53:17 pm »
The script itself should be functional.  I would recommend removing the luup.sleep() function which does nothing except block for a period of time.  The tcp and reply variables should be declared local.

I ran this code with dummy socket and luup modules defined like this:

Code: [Select]
local socket = {
  connect = function () return
    {
      settimeout = function () end,
      receive = function () return "123456" end,
      send = function () end,
      close = function () end,
    }
  end
}

local luup = {
  sleep = function () end,
  variable_set = print,
}

and it gave the expected output:

Code: [Select]
urn:upnp-org:serviceId:VContainer1 Variable3 123456 231

Once again, I would recommend inspecting the second status return parameter from the receive statement to see what error it might generate.

Quote from: http://w3.impa.br/~diego/software/luasocket/tcp.html#receive
If successful, the method returns the received pattern. In case of error, the method returns nil followed by an error message which can be the string 'closed' in case the connection was closed before the transmission was completed or the string 'timeout' in case there was a timeout during the operation.
3x Vera Lite-UI5/Edge-UI7, 25x Fibaro, 23x TKB, 9x MiniMote, 2x NorthQ Power, 2x Netatmo, 1x Foscam FI9831P, 9x Philips Hue,
Razberry, MySensors Arduino, HomeWave, AltUI, AltHue, DataYours, Grafana, openLuup, ZWay, ZeroBrane Studio.

Offline halo

  • Jr. Member
  • **
  • Posts: 56
  • Karma: +0/-0
Re: HEX answer over TCP and analysis
« Reply #7 on: December 30, 2017, 06:06:23 am »
Hi,

It took me a while but finally I have understood what you are trying to say to me :)

Following your sugeestions script updated as below:

local hex = {
  "FE", "FE", "09", "D7", "EB", "FE", "0D"
}
local binary = "";
for i, v in ipairs(hex) do
  binary = binary .. string.char(tonumber(v, 16))
end
local socket = require("socket")
local tcp = assert(socket.tcp())
tcp:settimeout(1)
tcp:connect("192.168.1.248", 7094)
tcp:send(binary .. "\r\n")
local reply, error = tcp:receive()
luup.log(reply)
luup.log(error)
tcp:close()
luup.variable_set("urn:upnp-org:serviceId:VContainer1", "Variable3",reply,231)

Lua logs shows "timeout".

Where to look for reasons of this timeout?




Offline akbooer

  • Master Member
  • *******
  • Posts: 6120
  • Karma: +274/-69
  • "Less is more"
Re: HEX answer over TCP and analysis
« Reply #8 on: December 30, 2017, 08:27:52 am »
For a start, I would try logging the status return from the send() function also.

Are you sure that you are, in fact, sending the right message.  Is there a way to tell at the remote device?
3x Vera Lite-UI5/Edge-UI7, 25x Fibaro, 23x TKB, 9x MiniMote, 2x NorthQ Power, 2x Netatmo, 1x Foscam FI9831P, 9x Philips Hue,
Razberry, MySensors Arduino, HomeWave, AltUI, AltHue, DataYours, Grafana, openLuup, ZWay, ZeroBrane Studio.

Offline halo

  • Jr. Member
  • **
  • Posts: 56
  • Karma: +0/-0
Re: HEX answer over TCP and analysis
« Reply #9 on: December 30, 2017, 09:28:05 am »
Hi,

Peer side received correct messages. I can control peer device sent TCP packets from Vera. As long as I do not want to analyze the response everything works as expected.

The problem starting if I want to analyze the response.

I did the port mirroring on the switch and capture the packages on the switch.

Wireshark attached.

Vera = 192.168.1.162
Peer device = 192.168.1.248

Adding the "binary" logging shows

50   12/30/17 15:15:05.137   luup_log:0: (null) <0x72662520>
50   12/30/17 15:15:05.138   luup_log:0: timeout <0x72662520>
50   12/30/17 15:15:05.138   luup_log:0: ??   ??? <0x72662520>

In attachment, print screen from wireshark.

Offline akbooer

  • Master Member
  • *******
  • Posts: 6120
  • Karma: +274/-69
  • "Less is more"
Re: HEX answer over TCP and analysis
« Reply #10 on: December 30, 2017, 11:33:51 am »
Looking at my own code in openLuup, and checking the documentation here
 
http://w3.impa.br/~diego/software/luasocket/tcp.html#setoption

...I see that I use

Code: [Select]
tcp:setoption ("tcp-nodelay", true)

which allows alternating reads and writes on the same socket.

Worth a try.
3x Vera Lite-UI5/Edge-UI7, 25x Fibaro, 23x TKB, 9x MiniMote, 2x NorthQ Power, 2x Netatmo, 1x Foscam FI9831P, 9x Philips Hue,
Razberry, MySensors Arduino, HomeWave, AltUI, AltHue, DataYours, Grafana, openLuup, ZWay, ZeroBrane Studio.

Offline halo

  • Jr. Member
  • **
  • Posts: 56
  • Karma: +0/-0
Re: HEX answer over TCP and analysis
« Reply #11 on: December 30, 2017, 11:58:52 am »
Thanks for suggestion, but no change.

Still the status is timeout.

Is it related to timers on TCP layer or it is just internal error code?



Offline akbooer

  • Master Member
  • *******
  • Posts: 6120
  • Karma: +274/-69
  • "Less is more"
Re: HEX answer over TCP and analysis
« Reply #12 on: December 30, 2017, 01:29:32 pm »
It's a genuine socket timeout.

One last-ditch effort (the LuaSocket library does have some imperfections dealing with timeouts) ... you might try with the timeout set to nil.

This does mean that if there is no response the plugin will hang, but just give it a go.  I've had to do this on an HTTP server in some circumstances.  Bizarre, I know, but do try it.

Otherwise, I am all out of suggestions.
3x Vera Lite-UI5/Edge-UI7, 25x Fibaro, 23x TKB, 9x MiniMote, 2x NorthQ Power, 2x Netatmo, 1x Foscam FI9831P, 9x Philips Hue,
Razberry, MySensors Arduino, HomeWave, AltUI, AltHue, DataYours, Grafana, openLuup, ZWay, ZeroBrane Studio.

Offline halo

  • Jr. Member
  • **
  • Posts: 56
  • Karma: +0/-0
Re: HEX answer over TCP and analysis
« Reply #13 on: December 30, 2017, 02:31:53 pm »
Akbooer

I think it has something to do with the response and the HEX code format.
Peer side has ability to estiblish only one connection at the time.

So I have simulated a connection using Hercules tool from my laptop and keep the connection.

In this same time I have executed the LUA script from Vera and now it works. The status was reported as it should be "Busy".

What is the difference in answer from peer side during interpretation in Vera??

10:42:75:73:79:21:0d:0a:2c:a5:a5:a5:a5:a5:a5:a5   - Busy

fe:fe:09:00:00:00:00:7d:b4:fe:0d   - Status of peer side

From the logs:

50   12/30/17 20:10:58.812   luup_log:0: Busy! <0x72956520>
50   12/30/17 20:10:58.812   luup_log:0: (null) <0x72956520>
50   12/30/17 20:10:58.812   luup_log:0: ??   ???
 <0x72956520>

Should I do some number conversion of the reply variable?? If yes, please help to suggest the code.



Offline halo

  • Jr. Member
  • **
  • Posts: 56
  • Karma: +0/-0
Re: HEX answer over TCP and analysis
« Reply #14 on: December 30, 2017, 04:13:27 pm »
It works - partially.

For some reasons only specifying amount of bytes in recieve function allows to receive the correct response.

The working script is:

local hex = {
  "FE", "FE", "09", "D7", "EB", "FE", "0D"
}
local binary = "";
for i, v in ipairs(hex) do
  binary = binary .. string.char(tonumber(v, 16))
end
local socket = require("socket")
local tcp = assert(socket.tcp())
tcp:connect("192.168.1.248", 7094)
tcp:settimeout(1)
tcp:send(binary .. "\r\n")
local reply, error = tcp:receive(11)
luup.log(reply)
luup.log(reply2)
luup.log(error)
luup.log(binary)
local a, b, c = tcp:getstats()
luup.log (a)
luup.log(b)
luup.log (c)
tcp:close()
luup.variable_set("urn:upnp-org:serviceId:VContainer1", "Variable3",reply2,231)

In Vera logs recevied answer is presented as below:

50   12/30/17 21:52:46.807   luup_log:0: ??    <0x71f4a520>
50   12/30/17 21:52:46.808   luup_log:0: (null) <0x71f4a520>
50   12/30/17 21:52:46.808   luup_log:0: (null) <0x71f4a520>
50   12/30/17 21:52:46.808   luup_log:0: ??   ???<0x71f4a520>
50   12/30/17 21:52:46.808   luup_log:0: 11 <0x71f4a520>
50   12/30/17 21:52:46.809   luup_log:0: 9 <0x71f4a520>
50   12/30/17 21:52:46.809   luup_log:0: 0.0028560161590576 <0x71f4a520>


Now the question is : how to make a correct presentation: like in attached Hercules software.
Received value is string but in strange format.

How to correct it's presentation?