Author Topic: How to send ASCII (with STX & ETX) via USB Serial Cable.  (Read 583 times)

Offline parkerc

  • Sr. Hero Member
  • ******
  • Posts: 2364
  • Karma: +33/-45
  • Life Moves Pretty Fast....
    • Node Central
How to send ASCII (with STX & ETX) via USB Serial Cable.
« on: September 09, 2017, 07:20:17 pm »
Hi

I'm curious - if the following code works successfully in sending HEX values, what has to change so it can send ASCII - see attached.?

Code: [Select]
local function send_command_to_hdmi_matrix_via_ser2net(...)
local socket = require("socket")
host = "192.168.1.204"
c = assert(socket.connect(host, 4002))
c:settimeout(5)
local sres, serr = c:send(string.char(0x02,0x32,0x31,0x31,0x03))
local data, rerr = c:receive(100)
c:close()end

I assumed rather simplistically that I could just change the "c:send(string.char" part but that does not seem to work ?

Code: [Select]
local sres, serr = c:send(string.char(<STX>221<ETX>))
« Last Edit: September 09, 2017, 08:14:36 pm by parkerc »

Offline cybrmage

  • Hero Member
  • *****
  • Posts: 1053
  • Karma: +105/-5
Re: How to send ASCII (with STX & ETX) via USB Serial Cable.
« Reply #1 on: September 09, 2017, 07:51:57 pm »

I assumed rather simplistically that I could just change the "c:send(string.char" part but that does not seem to work ?


Code: [Select]
local sres, serr = c:send(string.char(<STX>221<ETX>))

You were close.... Your mistake was assuming that the "<STX>" and "<ETX>" would be translated by the string.char function. They are not. So, you need to concatenate the string you wish to send... (also, strings need to be inside quotes - either single or double (usually double))

]

Code: [Select]
local sres, serr = c:send(string.char(0x02).."221"..string.char(0x03))

OR

Code: [Select]
local STX = string.char(0x02)
local ETX = string.char(0x03)
local sres, serr = c:send(STX.."221"..ETX)
« Last Edit: September 10, 2017, 05:05:35 am by cybrmage »
EVL3Vista - Wink Connect - Caseta Connect - Venstar Colortouch - WiFi UDP Switch Controller - Broadlink RM - MyQGateway

Offline parkerc

  • Sr. Hero Member
  • ******
  • Posts: 2364
  • Karma: +33/-45
  • Life Moves Pretty Fast....
    • Node Central
Re: How to send ASCII (with STX & ETX) via USB Serial Cable.
« Reply #2 on: September 09, 2017, 08:33:22 pm »
Many thanks @cybrmage - I'll give that a try.

Offline parkerc

  • Sr. Hero Member
  • ******
  • Posts: 2364
  • Karma: +33/-45
  • Life Moves Pretty Fast....
    • Node Central
Re: How to send ASCII (with STX & ETX) via USB Serial Cable.
« Reply #3 on: September 10, 2017, 04:40:07 am »
Quick question, does the code mean I'm actually just sending the numerical value of 0222103 ?

Offline cybrmage

  • Hero Member
  • *****
  • Posts: 1053
  • Karma: +105/-5
Re: How to send ASCII (with STX & ETX) via USB Serial Cable.
« Reply #4 on: September 10, 2017, 05:04:50 am »
No....

You are sending a series of (in this case 5) single byte (or character) values....

0x02 0x32 0x32 0x31 0x03

so...

Code: [Select]
local sres, serr = c:send(string.char(0x02).."221"..string.char(0x03))

sends exactly the same data as

Code: [Select]
local sres, serr = c:send(string.char(0x02,0x32,0x32,0x31,0x03))


You should read https://en.wikipedia.org/wiki/ASCII

EVL3Vista - Wink Connect - Caseta Connect - Venstar Colortouch - WiFi UDP Switch Controller - Broadlink RM - MyQGateway

Offline parkerc

  • Sr. Hero Member
  • ******
  • Posts: 2364
  • Karma: +33/-45
  • Life Moves Pretty Fast....
    • Node Central
Re: How to send ASCII (with STX & ETX) via USB Serial Cable.
« Reply #5 on: September 10, 2017, 06:50:42 am »
Many thanks

I guess It's all the references to STX and ETX that confuse me - as it makes me think I have to send those start and end  parts as "text" not as a byte - allowing then the Target device to do the conversion.

The first example I shared was to change a channel (I think)  221 - but there are other commands that are letters in e.g <STX>PON<ETX> - i assume for those I need to do the byte conversion for each letter then so P would be 0x80.

http://www.theasciicode.com.ar/ascii-control-characters/start-of-text-ascii-code-2.html

Thus resulting in..

Code: [Select]
local sres, serr = c:send(string.char(0x02,0x80,0x79,0x78,0x03))
But how would that work this way replacing 221 with PON, mixing numbers and text  ?

Code: [Select]
local STX = string.char(0x02)
local ETX = string.char(0x03)
local sres, serr = c:send(STX.."PON"..ETX)

« Last Edit: September 10, 2017, 03:40:23 pm by parkerc »

Offline a-lurker

  • Hero Member
  • *****
  • Posts: 752
  • Karma: +40/-7
Re: How to send ASCII (with STX & ETX) via USB Serial Cable.
« Reply #6 on: September 10, 2017, 09:53:52 pm »
The confusion probably comes about, due to the ways a byte's contents can be expressed and also utilised.

A byte consists of 8 binary bits, allowing 256 combinations of ones & zeroes. A particular combination can typically be expressed as it's binary contents (00000000 to 11111111) or as a decimal number (0 to 255) or as a hexdecimal number (0 to ff). In fact, any numbering base can be used, such as octal etc.

So we can now specify, which byte combination, we are talking about. Once we have a specific combination expressed in say decimal or in hex, what does that particular combination mean? Well it could just describe a number from 0 to 255. Or in the case of an ASCII table it could specify particular printable characters as used by the English language. In the ASCII table the binary combination for P is assigned the byte combination 80 decimal or 50 hexadecimal. The combinations could just as easily be assigned to different types of fruit, rather than printable English characters.

As it turns out, after assigning all the printable characters with a combination, some were left over. So (historically) binary combination 00000010 was assigned STX and 00000011 was assigned ETX, as it was figured these commands could be useful. They are just abbreviations for a control character meaning  End of TeXt  or  Start of TeXt. Since they can't be printed, they got names, based on what they did or indicated.

http://www.asciitable.com/

Code: [Select]
-- expressing unprintable bytes in hexadecimal in Lua 5.2
-- \x02 means insert the third binary combination, which in this case we consider represents the unprintable control code STX, into the string of bytes
-- sending <STX>PON<ETX> using hexadecimal
local sres, serr = c:send("\x02PON\x03")

-- expressing unprintable bytes in decimal ie \002 and \003 in Lua 5.2
local sres, serr = c:send("\002PON\003")

-- sending <STX>221<ETX> using hexadecimal
local sres, serr = c:send("\x02221\x03")

-- as above using decimal
-- always use leading zeroes to ensure numbers 0 to 99 are shown as a three digit cluster
-- this ensures genuine numbers ie 221 in this case, aren't confused by the system
-- due to the above, use the hex forms \x02 and \x03, as it's less prone to coder error
local sres, serr = c:send("\002221\003")

Note that you expressed the letter P as 0x80   That is you expressed a bit combination using hexadecimal. The combination for P is 80 decimal. 80 hex is not a binary combination used in the ASCII table, as the table is encoded iusing 7 bits, not eight. ie one byte with the most significant bit unused.

Hope this makes sense and I'm not explaining how to suck eggs.

Offline parkerc

  • Sr. Hero Member
  • ******
  • Posts: 2364
  • Karma: +33/-45
  • Life Moves Pretty Fast....
    • Node Central
Re: How to send ASCII (with STX & ETX) via USB Serial Cable.
« Reply #7 on: September 11, 2017, 04:57:41 am »
Thanks so much for that @a-lurker, it was by no means a lesson in sucking eggs far from it (it's an education) . However :) -  what I can't grasp is which one you use ?

If the RS-232 guide for e.g a TV or Matrix switcher just says use ASCII? How do you know what to use - Do you use decimal or hexadecimal  ?

Thanks to the helps of this community I now have my Octava HDMI Matrix switcher working by sending the Hexadecimal format.

So now looking at my TV (an old Pioneer Plasma) it's asking for ASCII. Extracts from the manual below.

Quote
1.3 Communication conditions Set the RS-232C communications settings on the computer to match the display's communications conditions. The display's communications settings are as follows:

Baud rate 1200/2400/4800/9600/19200/38400bps
Start bit 1bit
Data length 8bit (ASCII)
Parity bit None
Stop bit 1 bit 

With the structure of the commands and parameters needing to be as follows...

Quote
Command
STX - Start condition (fixed value 02h)
ID - Fixed value **
Command - 3byte (ASCII)
Parameter - 3byte (ASCII)
ETX - Stop condition (fixed value 03h)

Parameter (optional)
000~999 
UPn - Up n step n:0~9 (0 is increase 10)
DWn - Down n step n:0~9 (0 is decrease 10)
FWD - Forward TV channel
REV - Backward TV channel

An example 'Command' would therefore be VOL
An example 'Parameter' would be UP1 (increment 1 volume unit)

I assume 02h means 0x02 (Hexadecimal)


With the above instructions in mind; how do you know which one you should use - or are both equally valid ?

Code: [Select]
local sres, serr = c:send(string.char(\002**VOLUP1\003))
Or

Code: [Select]
local sres, serr = c:send(string.char(\x02**VOLUP1\x03))

Does the use of the word ASCII mean it can accept any format in the ASCII lookup/conversion table (http://www.asciitable.com/)?  Decimal, Hexadecimal, Oct(adecimal?) or HTML ?
« Last Edit: September 11, 2017, 05:15:39 am by parkerc »

Offline futzle

  • Beta Testers
  • Master Member
  • *****
  • Posts: 3247
  • Karma: +188/-9
How to send ASCII (with STX & ETX) via USB Serial Cable.
« Reply #8 on: September 11, 2017, 08:43:08 am »
What I can't grasp is which one you use ?

They are equivalent; it doesn't matter to the serial device you are talking to. You use the one that makes most sense to you as the programmer of the code.

I used to be a computer science teacher, so I would see this stage of comprehension a lot with students. Remember, above all, that computers deal with data only as binary digits (bits), usually in octets (sets of eight) called bytes. Serial ports are absolutely in this category; recall that when you configure a serial port you specify not only the speed but also the data size as (usually) 8 bits.

All those different ways of writing the c:send() call transmit exactly the same series of bits out over the serial port. Here are six different ways to represent the byte with bit sequence 01000001:

Code: [Select]
string.char(65)
string.char(0x41)
'A'
"A"
"\065"
'\065'

Lua, like most languages, gives you lots of different ways to do exactly the same thing. string.char() produces bytes from numbers, for instance (making it a spectacularly badly-named function, but it's historical and what programmers expect). The quote marks " and ' produce bytes by looking up the enclosed symbol in the ASCII table.

(Digression: Quotes have mechanisms to produce bytes that you can't directly type, which is what the backslash is doing. We say that backslashes in quotes are magical, that they "escape" what follows from normal interpretation rules. You don't need to escape this example as "\065" because you can type it as "A", but some bytes don't have keys you can type into a program.  STX and ETX are like that: you could type "\002" for STX, or you could just type string.char(2) instead. Until you are comfortable with working with sequences of bytes I'd suggest staying away from backslash escaping, and just using string.char().)

Here are a few ways to represent the two-byte sequence 01000001 00111001:

Code: [Select]
string.char(65) .. string.char(57)
string.char(0x41) .. '9'
"A" .. "9"
"A9"

The .. operator joins bytes together into a sequence. Sequences of bytes are often called "strings" but that's a loaded word and I'm going to avoid it. Either side of the .. operator can be any of the ways that you can construct bytes. I've deliberately mixed them up in my examples. You may or may not want to do that.

One very common shorthand is to put a sequence of ASCII characters inside quotes, like my last example "A9". That is exactly the same as saying "A" .. "9" but it's easier to type.

You can extend sequences to any length by adding more .. operators between bytes.  Just stick to some rules about making bytes:
    • string.char() wraps around numbers. It does not care about the ASCII table. If you find yourself putting ASCII characters inside string.char(), you're doing it wrong.
    • quotation marks wrap around characters, which are converted to bytes according to the ASCII table. The quote marks aren't optional. If you want to make a byte from an ASCII character, it must be surrounded by quotes.

I'm going to leave you with some homework.  Why aren't these two bytes the same?
Code: [Select]
string.char(9)
"9"

Edit: formatting
« Last Edit: September 11, 2017, 08:46:53 am by futzle »

Offline parkerc

  • Sr. Hero Member
  • ******
  • Posts: 2364
  • Karma: +33/-45
  • Life Moves Pretty Fast....
    • Node Central
Re: How to send ASCII (with STX & ETX) via USB Serial Cable.
« Reply #9 on: September 11, 2017, 11:13:48 am »
Thanks so much @futzle

Now you've put me on the spot.....  :o

... Why aren't these two bytes the same?
Code: [Select]
string.char(9)
"9"

Is it because they are different values : The number 9 in the ASCII table looks to be a control character for Horizontal Tab "HT" ?

Regarding...
.. string.char() wraps around numbers. It does not care about the ASCII table. If you find yourself putting ASCII characters inside string.char(), you're doing it wrong.

So that means rather than my earlier mistake of this as it seems string/char will send exactly what you specify, without doing any ASCII conversion..

Code: [Select]
local sres, serr = c:send(string.char(\002**VOLUP1\003))
It should be this  ?

Code: [Select]
local STX = string.char(0x02)
local ETX = string.char(0x03)
local sres, serr = c:send(STX.."**VOLUP1"..ETX)
-- which is the same as this
local sres, serr = c:send(string.char(0x02) .. "**VOLUP1" .. string.char(0x03))

or this ?

Code: [Select]
local sres, serr = c:send(string.char(002) .. "**VOLUP1" .. string.char(003))
or this ?

Code: [Select]
local sres, serr = c:send("2**VOLUP13")
« Last Edit: September 11, 2017, 11:29:11 am by parkerc »

Offline futzle

  • Beta Testers
  • Master Member
  • *****
  • Posts: 3247
  • Karma: +188/-9
Re: How to send ASCII (with STX & ETX) via USB Serial Cable.
« Reply #10 on: September 11, 2017, 10:27:44 pm »
Is it because they are different values : The number 9 in the ASCII table looks to be a control character for Horizontal Tab "HT" ?

Yes!

Quote
It should be this  ?

Code: [Select]
local STX = string.char(0x02)
local ETX = string.char(0x03)
local sres, serr = c:send(STX.."**VOLUP1"..ETX)
-- which is the same as this
local sres, serr = c:send(string.char(0x02) .. "**VOLUP1" .. string.char(0x03))

Yes!!

Quote
or this ?
Code: [Select]
local sres, serr = c:send(string.char(002) .. "**VOLUP1" .. string.char(003))

Yes!!!

Quote
or this ?
Code: [Select]
local sres, serr = c:send("2**VOLUP13")

... no (sorry). Because that would be the same as

Code: [Select]
local sres, serr = c:send("2" .. "**VOLUP1" .. "3")
and you've already correctly explained to me that "2" isn't the same as string.char(2).  A seasoned Lua user might write c:send("\002**VOLUP1\003"), taking advantage of backslash escaping, but if you're not comfortable with that then any of your other examples are spot on.

Offline parkerc

  • Sr. Hero Member
  • ******
  • Posts: 2364
  • Karma: +33/-45
  • Life Moves Pretty Fast....
    • Node Central
Re: How to send ASCII (with STX & ETX) via USB Serial Cable.
« Reply #11 on: September 12, 2017, 03:42:04 am »
So close !! :)  (Thank you so much for that test )


I think I'm almost there - It's just this part, where we need to use something within the first set in the ASCII table e.g STX, ETX, HT etc that I'm not 100% clear on..

Code: [Select]
local sres, serr = c:send("2**VOLUP13"
... no (sorry). Because that would be the same as

Code: [Select]
local sres, serr = c:send("2" .. "**VOLUP1" .. "3")
and you've already correctly explained to me that "2" isn't the same as string.char(2).  A seasoned Lua user might write c:send("\002**VOLUP1\003"), taking advantage of backslash escaping, but if you're not comfortable with that then any of your other examples are spot on.

In this table (http://www.asciitable.com/)?) 2 (for STX) looks to be just 2 in both DEC & HEX but 002 in OCT? Where as the number 2 is 50 in DEC, 32 in HEX and 062 in OCT.

Which made me think it could be any of the following for STX, which it seems is wrong, but how is it wrong when in the ASCII table for A  can write "A" or string.char(0x61) why not this for the serial port/device to know it's STX?  - sorry for this, as I know for you all this is an obvious thing  :-[

Code: [Select]
string.char(2)  -- DEC
string.char(0x2)   -- HEX
string.char(0x02)   -- OCT
'2'
"2"
"\2"
'\2'
« Last Edit: September 12, 2017, 03:57:26 am by parkerc »

Offline futzle

  • Beta Testers
  • Master Member
  • *****
  • Posts: 3247
  • Karma: +188/-9
Re: How to send ASCII (with STX & ETX) via USB Serial Cable.
« Reply #12 on: September 12, 2017, 08:14:17 am »
Where as the number 2 is 50 in DEC, 32 in HEX and 062 in OCT.

Careful there.  You're conflating "number" with "digit character", which you can get away with in the real world but not when talking to computers.

Numbers are quantities, they are the same no matter what base you express them in, and you can do arithmetic on them.  2 is a number, whether you write it 2, two, 0x02, 00000002 or (in binary) 10.  They are all the same quantity.  Likewise, 19 is 0x13 is nineteen is dix-neuf.  19 is greater than 2; the difference between 19 and 2 is 17; the average of 19 and 2 is 10.5.

Digits are a kind of character, as are letters, punctuation, spaces and anything else you can write or type.  A sequence of digits is not a number.  There are lots of real-world examples of sequences of digits which are not numbers.  Your telephone "number" isn't a number.  You can't add two telephone numbers.  If you add zeroes on the front of a telephone number you call a different person.  My telephone number isn't "bigger than" or "smaller than" your telephone number.  If you express your telephone number in a different base it ceases to be something that someone can dial.  Other examples: Dewey decimal system library codes are like this.  Barcodes on groceries are like this.  Your street "number" is like this.  US Social security "numbers" are like this.  US and many European postal codes are like this.  These are not numbers.  They are strings that happen to be made only of digits.

The character '2' is a digit, with the ASCII value number 00110010 (binary), 50 (decimal), 0x32 (hex), etc.

The character which has ASCII value number 00000010 (binary), 2 (decimal), 0x02 (hex), etc. is the special ASCII STX character, which you can't type on a keyboard.

See why the quotation marks are important?  They distinguish numbers (no quotes) from characters (quotes).  Lua insists on this distinction, because characters and numbers are different types in Lua.  (Other languages have soft, automatic, implied conversion between numbers and characters.  JavaScript is notorious for this, making it the butt of jokes from users of other languages.  But I digress.)

Quote
Which made me think it could be any of the following for STX, which it seems is wrong, but how is it wrong when in the ASCII table for A  can write "A" or string.char(0x61) why not this for the serial port/device to know it's STX?  - sorry for this, as I know for you all this is an obvious thing  :-[

Code: [Select]
string.char(2)  -- DEC
string.char(0x2)   -- HEX
string.char(0x02)   -- OCT

These will produce an STX byte with bit pattern 00000010.  (Except your third example isn't how to write an octal number in Lua, but I digress.)

Quote
Code: [Select]
'2'
"2"

These will produce a byte from the character "2", which (because it is inside quotation marks) is not a number, but a character to be looked up in the ASCII table.  It results in the byte with bit pattern 00110010.

Put another way, in the context of www.asciitable.com:
  • string.char(x) for any number x: look it up in the black column (only).
  • "x" for any single character x: look it up in the red column (only).

Come back to the rest of this post when you're comfortable with what's been said so far.

Quote
Code: [Select]
"\2"
'\2'

These are probably malformed Lua because IIRC the rules for backslash escaping require exactly three digits after the backslash.  If instead you write:
Code: [Select]
"\002"
'\002'
then you will have valid Lua, and you will be invoking Lua's backslash escaping syntax, which is:
  • Take the three characters after the backslash, which must all be digits.
  • Convert those three characters into a number, as if it was written in decimal
  • Result is string.char(that number)
which is the byte with bit sequence 00000010, as in your first three examples.  But as I said before, escaping is magical and breaks normal rules.  Understand how to do it without escaping first.

Offline parkerc

  • Sr. Hero Member
  • ******
  • Posts: 2364
  • Karma: +33/-45
  • Life Moves Pretty Fast....
    • Node Central
Re: How to send ASCII (with STX & ETX) via USB Serial Cable.
« Reply #13 on: September 12, 2017, 11:15:01 am »
I think the fog is lifting (thanks so much for your patience).  8)

A couple of clarification points

1) I appreciate it's not ideal/best practice - but theoretically a developer could mix the 'number systems/formats' they use in their code;  for example they can mix Dec & Hex - like below - and it would still work ?

local STX = string.char(0x02)  - STX in HEX
local ETX = string.char(3)       - ETX in DEC

local sres, serr = c:send(STX.."**VOLUP1"..ETX)

-- which would be the same as this

local sres, serr = c:send(string.char(0x02) .. "**VOLUP1" .. string.char(3))

2) I like your explanation here ..

Quote
        Put another way, in the context of www.asciitable.com:
        string.char(x) for any number x: look it up in the black column (only).
        "x" for any single character x: look it up in the red column (only).

Although - correct me if I'm wrong; would an exception to that red column rule be those control words in red e.g. STX, ETX etc. as

        If I looked to send "A" - the equivalent of that in HEX would be = string.char(0x41)
        If I looked to send "STX" - wouldn't the equivalent of that in HEX be split into 3 characters - string.char(0x53, 0x54, 0x78)


Offline parkerc

  • Sr. Hero Member
  • ******
  • Posts: 2364
  • Karma: +33/-45
  • Life Moves Pretty Fast....
    • Node Central
Re: How to send ASCII (with STX & ETX) via USB Serial Cable.
« Reply #14 on: September 12, 2017, 11:26:45 am »
I have to laugh  ;D - or cry  :'(!!

Reading back my first post within this thread again (with my new and improving knowledge) , the images I'd uploaded from two different serial devices one promoting HEX and the other ASCII are basically saying the same thing !!  So while I was asking about how I change this

Code: [Select]
local sres, serr = c:send(string.char(0x02,0x32,0x31,0x31,0x03))
in order to use STX at the start and ETX at the end   - yet it already does as the following code is actually sending the digits "211" (formatted in HEX), wrapped in STX and ETX  ::)

Quote
local sres, serr = c:send(string.char(0x02,0x32,0x31,0x31,0x03))

UPDATE : A bit more testing of the different formats and I can confirm all of these variations so far work with my Octava HDMI Matrix Switcher (who's manual just promotes HEX) 

Code: [Select]
local sres, serr = c:send(string.char(0x02) .. "211" .. string.char(0x03))
local sres, serr = c:send(string.char(0x02) .. "211" .. string.char(3))
local sres, serr = c:send(string.char(2) .. "211" .. string.char(0x03))
local sres, serr = c:send(string.char(0x02,0x32,0x31,0x31,0x03))
local sres, serr = c:send(string.char(2) .. "211" .. string.char(3))
local sres, serr = c:send("\002211\003")

Written again to highlight the STX and ETX parts in red

local sres, serr = c:send(string.char(0x02) .. "211" .. string.char(0x03))
local sres, serr = c:send(string.char(0x02) .. "211" .. string.char(3))
local sres, serr = c:send(string.char(2) .. "211" .. string.char(0x03))
local sres, serr = c:send(string.char(0x02,0x32,0x31,0x31,0x03))
local sres, serr = c:send(string.char(2) .. "211" .. string.char(3))
local sres, serr = c:send("\002211\003")
« Last Edit: September 12, 2017, 12:17:11 pm by parkerc »