Author Topic: ALTUI: Speech recognition  (Read 2576 times)

Offline amg0

  • Moderator
  • Sr. Hero Member
  • *****
  • Posts: 2962
  • Karma: +193/-8
ALTUI: Speech recognition
« on: March 05, 2016, 11:33:21 am »
This thread is meant to discuss speech recognition technic & features in ALTUI v >= 1237. I ll start by explaining the current features.

ALTUI built in speech recognition is a "low end cost" speech enabled solution which lets you talk to your browser and ask simple commands like switching on/off a binaryswitch object, running a scene, opening a specific page in ALTUI.  it is "low end" because it can only work when your browser is opened in front of you ( as opposed to Amazon Echo / Siri or similar kind of devices which are constantly listening ) but it is low end and low effort as the only thing you need is :
  • ALTUI installed and running on your vera ( or openLuup )
  • a Speech compatible browser/device like Chrome on a desktop PC

How it works
ALTUI uses the HTML5 web speech api as provided by the webkitSpeechRecognition() javascript object.
the standard documentation can be found here here and unfortunately the browser support is limited ( shown here ) and I have only seen working on Chrome Desktop version so far.

When the browser is open and if the browser supports speech recognition, there is a loudspeaker icon next to the message area of the page. you can click on that icon and it will turn the listening mode on and change that icon color to red. cf the screen shot

note that if you opened ALTUI locally ( using http protocol ) , the browser will ask you the permission to do this every time, which is a bit of a pain.  if you opened it remotely ( using https protocol ) , it should only ask you once.

What commands are supported
Commands are based on regular expression match on the recognized text so some flexibility is possible but not a whole lot. Switch on Blue Lamp ,  or Switch on The Blue lamp will work but it requires that a device is named "Blue Lamp"
  • switch on (off) or open (close) <devicename> for devices supporting the service:"urn:upnp-org:serviceId:SwitchPower1", action:"SetTarget"
  • run (launch) <scenename>
  • show (open) <pagename> where pagename is either "Device", "Scene" or "Custom Pages"

Supported commands are based on a data structure which can be extended, (although there is no user interface for that yet ). you will recognize a regular expression and some substitution variables. %name% for device or scene name, %page% for a ALTUI page name. Rules are evaluated in order of preference so the first that matches is the one that will be used. as you can see it is easy to add more and to add synonims to commands

Code: [Select]
SpeechManager.initRules([
{r:"(switch on|turn on|open)\\s+.*(%name%)", t:"device", a:{service:"urn:upnp-org:serviceId:SwitchPower1", action:"SetTarget", params:"{ \"newTargetValue\":1}"}},
{r:"(switch off|turn off|close)\\s+.*(%name%)", t:"device", a:{service:"urn:upnp-org:serviceId:SwitchPower1", action:"SetTarget", params:"{ \"newTargetValue\":0}"}},
{r:"(run|launch)\\s+.*(%name%)", t:"scene" },
{r:"(show|open)\\s+.*(%name%)", t:"altui" },
{r:"(show|open)\\s+.*(%name%)", t:"room" }
]);

What language are supported
  • English
  • French

ALTUI language support can be extended and so does voice commands. I will gladly help anybody who wants to contribute to this and it is designed to be extensible if somebody is willing to write a custom language file like this french one for instance.
http://code.mios.com/svn_public/mios_alternate_ui/J_ALTUI_loc_fr.js

ALTUI supports a url parameter to override the default browser language, and the speech recognition will follow that choice as well

in French support voice commands are:
Code: [Select]
SpeechManager.initRules([
{r:"(allumer|allume|ouvrir|ouvre)\\s+.*(%name%)", t:"device", a:{service:"urn:upnp-org:serviceId:SwitchPower1", action:"SetTarget", params:"{ \"newTargetValue\":1}"}},
{r:"(?teindre|?teins|fermer|ferme)\\s+.*(%name%)", t:"device", a:{service:"urn:upnp-org:serviceId:SwitchPower1", action:"SetTarget", params:"{ \"newTargetValue\":0}"}},
{r:"(ex?cuter|lancer)\\s+.*(%name%)", t:"scene" },
{r:"(montre|montrer|ouvre|ouvrir)\\s+.*(%name%)", t:"altui" },
{r:"(montre|montrer|ouvre|ouvrir)\\s+.*(%name%)", t:"room" }
]);

NOTE: version < 1239 requires a hot fix ( attached ) for proper language support in remote access

Can I extend commands
not yet unless by asking me or modifying the source code yourself at this moment. search for "SpeechManager.initRules" and modify the table data structure in J_ALTUI_uimgr.js for english and J_ALTUI_loc_xx.js for language related one

Caveats to know
  • first of all, it requires internet access
  • browser support. quite limited and unfortunately not on IOS for now, not sure for Android
  • recognition quality although I find it quite good but I have trouble using the english one for instance ( due to my crappy accent :-) )
  • device names and scenes names are to be clean and pronouncable things, otherwise you run into trouble, I do filter their non alphabetical characters out so things like (*) should be harmless
  • for now device & scene names need to be unique accross all rooms , I ll see if I can work on that one

« Last Edit: March 06, 2016, 09:14:27 am by amg0 »

Offline ilikelife

  • Full Member
  • ***
  • Posts: 129
  • Karma: +2/-0
Re: ALTUI: Speech recognition
« Reply #1 on: March 05, 2016, 01:32:24 pm »
It's pretty cool already!  8)

One suggestion for an added command would be "Turn" on/off  (As a Midwestern American, "Switch on" seems a bit European to me)   :)

I'm happy to add it myself with your excellent instructions.

And to add to your caveats about "Clean Names" being important, I had trouble with my "Switch1" until I added a space to name it "Switch 1" and my "Smartswitch" had to change to "Smart switch"

It's VERY helpful that you show the process of parsing the voice commands.  I saw "smartswitch become two words, so I understood why it didn't match.  I would like to see the successful commands display a little longer, but that's probably just because it's new to me.

I agree the permission confirmation each time is a pain, but this amazing for the first version!

Offline amg0

  • Moderator
  • Sr. Hero Member
  • *****
  • Posts: 2962
  • Karma: +193/-8
Re: ALTUI: Speech recognition
« Reply #2 on: March 06, 2016, 08:33:22 am »
It's pretty cool already!  8)

One suggestion for an added command would be "Turn" on/off  (As a Midwestern American, "Switch on" seems a bit European to me)   :)

I'm happy to add it myself with your excellent instructions.

And to add to your caveats about "Clean Names" being important, I had trouble with my "Switch1" until I added a space to name it "Switch 1" and my "Smartswitch" had to change to "Smart switch"

It's VERY helpful that you show the process of parsing the voice commands.  I saw "smartswitch become two words, so I understood why it didn't match.  I would like to see the successful commands display a little longer, but that's probably just because it's new to me.

I agree the permission confirmation each time is a pain, but this amazing for the first version!

v1240 will
add vocal command "show rooms"
add vocal command "show <roomname>"
add vocal command "turn on/off <device>"