Author Topic: MQTT Client implementation  (Read 13120 times)

Offline 4integration

  • Sr. Newbie
  • *
  • Posts: 47
  • Karma: +0/-0
MQTT Client implementation
« on: November 07, 2013, 04:43:14 pm »
I am trying to load a lua module but fails...all files uploaded

In "I_SensorMqtt1.xml" I have the following code but it fails at

u = require("utility")

Do you have any hints what's wrong?

Code: [Select]
<?xml version="1.0"?>
<implementation>
    <functions>
        local p
    local u
        local m
       
        function startup(lul_device)
            luup.log("SensorMqtt plugin: loading library L_SensorMqtt ...")
            if (package.path:find ("/etc/cmh-ludl/?.lua;/etc/cmh-lu/?.lua", 1, true) == nil) then
                package.path = package.path .. ";/etc/cmh-ludl/?.lua;/etc/cmh-lu/?.lua"
            end
           
            package.loaded.L_SensorMqtt = nil
            p = require("L_SensorMqtt")
            if (package.loaded.L_SensorMqtt == nil)
            then
                luup.log("SensorMqtt plugin: plugin is not installed correctly. Library L_SensorMqtt cannot be loaded.", 1)
                luup.task("Plugin not correctly installed", 2, "SensorMqtt plugin", -1)
                return false
            end
            luup.log("SensorMqtt plugin: library L_SensorMqtt loaded")

            luup.log("SensorMqtt plugin: loading library utility ...")
            package.loaded.utility = nil
            u = require("utility")
            if (package.loaded.utility == nil)
            then
                luup.log("SensorMqtt plugin: plugin is not installed correctly. Library utility cannot be loaded.", 1)
                luup.task("utility not correctly installed", 2, "SensorMqtt plugin", -1)
                return false
            end
            luup.log("SensorMqtt plugin: library utility loaded")

            package.loaded.mqtt_library = nil
            m = require("mqtt_library")
            if (package.loaded.mqtt_library == nil)
            then
                luup.log("SensorMqtt plugin: plugin is not installed correctly. Library mqtt_library cannot be loaded.", 1)
                luup.task("mqtt_library not correctly installed", 2, "SensorMqtt plugin", -1)
                return false
            end
            luup.log("SensorMqtt plugin: library mqtt_library loaded")
           
            return p.startup(lul_device)
        end
    </functions>
    <startup>startup</startup>
</implementation>
« Last Edit: November 10, 2013, 06:17:48 am by 4integration »

Offline watou

  • Hero Member
  • *****
  • Posts: 866
  • Karma: +43/-12
Re: Loading lua modules
« Reply #1 on: November 07, 2013, 04:50:01 pm »
I am trying to load a lua module but fails...all files uploaded

Loading would fail if there were a syntax error in utility.lua . 

Also, if your module is encrypted (file name extension .luaenc) then it has to explicitly name itself in the first line like this:

Code: [Select]
module("utility",package.seeall)
If you tried to instead use:

Code: [Select]
module("...",package.seeall)
Then MIOS's custom module loader for encrypted modules will fail (probably a minor bug in their loader, at least as of 1.5.408).

watou

Offline 4integration

  • Sr. Newbie
  • *
  • Posts: 47
  • Karma: +0/-0
Re: Loading lua modules
« Reply #2 on: November 08, 2013, 06:09:28 pm »
Thanks watou, tried that but no success. Moved the other loading into the lua code and that works but now I have another issue.

The MQTT Lua modules is here: http://git.eclipse.org/c/paho/org.eclipse.paho.mqtt.lua.git/tree/lua

Do you see anything strange? (I know it's a bit out of scope for MCV but since you are skilled Lua guys/girls)

Error:
Code: [Select]
01      11/09/13 0:00:36.101    luup_log:62: SensorMqtt plugin: Enter sendToMqtt <0x2d773680>
01      11/09/13 0:00:36.101    luup_log:62: SensorMqtt plugin: Create MQTT Client <0x2d773680>
01      11/09/13 0:00:36.102    luup_log:62: SensorMqtt plugin: Connect to 192.168.1.21 and port 1883 <0x2d773680>
01      11/09/13 0:00:36.102    LuaInterface::CallFunction_Timer-5 function sendToMqtt failed [string "module("L_SensorMqtt", package.seeall)..."]:47: attempt to index field 'client' (a nil value) <0x2d773680>

The code:
Code: [Select]
module("L_SensorMqtt", package.seeall)

-- Service ID strings used by this device.
SERVICE_ID = "urn:upnp-sensor-mqtt-se:serviceId:SensorMqtt1"

local mqttServerIp = nil
local mqttServerPort = 0

local MQTT = require("mqtt_library")
local SENSOR_MQTT_LOG_NAME = "SensorMqtt plugin: "

local function log(text, level)
luup.log("SensorMqtt plugin: " .. text, (level or 50))
end

function startup(lul_device)
_G["sendToMqtt"] = sendToMqtt
-- Help prevent race condition ???
-- luup.io.intercept()

SENSOR_MQTT_DEVICE = lul_device
log("Initialising SensorMqtt", 1)

--Reading variables
mqttServerIp = luup.variable_get(SERVICE_ID, "mqttServerIp", SENSOR_MQTT_DEVICE)
if(mqttServerIp == nil) then
mqttServerIp = "0.0.0.0"
luup.variable_set(SERVICE_ID, "mqttServerIp", mqttServerIp, SENSOR_MQTT_DEVICE)
end

mqttServerPort = luup.variable_get(SERVICE_ID, "mqttServerPort", SENSOR_MQTT_DEVICE)
if(mqttServerPort == nil) then
mqttServerPort = "0"
luup.variable_set(SERVICE_ID, "mqttServerPort", mqttServerPort, SENSOR_MQTT_DEVICE)
end
log("Start - Preparing worker thread", 1)
-- Prepare the worker "thread"
luup.call_delay("sendToMqtt", 20, "")
log("Done - Preparing worker thread", 1)
end

function sendToMqtt(luup_data)
luup.log(SENSOR_MQTT_LOG_NAME .. "Enter sendToMqtt", 1)
luup.log(SENSOR_MQTT_LOG_NAME .. "Create MQTT Client", 1)
luup.log(SENSOR_MQTT_LOG_NAME .. "Connect to " .. mqttServerIp .. " and port " .. mqttServerPort, 1)
mqttServerPort = tonumber(mqttServerPort)
local mqtt_client = MQTT.client.create(mqttServerIp, mqttServerPort)

luup.log(SENSOR_MQTT_LOG_NAME .. "Connect MQTT Client", 1)
mqtt_client:connect("Vera")

luup.log(SENSOR_MQTT_LOG_NAME .. "Publish MQTT message", 1)
mqtt_client:publish("Vera/Test", "*** Vera test start ***")

--mqtt_client:subscribe({ args.topic_s })

luup.log(SENSOR_MQTT_LOG_NAME .. "Destroy MQTT Client", 1)
mqtt_client:destroy()

luup.log(SENSOR_MQTT_LOG_NAME .. "Done with MQTT", 1)

luup.call_delay("sendToMqtt", 10, "")
luup.log(SENSOR_MQTT_LOG_NAME .. "Leaving sendToMqtt", 1)
end


Offline watou

  • Hero Member
  • *****
  • Posts: 866
  • Karma: +43/-12
Re: Loading lua modules
« Reply #3 on: November 08, 2013, 06:22:42 pm »
I think your MQTT variable needs global scope so it can be seen when luup.call_delay calls your sendToMqtt function.  Since it's not visible in scope MQTT.client has a nil value, since it's not the MQTT you think it is.

I think that's it, anyway.

watou



Offline 4integration

  • Sr. Newbie
  • *
  • Posts: 47
  • Karma: +0/-0
Re: Loading lua modules
« Reply #4 on: November 09, 2013, 08:07:57 am »
Thanks for you idea, tested with both removing "local" as well as using _G ...
Code: [Select]
MQTT = require("mqtt_library")
_G["MQTT"] = MQTT

To test more simple code I removed "luup.call_delay(...)" and called MQTT in the startup method, but got the same result :(

Offline watou

  • Hero Member
  • *****
  • Posts: 866
  • Karma: +43/-12
Re: Loading lua modules
« Reply #5 on: November 09, 2013, 08:19:21 am »
Thanks for you idea, tested with both removing "local" as well as using _G ...
Code: [Select]
MQTT = require("mqtt_library")
_G["MQTT"] = MQTT

To test more simple code I removed "luup.call_delay(...)" and called MQTT in the startup method, but got the same result :(

I just noticed that the mqtt_library.lua at the URL you provided (http://git.eclipse.org/c/paho/org.eclipse.paho.mqtt.lua.git/tree/lua/mqtt_library.lua) does not have a module statement at its top.  I don't know why that is or what the ramifications are of that.

If you add this:

Code: [Select]
module("mqtt_library", package.seeall)
to the top of that mqtt_lbibrary.lua, does it help?  If this is also a dead-end, I'm probably out of advice (even bad advice :) ). 

But I'm very interested to see the results of your labor with MQTT.  I'm trying to get into some Arduino tinkering, and I think this is a good middleware to start with.

Regards,
watou

Offline 4integration

  • Sr. Newbie
  • *
  • Posts: 47
  • Karma: +0/-0
Re: Loading lua modules
« Reply #6 on: November 10, 2013, 05:08:51 am »
Hey,

I got it working! :D

But I have to admit I am on deep water and no fully satisfied :)

I read up a little on Lua modules and checked the Vera-style and MQTT-style. It seems that Vera implements the the old style (and deprecated) way for Lua 5.0 and early 5.1 and MQTT uses the new for Lua 5.1 and 5.2. Then suspected them somewhat being incompatible.

MQTT and Utility uses "objects" (or "classes"), not sure about the correct terminology since I new at Lua).
So last night I hacked mqtt_library and utility and removed these objects and made some other small changes required since I introduced some conflicts that way.

I have attached the hacked MQTT source so you can diff in full but to mention some changes

Added:
Code: [Select]
module("mqtt_library", package.seeall)
Removed:
Code: [Select]
local MQTT = {}and all references to MQTT in mqtt_library e.g.:

Code: [Select]
MQTT.Utility = require("utility")
...
function MQTT.client.create(
to
Code: [Select]
Utility = require("utility")
...
function client.create(

The variable conflict solved from:
Code: [Select]
MQTT.message = {}
MQTT.message.TYPE_RESERVED    = 0x00
...etc
to
Code: [Select]
messageType = {}
messageType.TYPE_RESERVED    = 0x00
...etc

For Utility I had to remove the "object" and it functions and make all local functions global instead of local.

Even if I got it working I don't like this solution since now I have my own fork of MQTT Lua libraries.

So some questions to you guys:
1.  Is it a fact that modules implemented with different Lua versions are incompatible?
2. Can this be solved without touching the MQTT library?
3. Is it possible to make use of the new module style in Vera? I made a quick test to use the new module style in Vera but seems to fail even if it's 5.1 (maybe it's the "early 5.1" as mentioned in the link above


Offline Thiemen

  • Jr. Member
  • **
  • Posts: 85
  • Karma: +0/-0
Re: Loading lua modules
« Reply #7 on: November 10, 2013, 05:32:57 am »
I hope some other people can assist you, since i think this can be an interesting add on to vera, especially with the "internet of things" coming.
For now i will follow this tread...

Maybe you should also rename the topic of this tread, so other people recognise it is about MQTT?
« Last Edit: November 10, 2013, 05:35:53 am by Thiemen »

Offline averell

  • Newbie
  • *
  • Posts: 19
  • Karma: +0/-0
Re: MQTT Client implementation
« Reply #8 on: May 19, 2014, 06:33:56 am »
I managed to get this MQTT integration working, based on the code you posted here. My idea is to try to build a plugin that allows me to control devices on the vera by sending messages on MQTT, and the other way atround, to offer a generic way of creating devices in vera that communicate via MQTT to the raspberry, some arduino's etc. that do the real work. This way I can use the vera sometimes as a master (I love my homewave app!), but sometimes as a slave when I want to control zwave devices from my other solutions.
Anybody else out there already playing with this? My next step is going to be to handle incoming messages.

Averell

Offline hek

  • Hero Member
  • *****
  • Posts: 714
  • Karma: +60/-1
Re: MQTT Client implementation
« Reply #9 on: May 19, 2014, 10:30:20 am »
Looking forward to see your results.

Offline aPL

  • Developers
  • Jr. Member
  • *****
  • Posts: 72
  • Karma: +0/-2
Re: MQTT Client implementation
« Reply #10 on: May 20, 2014, 08:19:19 am »
As a side-note I'm building a logging module in node for mqtt messages with storage in memory, memcache, redis or mysql.
working @ Vera / MiOS | anything stated is my own opinion and does not reflect the thoughts and opinions of my employer

Offline averell

  • Newbie
  • *
  • Posts: 19
  • Karma: +0/-0
Re: MQTT Client implementation
« Reply #11 on: May 21, 2014, 08:59:17 am »
So the good thing is that I can indeed react to incoming messages. The bad thing is, it's not working based on a push, but I need to pull MQTT with the mqtt_client:handler("") function from the MQTT library. I'm used to working with node.js where the callback function get's invoked whenever a messages comes in, here I need to tell the MQTT client to go check, only then it calls the callback function.
For building devices that just need to be updated with some information like temperature etc. it is quite ok, I can call the client:handler every few seconds or so, and update the device status. For turning on devices, ideally I would see my very react a bit faster, but I wonder about the overhead of calling this function to often.
I will play a bit with this to see how fast I can make this work without causing too much problems, every second or so might work out ok, or I might go for a solution where I limit the direct MQTT connectivity to updates to devices (incoming) and switching external stuff on (outgoing), and directly swithing on devices on the vera via http, letting my raspbberry translate mqtt commands to HTTP. Unsatisfactory though it may be...

Av.

Offline xAPPO

  • Jr. Member
  • **
  • Posts: 94
  • Karma: +6/-7
    • xAP Automation
Re: MQTT Client implementation
« Reply #12 on: May 23, 2014, 06:37:41 am »
Watching this topic with great interest. MQTT support would be a really useful addition to Vera.
xAP, Fibaro HC2, Vera 3 Lite Edge Plus , SmartThings, HomeVision, Home Automation Hub, Hue, LIFX, C-Bus, Sonos, Barionet, Crestron, AMX, Loxone, Harmony, HouseBot, HomeSeer, Indigo, MQTT, IFTTT, IOLogix, 1-wire, IDRATek.

Offline 4integration

  • Sr. Newbie
  • *
  • Posts: 47
  • Karma: +0/-0
Re: MQTT Client implementation
« Reply #13 on: May 24, 2014, 05:06:44 am »
I have a MQTT plugin running that monitor variables and fire off MQTT messages. The configuration is pretty "raw" though but doesn't change so often

Offline averell

  • Newbie
  • *
  • Posts: 19
  • Karma: +0/-0
Re: MQTT Client implementation
« Reply #14 on: May 25, 2014, 04:37:15 am »
Very interested to see what you did 4integration, as part of what I want to do, I also want to publish out all variables. I didn't get far due to travel, but I got child devices to send messages, and have temperature sensors publish messages over MQTT that set the device current temperature on the vera. Now thinking about the architecture, naming, making it hardcoded or flexible, etc. I will probably end up with a table with some hardcoded stuff that suits my needs. Some of the knowledge I will keep outside of the vera anyway, the whole idea is to keep some of the rules etc. away from the vera.