Author Topic: ALTUI & DataStorage Providers  (Read 6750 times)

Offline amg0

  • Moderator
  • Sr. Hero Member
  • *****
  • Posts: 2714
  • Karma: +163/-8
Re: ALTUI & DataStorage Providers
« Reply #15 on: February 29, 2016, 09:14:36 am »
Isn't that what the newJsonParameters variable is for?

When registering DataYours (see DataYours - a Data Storage Provider for AltUI)  I use the following Lua code:
Code: [Select]
local newJsonParameters = {
    {
        default = "unknown",
        key = "target",
        label = "Metric Name",
        type = "text"
      },{
        default = "/data_request?id=lr_render&target={0}&hideLegend=true&height=250&from=-y",
        key = "graphicurl",
        label = "Graphic Url",
        type = "url"
      }
    }
  local arguments = {
    newName = "datayours",
    newUrl = "http://127.0.0.1:3480/data_request?id=lr_DataWatcherRelay&target={0}",
    newJsonParameters = json.encode (newJsonParameters),
  }

  luup.call_action ("urn:upnp-org:serviceId:altui1", "RegisterDataProvider", arguments, AltUI)

When selecting a variable for storage through the AltUI interface, the "Metric Name" label parameter appears on the menu to be filled in with the desired name.  In my case, I use this to indicate the name of the file to be stored and the maximum storage time of the archive.  But it could be whatever you like for your own provider code.

that is correct. I was wondering if he wanted something like the device's name to be added automatically to the list of parameters of the callback url,  independently of the provider specific parameters that are declared and which the user has to fill in

Offline rmitsos

  • Sr. Newbie
  • *
  • Posts: 44
  • Karma: +1/-1
Re: ALTUI & DataStorage Providers
« Reply #16 on: September 09, 2016, 07:05:58 am »
Hi

Is there any possibility that we can utilize system variables for monitoring basic performance values like CPU, HardDisk, Logins, etc ?

Thank you

Offline amg0

  • Moderator
  • Sr. Hero Member
  • *****
  • Posts: 2714
  • Karma: +163/-8
Re: ALTUI & DataStorage Providers
« Reply #17 on: September 10, 2016, 01:10:56 pm »
Hi

Is there any possibility that we can utilize system variables for monitoring basic performance values like CPU, HardDisk, Logins, etc ?

Thank you

you could install such a plugin ( I think there are a few out there ) then ALTUI will allow you to send these devices variables to your storage

Offline rmitsos

  • Sr. Newbie
  • *
  • Posts: 44
  • Karma: +1/-1
Re: ALTUI & DataStorage Providers
« Reply #18 on: September 13, 2016, 11:44:55 am »
Yeap, I have installed "System Monitor" which is the only I found about that. Unfortunately it provides information only about cpu, memory but not anything else like disk, users, etc.

Thanx for replying


Offline sle118

  • Sr. Newbie
  • *
  • Posts: 39
  • Karma: +4/-0
Re: ALTUI & DataStorage Providers
« Reply #19 on: October 31, 2016, 03:35:34 pm »
I've been trying to setup a local instance of emoncms and I'm scratching my head on setting it up as a valid data provider.

First, I have created a copy of function sendValueToStorage_emoncms to sendValueToStorage_Local_emoncms, changed the url to point to the local instance, and defined it in the startup lua script.

Next, from the examples in this thread, I ran the following code in the "test lua"
Code: [Select]
local json_loc      = require "openLuup.json"
local newJsonParameters =  {
    callback = "sendValueToStorage_Local_emoncms",
    parameters = {{
        default = 1,
        key = "nodeid",
        label = "Node ID",
        type = "number"
      },{
        key = "feedid",
        label = "Feed ID",
        type = "number"
      },{
        key = "999999999222222222233333334444474",
        label = "Input Key name",
        type = "text"
      },{
        key = "999999999222222222233333334444476",
        label = "Read/Write API Key",
        type = "text"
      },{
        default = "http://127.0.0.1/emoncms/vis/editrealtime?feedid={1}&embed=1&apikey={3}",
        ifheight = 460,
        key = "graphicurl",
        label = "Graphic Url",
        type = "url"
      }},
    url = ""
  }
  local arguments = {
    newName = "emoncms_local",
    newUrl = "",
    newJsonParameters = json_loc.encode (newJsonParameters),
  }

  luup.call_action ("urn:upnp-org:serviceId:altui1", "RegisterDataProvider", arguments, AltUI)

I was hoping to have a new data provider template added with all the defaults (and the keys).  Although I now see the new data provider, I am simply unable to have the api keys "stick" in place... 

I have since updated the code above. The keys had to be placed in the "key" elements of the structure; doing this resolved my initial issue.  The next thing I found is that ALTUI does not save the callback parameters in the json. I was hoping a "set and forget" where I would run the code and let ALTUI restore the values when the luup engine is restarted, but that didn't work as expected. The reason seems to be that the callback and callback names are decoupled in a separate lookup which gets constructed only when they are fully registered.

So I forked the code and added some logic to get an actual function added to the callback table if the function is found in the global table. Then I created a new pull request after testing it successfully in my own system.
https://github.com/amg0/ALTUI/pull/2




« Last Edit: November 01, 2016, 10:48:00 am by sle118 »

Offline amg0

  • Moderator
  • Sr. Hero Member
  • *****
  • Posts: 2714
  • Karma: +163/-8
Re: ALTUI & DataStorage Providers
« Reply #20 on: November 01, 2016, 11:44:26 am »
I've been trying to setup a local instance of emoncms and I'm scratching my head on setting it up as a valid data provider.

First, I have created a copy of function sendValueToStorage_emoncms to sendValueToStorage_Local_emoncms, changed the url to point to the local instance, and defined it in the startup lua script.

Next, from the examples in this thread, I ran the following code in the "test lua"
Code: [Select]
local json_loc      = require "openLuup.json"
local newJsonParameters =  {
    callback = "sendValueToStorage_Local_emoncms",
    parameters = {{
        default = 1,
        key = "nodeid",
        label = "Node ID",
        type = "number"
      },{
        key = "feedid",
        label = "Feed ID",
        type = "number"
      },{
        key = "999999999222222222233333334444474",
        label = "Input Key name",
        type = "text"
      },{
        key = "999999999222222222233333334444476",
        label = "Read/Write API Key",
        type = "text"
      },{
        default = "http://127.0.0.1/emoncms/vis/editrealtime?feedid={1}&embed=1&apikey={3}",
        ifheight = 460,
        key = "graphicurl",
        label = "Graphic Url",
        type = "url"
      }},
    url = ""
  }
  local arguments = {
    newName = "emoncms_local",
    newUrl = "",
    newJsonParameters = json_loc.encode (newJsonParameters),
  }

  luup.call_action ("urn:upnp-org:serviceId:altui1", "RegisterDataProvider", arguments, AltUI)

I was hoping to have a new data provider template added with all the defaults (and the keys).  Although I now see the new data provider, I am simply unable to have the api keys "stick" in place... 

I have since updated the code above. The keys had to be placed in the "key" elements of the structure; doing this resolved my initial issue.  The next thing I found is that ALTUI does not save the callback parameters in the json. I was hoping a "set and forget" where I would run the code and let ALTUI restore the values when the luup engine is restarted, but that didn't work as expected. The reason seems to be that the callback and callback names are decoupled in a separate lookup which gets constructed only when they are fully registered.

So I forked the code and added some logic to get an actual function added to the callback table if the function is found in the global table. Then I created a new pull request after testing it successfully in my own system.
https://github.com/amg0/ALTUI/pull/2

in fact I did not support calling directly another LUA function which comes from a code I do not know ( cannot trust ). so only ALTUI internally calls a function to push to thingspeak or emoncms,  but externally registered data providers using the UPNP action works exclusively with the url mecanism.  The data provider must provide a url ( like a handler url if your data provider is a lua device ) that ALTUI will call with a wget() passing in argument all the data providers provided parameters in the json structure as well as lul_device lul_service lul_variable old new lastupdate values

your code is working fine but a) it poses a little supportability threat to me as ALTUI would call unknown code in its own lua space and b) I am not sure I understand fully the part of recursively searching inside the _G table, can you please explain that part a little bit ?

Online akbooer

  • Master Member
  • *******
  • Posts: 5472
  • Karma: +236/-69
  • "Less is more"
Re: ALTUI & DataStorage Providers
« Reply #21 on: November 01, 2016, 12:48:03 pm »
I don't follow this exactly, either, and not sure I see the need.  The Data Storage Provider mechanism for additional providers works as advertised (the DataYours plugin uses it) and, under openLuup (which you seem to be) the HTTP request to a localhost get routed directly to the local plugin, so it's actually very efficient (or, at least, not inefficient.)

If, for some reason, this doesn't work with your local emoncms instance, you could, nevertheless, use it with a small piece of code that acts as a proxy to your local instance.
3x Vera Lite-UI5/Edge-UI7, 25x Fibaro, 23x TKB, 9x MiniMote, 2x NorthQ Power, 2x Netatmo, 1x Foscam FI9831P.
Razberry, MySensors Arduino, HomeWave, AltUI, DataYours, openLuup, ZWay, ZeroBrane Studio.

Offline sle118

  • Sr. Newbie
  • *
  • Posts: 39
  • Karma: +4/-0
Re: ALTUI & DataStorage Providers
« Reply #22 on: November 01, 2016, 01:58:04 pm »
I don't follow this exactly, either, and not sure I see the need.
I realize my post wasn't really clear.

I setup a local openLuup to do what others are doing: offloading plugin processing, etc on a more powerful box (thanks for your efforts!). Then I wanted to get a local instance of emoncms as I like reducing reliance on external services for this type of things. Within ALTUI, there is already a default provider which was created for emoncms, but it is actually pointing to the the outside world and I wanted to :
  • Point to local emoncms instance
  • Default API Keys when "pushing to" data provider

In order to achieve this, I did the following
  • Copy the data provider "push" function and change it slightly
  • Define that new function in the lua startup section
  • Bind this new function to a data provider

I didn't want to write a plugin for this, I have never tried and have the belief that it's very complicated (probably not that much but anyhow). Everything seemed to be just fine in my code and implementation until I realized that ALTUI's implementation of the provider uses a mechanism to store and retrieve providers. The idea is that when the providers are loaded from the existing json, callback functions are not loaded in the callback lookup table and therefore these callbacks need to be re-registered at every start by code rather than being reloaded from save.

I'm going to review my pull request and likely come back with something along these lines
Code: [Select]
local function _loadDataProviders()
local str = luup.variable_get(ALTUI_SERVICE, "DataStorageProviders",  lul_device) or "{}"
DataProviders = json.decode(str)
for k,v in pairs (DataProviders) do
if (v["callback"] ~= nil)
local callback_fn = table_search(_G,DataProviders[provider]["callback"],"",0)
if callback_fn ~= nil then
DataProvidersCallbacks[DataProviders[provider]["callback"]] = callback_fn
end
end
end
end

Offline sle118

  • Sr. Newbie
  • *
  • Posts: 39
  • Karma: +4/-0
Re: ALTUI & DataStorage Providers
« Reply #23 on: November 01, 2016, 03:24:33 pm »
I ended up closing my pull request and opening a new one, which has an enhanced implementation of "_loadDataProviders", which is where providers are loaded. If the PR is merged, ALTUI will be able to grab data providers defined in the lua startup script, which was my original intent.

see PR
https://github.com/amg0/ALTUI/pull/3

Offline amg0

  • Moderator
  • Sr. Hero Member
  • *****
  • Posts: 2714
  • Karma: +163/-8
Re: ALTUI & DataStorage Providers
« Reply #24 on: November 01, 2016, 07:00:45 pm »
Hello Sle118
thanks for this proposal,  I am going to look at it to consider an integration

Question : where did you test this ?  on openluup only or also on VERA  ?  thx

Offline fullmoonguru

  • Full Member
  • ***
  • Posts: 170
  • Karma: +0/-1
Re: ALTUI & DataStorage Providers
« Reply #25 on: November 25, 2016, 04:00:40 pm »
I would appreciate some help with this.  I need really basic instructions.  I created an account at emoncms.org and there I can see a write API key & a read API key. 

If I go to my AL HEM in AltUI I go to variables and click the icon for "push to".  I check the box to enable pushing data to emoncms and I see the 5 things needing input. 

The first is Node ID and that's the node ID from my HEM that is listed as altid in Vera UI7 right?

The second is the Feed ID.  It wants a number here.  A assume that's whatever number I want to have emon use right?

The third one is Input Key Name.  Is this just what I want to name it or is it urn:schemas-micasaverde-com:device:PowerMeter:1 as listed for device type in Vera, or something else?

The next one is Read/Write API.  The thing is, I have one for each on my emoncms account page so?

Finally the Graphic Url.  It has this showing: http://emoncms.org/vis/editrealtime?feedid={1}&embed=1&apikey={3}.  I don't know if that's set up correctly as-is or if I'm suppost to edit it with my API key or what?

Offline alekseyzp

  • Newbie
  • *
  • Posts: 11
  • Karma: +0/-0
Re: ALTUI & DataStorage Providers
« Reply #26 on: December 01, 2016, 03:39:40 am »
Hi, amg0.
I need help.
I'm only a week, I bought VeraEdge and only understand it. I installed your plugin ALTUI and trying to figure out how to send data to the emoncms. Emoncms I have installed locally on my server Synology.

Now I have installed on Synology: OpenRemote application and AeonLab Zwave stick S2 and different zwave devices. Data sent from OpenRemote to Emoncms in this way:

Code: [Select]
</bean>

  <bean id = "rrd4jLogger" class = "org.openremote.controller.statuscache.rrd4j.Rrd4jDataLogger" />
  <bean id = "emonCMSLogger" class = "org.openremote.controller.statuscache.emoncms.EmonDataLogger" >
    <property name="emonURL" value="http://192.168.1.7/emoncms/input/post.json?&amp;apikey=5f51ed68e223863187f53820f6******&amp;node=20&amp;json=" />
    <property name="sensors">
      <list>
      <value>NET_TEMPERATURE_SENSOR</value>
        <value>NET_PRESSURE_SENSOR</value>
<value>NET_HUMIDITY_SENSOR</value>
        <value>FIBARO_S_SMOKE_AIR_TEMPERATURE_SENSOR</value>
<value>FIBARO_S_MOTION_AIR_TEMPERATURE_SENSOR</value>
        <value>CH_TSTAT_AIR_TEMPERATURE_SENSOR</value>
<value>AEO_HEM1_METER_SCALE_V_SENSOR</value>
        <value>AEO_HEM1_METER_SKALE_A_SENSOR</value>
<value>AEO_HEM1_METER_SKALE_W_SENSOR</value>
        <value>AEO_HEM1_METER_SKALE_KWH_SENSOR</value>
<value>NET_WIND_DIR_DEGREE_SENSOR</value>
<value>NET_WIND_SPEED_SENSOR</value>
<value>CH_AC_TSTAT_POWER_SCALE_WATT_SENSOR</value>
<value>CH_AC_TSTAT_POWER_SCALE_WATT_OUT_SENSOR</value>
      </list>
    </property>
  </bean>

and resultat on screenshot.

I'm going to all the zwave devices to reconnect from AeonLab Zwave stick S2 to VeraEdge. Please tell me, how do I properly set up to send data from VeraEdge in a locally installed Emoncms? It is possible to customize to the send on http://192.168.1.7/emoncms    ?
Thank.

« Last Edit: December 01, 2016, 03:46:23 am by alekseyzp »

Offline amg0

  • Moderator
  • Sr. Hero Member
  • *****
  • Posts: 2714
  • Karma: +163/-8
Re: ALTUI & DataStorage Providers
« Reply #27 on: December 05, 2016, 03:53:51 am »
1) on the device variable dialogbox, click on the histogram icon to open the push config dialog
2) enter parameters ( nodeid, feedid, inputkey, readwritekey ) , you can leave url as default. 
3) do not forget to click again the histogram bar to save the changes
4) at every variable change, ALTUI will call http://emoncms.org/input/post.json?node=<nodeid>&json={<inputkey>:<value>}&apikey=<readwritekey>
5) in variable watch page or in device variable page once you click the histogram you will be able to see the bar chart as rendered by emoncms

the url parameter can later on be customized to pass in your preferences in terms of zooming, period etc.


NOTE this is for internet emoncms.  for local one there is no solution yet, you probably need to go through the dynamic registration and write a data provider plugin , until I look at this to see if I could offer another way. there are also some other data provider that akboer has written for instance using his Datayours solution I think. I have put a pointer on msg 1 of this thread to Datayours thread


« Last Edit: December 05, 2016, 03:55:51 am by amg0 »

Offline alekseyzp

  • Newbie
  • *
  • Posts: 11
  • Karma: +0/-0
Re: ALTUI & DataStorage Providers
« Reply #28 on: December 05, 2016, 06:07:51 am »
Thanks for your reply.

And if in your plugin jast change path http://emoncms.org at http://192.168.1.7/emoncms?  I have an account on emonsms.org, but would like to use locally, as it is now with OpenRemote.

Online akbooer

  • Master Member
  • *******
  • Posts: 5472
  • Karma: +236/-69
  • "Less is more"
Re: ALTUI & DataStorage Providers
« Reply #29 on: December 05, 2016, 07:38:30 am »
Can you not achieve this in Startup Lua by registering the URL that you require?  You'll probably just need to name the storage provider differently from the pre-installed emoncms one.
3x Vera Lite-UI5/Edge-UI7, 25x Fibaro, 23x TKB, 9x MiniMote, 2x NorthQ Power, 2x Netatmo, 1x Foscam FI9831P.
Razberry, MySensors Arduino, HomeWave, AltUI, DataYours, openLuup, ZWay, ZeroBrane Studio.