Recent Posts

Pages: 1 ... 8 9 [10]
I've several remote DY on Vera system sending data to a centralized Openluup/DY on a RPI and a web application to plot,graph,analyze this data.

Yes, I recall.

With the new Openluup can I continue to use Datayours ?

Yes, you can.

Can I migrate smoothly to Historian and can I maintain the storage-schemas and storage-aggregations of the variables archived in whisper databases ? 

Yes, see my previous answer to @bruring here.  You could just copy and rename files to the history/ directory.  Before doing that, you could just try Data Historian with its default schemas, and run it in parallel with DataYours.

Finally can I preserve the actual funtionality of my system ?
Yes, for sure.  It just gets better with time.  I try not to break anything that is already working.
In the meantime I've been playing around making sure file permissions are correct

Good plan.  I think this is your problem.

My DataYours CONFIG_DIR is /www/ and there are no files there.

Actually, THIS is really your problem, but I think it's happened because of permissions.  Your .conf configuration files should be stored here.  It was a dreadful choice for a default location, but this was made in my Vera days before openLuup.  I suggest you change this to somewhere under cmh-ludl (perhaps cmh-ludl/www/) and try again.

As it stands, the reloaded system has no list of variables to watch, so nothing gets updated in the absence of a DataWatcher.conf file.
dataMine Plugin / Re: Datayours - where's my data?
« Last post by akbooer on August 11, 2018, 01:12:35 pm »

Default value for CONFIG_DIR (DataYours device variable) is /www (a bad choice, in hindsight.)
Hi akbooer,

I've several remote DY on Vera system sending data to a centralized Openluup/DY on a RPI and a web application to plot,graph,analyze this data.

With the new Openluup can I continue to use Datayours ? Can I migrate smoothly to Historian and can I maintain the storage-schemas and storage-aggregations of the variables archived in whisper databases ? 

Finally can I preserve the actual funtionality of my system ?


Lighting & Load Control / Re: monoprice Zwave RGB Smart bulb
« Last post by fwsfish on August 11, 2018, 12:12:05 pm »
I bought eight of them. Four ground level and two for each porch level. They work great. I set scenes where each floor is different ie red-white-blue  or green for St Patties day etc. My only complaint is changing the colors is tedious. In order to get each bulb exactly the same you have to use your computer (phone and ipad much less precision) ......find the color you like.... and go into the variables screen to copy the numerical value. You must then copy that value to each other device as current color and target color. Too bad there is not a way to simply choose basic colors like "warm white"-"red"- "blue" etc.
Thanks for the detailed answer, very much appreciated! It answers a lot of questions! Seems I had a weird workaround to label the variables in Grafana... :-)

In the meantime I've been playing around making sure file permissions are correct, going through the logs, etc. but couldn't find anything. I only found that all device .wsp files don't get updated, but the memory/cpu stats etc files do get updated after a Luup reload. Weird.

I must admit I forgot about the configure option from the DataYours dashboard, found it out while trying to answer your last question, thanks! This really helps troubleshooting. With the information you provided before I'm going to migrate to Historian, but now I feel that I'm so close to fixing this problem that I might do that first so at least I have a running system now. I will let you know how the migration went.

To answer your questions:

My DataYours CONFIG_DIR is /www/ and there are no files there.

This is what I found out before and after the LUUP reload:

Before: DataWatcher Daemon has sent 64225 messages through UDP
After: 0 messages

Before: DataWatcher shows a big array of watched devices.
After: array is empty.

Before: DataCache has a big array of watched devices in tally
After: no dives in tally

Before & after: metric list shows all watched devices and storage schemas.

So it seems like DataWatcher does not receive any value changes from openLuup? If unwatch and then re-watch variables, they still don't get updated.

I've atttached the before and after config files.
USB RFXtrx Transceiver Plugin / Another plugin update
« Last post by tinman on August 11, 2018, 11:59:01 am »
Added a new state variable: IPPort. This can be used by those who want to connect their RFXtrx transceiver using wifi adaptor. It can be set to the desired IP port. This setting does not appear in the RFXtrx settings dialog since it will probably not be used by many. When using the normal USB connection, setting the IPPort value will have no effect.
Added the signal strength to the data shown in the Managed Devices tab. Also removed signal strength and battery state from the Temperature and Humidity page.
Added ability to decode messages from Kangtai and Cotech devices. These will be seen as door sensors.
Lots of code changes to improve readability by implementing classes. Improved use of binary variables.
The attached zipfile contains all the files but only 4 are changed:
dataMine Plugin / Re: Datayours - where's my data?
« Last post by Chelseagroup on August 11, 2018, 11:50:30 am »
Where are the graph controls stored?
I need to change one and cannot seem to get the change to take when I save it with the same name as the existing one.
These questions put you into the "advanced user" league!

Would love to migrate to historian!
I see the structure is a bit different from DataYours. Could you point me in the right direction for the following:

You mean the metric naming hierarchy?  Yes, for good reason.  I feel the need to justify this.  There are, in fact, three naming conventions that you may run into when using Data Historian:
  • Standard Luup naming - full serviceId, variable name, LOCAL device number
  • Graphite API naming (as seen by Grafana) - nodeName, REMOTE device number/name, short serviceID, variable name
  • Whisper archive files - nodeNumber, REMOTE device number, short serviceId, variable name
  • nodeNumber - 0 for openLuup otherwise PK_AccessPoint or remote Vera 
  • nodeName - the name of the associated VeraBridge, or "openLuup"
  • shortSid - final apha-numeric part of the full serviceId
  • shortDevName - device name with all non-alphanumerics removed

Convention (1) is inevitable for Luup access, as in luup.variable_...(), (2) seemed to be the most readable and understandable, (3) is locked to the remote machine ID.

This way, even if the order of the bridges is changed in openLuup, the files are the same.  If you have to swap out a Vera, then its PK_AccessCode will change and you'll have to rename files to continue using them, but the finder metrics names will not change (unless you rename the associated VeraBridge.)

Pattern-matches for schemas, finders, and cacheVariables() ALL use the API paths and wildcards described here:

- I've set up watched variables and retention resolution and aggregation manually per watched variable. Is there a way to customize this in Historian?

Yes, certainly.  There's two levels of definition/indirection, in order to maintain compatibility with DataYours and the original Graphite/Carbon system on which it is based.
  • There are two files which may be put in the historian/ data directory (whatever you do, don't alias or combine this with the DataYours directory): storage-schemas.conf, and storage-aggregation.conf.  In their absence, virtual defaults are used.  You can find them in the openLuup/virtualfilesystem.lua file.  These are the absolutely standard Graphite configuration files which define archive structure depending on the written metric filename.  However, for Data Historian, these rules are used in a different way...
  • ...using the archive_rules found in the openLuup/servertables.lua file.  These provide the name of which schema to use based on the LOCALdeviceNo.shortServiceId.variable name pattern used internally by Data Historian.
These rules are only used to create missing files.  As you seem already to have discovered, if you create a correctly-named file with any archive structure you like, then that file will be used for the matching variable.

I'm sorry this is complicated, but it's logical, and preserves existing conventions.

- I made symlinks in my Whisper DB folder to give everything more human readable names in Grafana (i.e. I've linked Humidity.Living.wsp to Vera-88800000.10391.urn^micasaverde-com^serviceId^HumiditySensor1.CurrentLevel.wsp, Temp.Basement.wsp to Vera-88800000.10461.urn^upnp-org^serviceId^TemperatureSensor1.CurrentTemperature.wsp). So in the Grafana dropdown query menu's, I can choose Temperature, then Basement. This looks a lot nicer in the Legend as well. Not sure if this is the best way to do it though.

This ugly DataYours naming was something I abandoned, and, in fact, if you were to have used the AltUI Data Providers route (don't do that now) then you could choose arbitrary names according to your fancy.

If you've not defined the openLuup.Historian.DataYours attribute to point to your DataYours Whisper directory, then those metrics will all appear at the top level of the Graphite search tree.  If you do define it, then they will be moved to a sub-tree called DataYours.  (Equally, there is a DataMine attribute which will bring your dataMine database into the tree as well, but perhaps you have abandoned that already.)

The MODERN way to make your metrics legend and labels look nice is to use the additional functions at the trailing end of the Grafana data chooser: alias, aliasByNode, aliasByMetric.  Suppose you have a metric:

Code: [Select]

then following that metric selection with these functions will give:
  • alias ("Foo") - "Foo"
  • aliasByMetric - "CurrentTemperature"
  • aliasByNode (1) - "303_OutdoorTemperature"
  • aliasByNode (0,3) - "openLuup.CurrentTemperature"

Is there a way to migrate the DataYours WSP database files to Historian?

If I've managed to enable you to understand any of the above, then the answer is "yes".  You simply rename the old file to the one which Data Historian expects for that variable and move it into the historian/ directory.  WHATEVER YOU DO, do NOT do this when the system is running.  Whisper file headers are cached and changing the file mid-stream will completely trash it.

- On a side note: am I correct by saying that variables first get logged in memory, and written to disk every x minutes? So the Grafana charts only update every x minutes?

No, not quite.  Variables are written synchronously to the disk archive - this is one reason why you should really have it on a SSD and separately back up the database elsewhere from time to time.  It's written to be very fast, but obviously is at the mercy of I/O speed.  Grafana can be configured to update at any rate.  The memory cache saves variable update times to sub-millisecond precision.  Once they get written to disk they are quantised according to the retention rules.  If a Grafana graph time interval falls completely within the memory cache then you get times to maximum precision.

Let me know if there are any solutions for above, then I can migrate everything. If not, I would prefer to fix my DataYours whisper database, and migrate slowly to Historian. It's definitely the next step forward!

Just to curiosity, then, can you tell me what the value of the DataYours CONFIG_DIR is, whether there are actually any files there, and what the link from the DataYours device panel to DataWatcher config shows you?
Lighting & Load Control / Chacon 54660
« Last post by ledavidou on August 11, 2018, 10:05:35 am »
Hello everyone,

I own some Chacon 54660 and an RFXTRX433e

At the moment, the RFX is used to control my shutter rollers and it works perfectly.
I now would like to add my chacon devices but... i'm stuck.
On the rfx, "arc", "ac" and "lighting4" protocols are enabled.

Can anyone give me some help ?
Pages: 1 ... 8 9 [10]