Author Topic: LUA scripting - What should we do to improve it  (Read 2498 times)

Offline melih

  • Administrator
  • Full Member
  • *****
  • Posts: 117
  • Karma: +29/-2
LUA scripting - What should we do to improve it
« on: August 24, 2018, 09:14:08 pm »
So that developers can develop even better applications...

(Please note, we are aware of stability, bug fixes etc...you don't need to tell us about them here, just want to learn all the new and funky capabilities you want to see so that we can schedule them for you).

thanks
« Last Edit: August 27, 2018, 11:45:16 am by Sorin »

Offline rigpapa

  • Hero Member
  • *****
  • Posts: 674
  • Karma: +101/-1
Re: LUA scripting - What should we do to improve it
« Reply #1 on: August 24, 2018, 10:39:55 pm »
I listed most of these in the development environment thread, but to restate:

* Make sure all device, rooms, scene, service, etc. information is available through the API, without having to make HTTP requests to 127.0.0.1 and parse the JSON or XML we get back;

* There should be a high degree of parity between the Lua API and the JavaScript API; in theory, this may someday facilitate the choice of either as an implementation language for the core of a plugin (not just UI);

* Plugins should be segregated from each other, not lumped into a single directory; use subdirectories or packages;

* Create a persistent store that isn't user_data and is guaranteed (as much as anything can be) to be saved to non-volatile storage when written; state variables/user_data saved every six minutes don't cut it; if this has to be the filesystem, OK, but then give me a chroot jail for the directory where (only) my plugin lives for that;

* Enforce complete reentrancy in the plugin environment; in support of this, you'll need to make sure all callbacks pass in sufficient context for the code being called (for example, current variable watch callbacks do not receive the device number of the device handling the call, only the device number of the watched device--not at all the same! Job callback is called for every job/all devices, with no filtering and no context; timer/delay callbacks receive a string parameter, but why not device?); remove luup.device from the luup module;

* Having done the previous, it should then be easier to have modules truly shared (no more per-device copies in RAM of implementation code and each module it requires);

* Where functions are used as parameters, use a function type, not the string name of the function (prefer upvalues over globals);

* Fix require() so that we can use modern Lua module structure reliably;

* Extend the API to include functions for fast lookup/search of system objects: give me a list of every device in a room, or every SwitchPower1 device, child of a parent device, etc.; right now we have to traverse luup.devices;

* Delete a state variable without an HTTP request;

* Upgrade the included set of Lua libraries to include bit, lfs, etc.; many have made similar suggestions elsewhere;

* Upgrade the job processing facilities to allow any function to be submitted as a job (e.g. startjob( function, options ) ), allow waiting and running jobs to be stopped, and improve job status; it may be desirable to schedule jobs, in which case the delay/timer callbacks become redundant;

* Allow plugins to delay their startup as a dependency on other plugins, devices, or system features (e.g. don't start this plugin until Z-wave has completed startup);

* A more modern logging model, preferably with multiple, configurable destinations (don't clog Vera's log with plugin chatter); make it easier for users to get to logs and get them to plugin developers for forensics;

* Adopt the philosophy that browsers take toward JavaScript: to the greatest extent possible, scripts are not allowed to take down the app. Browsers go to great lengths to contain the JS environment so the browser stays running, only the page running the script dies. Lua should be the same: bad or dumb things that happen in the Lua environment may stop the device/integration/feature, but should never cause watchdog trips (luup.sleep()), deadlocks, core dumps, etc., nor should it create a conflict with another plugin or built-in facility that causes system instability.
Author of Reactor, DelayLight, SiteSensor, Rachio, Deus Ex Machina II, Intesis WMP Gateway, Auto Virtual Thermostat and VirtualSensor plugins. Vera Plus w/100+ Z-wave devices. Vera3 sandbox.

Offline therealdb

  • Full Member
  • ***
  • Posts: 162
  • Karma: +2/-0
  • Automate all the things!
Re: LUA scripting - What should we do to improve it
« Reply #2 on: August 25, 2018, 01:26:24 am »
To the very good list made by rigpapa, I want to add
- the ability to view debug/print via the ui and not only with ssh
- websockets support (both as server and as client)
Vera Edge EU, Fibaro FGRM 222 (12), Fibaro FGS 223 (20), Fibaro FGS 222 (5), Fibaro Universal Binary Sensor (2), Fibaro Plug (3), NeoCoolCam Door Sensor (3), NeoCoolCam PIR (2), Nest (3), Home Server running my own integrations, Harmony Hub, OpenSprinkler, Personal Weather Station, Sonoff TH & more

Offline reneboer

  • Hero Member
  • *****
  • Posts: 1381
  • Karma: +79/-30
Re: LUA scripting - What should we do to improve it
« Reply #3 on: August 27, 2018, 05:17:19 am »
Hi,

Many items mentioned other places, but a bit less prone the LUA/luup crashes and reloads.

Better (HTTP) communication options that rely less on native LUA support as LUA lacks a lot in this space (OAuth, SNI, Cookies, etc) . luup.inet.request is a start (once it works, and I'd like the return headers to be passed back).

But, mostly; be well documented! i now spend too much time in trial and error (mostly error).

Cheers Rene
2xVeraLite, VeraEdge, openLuup, ALTUI, 20 switches, 10 dimmers, 20 sensors, 10 scene controllers, 1 Harmony Hub, many plug-ins. Not enough time.

Offline markoe

  • Full Member
  • ***
  • Posts: 130
  • Karma: +4/-3
Re: LUA scripting - What should we do to improve it
« Reply #4 on: August 27, 2018, 09:25:23 am »
I have two items on my list

1. editor . Now I cut paste the Code from  notepad.
2. Debug option + real log to monitor the scrip.


Offline RV

  • Full Member
  • ***
  • Posts: 125
  • Karma: +0/-0
Re: LUA scripting - What should we do to improve it
« Reply #5 on: August 27, 2018, 10:15:30 am »
what they said... and, maybe some kind of template editor where you can input the devices, tasks, parameters, etc and it builds the correct code for you.

Offline Petra123

  • Newbie
  • *
  • Posts: 1
  • Karma: +0/-0
Re: LUA scripting - What should we do to improve it
« Reply #6 on: August 28, 2018, 10:30:39 am »
yes they can devolpe better aplications

Offline pit

  • Sr. Newbie
  • *
  • Posts: 43
  • Karma: +2/-0
Re: LUA scripting - What should we do to improve it
« Reply #7 on: September 12, 2018, 11:54:44 am »
For me the app luaTest is indispensable to test and debug lua code. Such a native tool with closer integration to vera would be helpfull.

The other helpfull tool ist eventWatcher for devices, watched variables and scene lists.
! It would be very helpfull to see the "unzipped" lua code in a table/listing of the scenes. !

luup.call_delay has only one variable. If you want to call any action (setTarget, runScen,...) delayed, you are forced to create separate global functions. This makes the code big and confusing.
It would be helpfull, to allow more variables for luup.call_delay. This  would create the opportunity to run luup.call_delay in more universal functions. (In the moment I do that with some variables put together to a string to call a universal global function and then split this string in the global function.)

I had the impression, that it's not good for the performace of vera to hold all global functions in the startupLua. So I created some modules and linked them per "require". A native option to do that would be helpfull.

I would very appreciate a native, flexible "logical" logging function in vera. The "technical" logfile is not very helpfull for this requirement (although the ElVira app improves a lot). These logical logfiles should be easy to retrieve and outside the vera box.
Reason: With growing complexity of my lua based processes I urgently needed a "logical" longtime-logging of scene- and lua processing (with actions and relevant variable content etc.). So I am able to watch and to debug the process chains and results.
For me I built it with an universal global function (inet.wget and a cgi-string for fileName, event texts, variable content etc.) and an universal php scripct to write all these eventlines into several logs on my webserver.

Offline jeubanks

  • Full Member
  • ***
  • Posts: 215
  • Karma: +10/-4
Re: LUA scripting - What should we do to improve it
« Reply #8 on: September 12, 2018, 12:03:22 pm »
The NUMBER 1 thing not mentioned yet.

Quality and Correct Documentation!  This includes usable/functioning examples.

What is available is either flat out wrong or is so old that it's no longer relevant thus again making it wrong.  Documentation, all of it should be readily available and not a hidden secret that we have to go through multiple levels of support to get a password protected copy of a PDF sent to us to know how to integrate properly.


Offline bogdanf

  • Steering Box Fanclub
  • Administrator
  • Full Member
  • *****
  • Posts: 222
  • Karma: +8/-1
  • Did you change the oil ?
Re: LUA scripting - What should we do to improve it
« Reply #9 on: September 13, 2018, 04:30:04 am »
For me the app luaTest is indispensable to test and debug lua code. Such a native tool with closer integration to vera would be helpfull.

The other helpfull tool ist eventWatcher for devices, watched variables and scene lists.
! It would be very helpfull to see the "unzipped" lua code in a table/listing of the scenes. !

luup.call_delay has only one variable. If you want to call any action (setTarget, runScen,...) delayed, you are forced to create separate global functions. This makes the code big and confusing.
It would be helpfull, to allow more variables for luup.call_delay. This  would create the opportunity to run luup.call_delay in more universal functions. (In the moment I do that with some variables put together to a string to call a universal global function and then split this string in the global function.)

I had the impression, that it's not good for the performace of vera to hold all global functions in the startupLua. So I created some modules and linked them per "require". A native option to do that would be helpfull.

I would very appreciate a native, flexible "logical" logging function in vera. The "technical" logfile is not very helpfull for this requirement (although the ElVira app improves a lot). These logical logfiles should be easy to retrieve and outside the vera box.
Reason: With growing complexity of my lua based processes I urgently needed a "logical" longtime-logging of scene- and lua processing (with actions and relevant variable content etc.). So I am able to watch and to debug the process chains and results.
For me I built it with an universal global function (inet.wget and a cgi-string for fileName, event texts, variable content etc.) and an universal php scripct to write all these eventlines into several logs on my webserver.

Thank you for your feedback. We have added your thoughts as feature requests in our internal system.

Offline bogdanf

  • Steering Box Fanclub
  • Administrator
  • Full Member
  • *****
  • Posts: 222
  • Karma: +8/-1
  • Did you change the oil ?
Re: LUA scripting - What should we do to improve it
« Reply #10 on: September 13, 2018, 04:32:29 am »
The NUMBER 1 thing not mentioned yet.

Quality and Correct Documentation!  This includes usable/functioning examples.

What is available is either flat out wrong or is so old that it's no longer relevant thus again making it wrong.  Documentation, all of it should be readily available and not a hidden secret that we have to go through multiple levels of support to get a password protected copy of a PDF sent to us to know how to integrate properly.

This is indeed a MUST. Noted down.

Offline a-lurker

  • Hero Member
  • *****
  • Posts: 859
  • Karma: +63/-8
Re: LUA scripting - What should we do to improve it
« Reply #11 on: September 13, 2018, 11:29:33 pm »
Need to have a look at the IO and how it's documented.

This tag should be removed; as per http://wiki.mios.com/index.php/Luup_Plugins_ByHand#.3CioPort.3E.

Code: [Select]
<ioPort>4998</ioPort>
Only the GlobalCache plugin uses it as far I can see.  It's totally unnecessary and just complicated matters. It seems that by using it you don't need to use the luup.io.open()  function at all (refer GC100 plugin in store). That in itself is not well documented. Here is an updated version of the plugin that eliminates it:  http://forum.micasaverde.com/index.php/topic,37268.msg320919.html#msg320919

What does this do and how should it be utilised - needs to be clearly documented:

Code: [Select]
luup.io.intercept()
How does it fit in with luup.io.read() & luup.io.write() and the <incoming> tag? The global <incoming> tag receives any RX'ed data that is not handled by lower nested <incoming> tags. Refer to  processIncomingGC100CatchAll() and processIncomingGC100Job() I_GC100.xml here http://forum.micasaverde.com/index.php/topic,37268.msg320919.html#msg320919

There must be an incoming queue and the data is handled in different ways based on the various actions set up - needs to be documented with a flow diagram.

The above also works with:   luup.job.setting and luup.job.set   which allows arbitrary key/value pairs to label returned data with jobs that invoked the data. Jobs in general are also so not well explained.

Also why is there no   luup.io.close()   function????

Offline timmy

  • Newbie
  • *
  • Posts: 6
  • Karma: +0/-0
Re: LUA scripting - What should we do to improve it
« Reply #12 on: September 19, 2018, 12:10:26 pm »
I may repeat what has already been said but I think it is important to add these functionalities:
Use subdirectories to separate the plugins, allow the plugins to delay their startup, and the possibility to see the debug


___________________________________________________________________________________
Minecraft Pocket Edition Google Play Services Counter-Strike
« Last Edit: September 23, 2018, 02:00:47 pm by timmy »

Offline akbooer

  • Master Member
  • *******
  • Posts: 6158
  • Karma: +275/-69
  • "Less is more"
Re: LUA scripting - What should we do to improve it
« Reply #13 on: September 19, 2018, 12:59:33 pm »
* Enforce complete reentrancy in the plugin environment; in support of this, you'll need to make sure all callbacks pass in sufficient context for the code being called (for example, current variable watch callbacks do not receive the device number of the device handling the call, only the device number of the watched device--not at all the same!

Another case of this is that scene code does not know the number of the scene in which it is running!  In line with some other callback functions, I used a variable called 'lul_scene', available during scene code execution, to indicate this in my openLuup implementation.

Yet another case is the variable_watch callback which really needs another parameter added to indicate the actual time (ideally with millisecond resolution) at which the variable was changed.

The other things mentioned by @rigpapa weren't at all bad, either!  You should be able to do anything with a luup call that you can do with an HTTP request.

3x Vera Lite-UI5/Edge-UI7, 25x Fibaro, 23x TKB, 9x MiniMote, 2x NorthQ Power, 2x Netatmo, 1x Foscam FI9831P, 9x Philips Hue,
Razberry, MySensors Arduino, HomeWave, AltUI, AltHue, DataYours, Grafana, openLuup, ZWay, ZeroBrane Studio.

Offline RitterIwan

  • Sr. Newbie
  • *
  • Posts: 36
  • Karma: +0/-0
Re: LUA scripting - What should we do to improve it
« Reply #14 on: September 26, 2018, 08:15:16 am »
Hi,

I need some kind of handler to react to data of certain command classes send by the device.
Currently I send data. But I cannot see / react on what's comming in.