GCAL3 - General Discussion

[Stuart]

V2.0 will be release in the marketplace shortly.

It's got a lot in it - mostly usability changes which will make life easier, especially with initial setup.  The core functionality is unchanged.  Unfortunately you will need to delete existing plugins and reinstall.  Sorry but it's just the nature of vera , given some of the changes that I made.

You can install it directly from the following link

http://<your vera IP>:3480/data_request?id=action&serviceId=urn:micasaverde-com:serviceId:HomeAutomationGateway1&action=CreatePlugin&PluginNum=7016&Version=30931


To upgrade to  V2 you will need to do the following:
1.  Take a note of the settings in your existing
  Advanced --> Variables tab.
2.  DELETE your existing plugins
3.  Install version 2.0 Note: If you get a "Lua Startup Failed" message, rebooting should clear the error.
4.  Set the Calendar ID using the Control Tab -  - wait a bit as vera will restart
5.  If you are using a credential file different to the default name (GCal3.json)
    set it using the Control Tab -  - wait a bit as vera will restart
4.  Set the appropriate variables in the Advanced --> Variables tab
6.  Change any scenes etc to use the new plugin number.
     (See especially the note below on JSON Parser)

UI Changes
Front Panel
Added the Check Button to the front panel
Fixed the non-operating front panel buttons in UI5

Control Tab
Added controls to set set the Calendar ID and Credentials file name.
It provides better control and error handling than prior versions
(which used the Advanced --> Variables tab)
Added a GetLog function.  This will create a log file for the plugin insstance.
When created - you can download the logfile from
Apps --> Develop Apps --> Luup files.  The log file listed near the top with a filename "<device #> - GCal3.log".
Download it and view it with a good text editor.  I use Notepad++.

Calendar Tab
Increased the size of the calendar display.
Changed the behavior:  The tab will display google calendars
in the normal way. If an iCalendar is used the tab will cause
the iCalendar file to download.

Advanced Tab
Control variables Removed from the Advanced Tab
A number of variables used to be displayed on the advanced tab.
They should never be manually changed by a user and they were
taking up unnecessary screen space.
They are now grouped into a single variable gc_Variables.

Functional Changes
iCalendar Improvements
There were problems with certain iCalendars due to some underlying lua modules.
Gcal now uses a better approach (curl is used instead of luasec for those interested).
Three forms of iCalendar url are recognized
See also the notes on using iCalendars.

Error Message Improvements
Some error messages to the GCal pannel were being 'lost'
because they were being over-written by messages higher up in the code.

Startup Improvements
Changed to startup logic.  Semaphore files are no longer used.
On first install a reboot is required (UI7).  After the reboot you
may have to press F5 for the UI to display (first time only).

JSON Parser
Some versions of vera came with a json parse (dkjson.lua module), some did not.
Prior versions of GCal forced the download of a parser (json.lua).
With this release: if the dkjson.lua module is present then json.lua
is not downloaded and if a json.lua exists - it is removed.
If you have scenes that rely on json.lua (and dkjson.lua exists on your vera)
then you will need to change any require("json.lua") to require("dkjson.lua)").
If dkjson is not on your vera then nothing changes.
This was done to save space on the vera unit.

[CudaNet]

Going to give this a go with openLuup/AltUI and report back....

[Stuart]

Going to give this a go with openLuup/AltUI and report back....

Hopefully it will be a pleasant experience.   The changes are mainly execution sequence, managing the calendar id and credential file setting, some fairly minor function tweaks, some UI re-arrangement and some new (simple) functionality.

What can go wrong ??

 

[CudaNet]

I have nothing but great things to say about GCal3 so I'm really looking forward to seeing what you've done here. SO, karma at you for your promptness to questions and attention to detail.

Going to give this a go with openLuup/AltUI and report back....

Hopefully it will be a pleasant experience.   The changes are mainly execution sequence, managing the calendar id and credential file setting, some fairly minor function tweaks, some UI re-arrangement and some new (simple) functionality.

What can go wrong ??

 

[CudaNet]

EXTREMELY low priority as this was just a litmus test for openLuup.. I loaded my /GCal3 (/etc/cmh-ludl/GCal3) folder which is 'ancient'. My distro is Debian Jessie 8 (Raspbian; Raspberry Pi3)  so it was absolutely expected for opkg anything to fail. openSSL is installed as is dkjson and pretty much everything else needed for plugin support.

Edit: Removed all files in /GCal3 with exception of the .json file (figured I, at a minimum needed that)...

Code: [Select]

2016-03-24 21:04:55.275   luup.variable_set:4: 4.urn:srs-com:serviceId:GCalIII.gc_NextEvent was: Restarting ... now: Initial Startup #hooks:0
2016-03-24 21:04:55.275   luup.variable_set:4: 4.urn:srs-com:serviceId:GCalIII.gc_NextEventTime was: Changing Calendar ID now:  #hooks:0
2016-03-24 21:04:55.281   luup_log:4: GCal3 device: 4 - Command /bin/ls /etc/cmh-ludl/GCal3/ returned 0
2016-03-24 21:04:55.287   luup_log:4: GCal3 device: 4 - Command /bin/ls /etc/cmh-ludl/GCal3/GCal3.sem returned 512
2016-03-24 21:04:55.294   luup_log:4: GCal3 device: 4 - Command /bin/ls /etc/cmh-ludl/GCal3/*.token returned 512
2016-03-24 21:04:55.300   luup_log:4: GCal3 device: 4 - Command /bin/ls /usr/lib/lua/googlejwt.sh returned 512
2016-03-24 21:04:55.306   luup_log:4: GCal3 device: 4 - Command /bin/ls /etc/cmh-ludl/D_GCal37.json.lzo returned 512
2016-03-24 21:04:55.311   luup_log:4: GCal3 device: 4 - Existing openssl version is: false
2016-03-24 21:04:55.312   luup_log:4: GCal3 device: 4 - Installing openssl
2016-03-24 21:04:55.315   luup_log:4: GCal3 device: 4 - Command /bin/opkg update && opkg install openssl-util returned 32512
2016-03-24 21:04:55.315   luup_log:4: GCal3 device: 4 - Fatal error could not install openssl
2016-03-24 21:04:55.315   luup_log:4: GCal3 device: 4 - Checking for DKJSON module at: /usr/lib/lua/dkjson.lua
2016-03-24 21:04:55.321   luup_log:4: GCal3 device: 4 - Command /bin/ls /usr/lib/lua/dkjson.lua returned 512
2016-03-24 21:04:55.322   luup_log:4: GCal3 device: 4 - Checking for JSON module at: /usr/lib/lua/json.lua
2016-03-24 21:04:55.328   luup_log:4: GCal3 device: 4 - Command /bin/ls /usr/lib/lua/json.lua returned 512
2016-03-24 21:04:55.328   luup_log:4: GCal3 device: 4 - Downloading /usr/lib/lua/json.lua
2016-03-24 21:04:55.328   luup_log:4: GCal3 device: 4 - Attempting to download http://code.mios.com/trac/mios_google_calendar_ii_plugin/raw-attachment/wiki/WikiStart/json.lua
2016-03-24 21:04:55.328   luup_log:4: GCal3 device: 4 - local function: os_command
2016-03-24 21:04:55.328   luup_log:4: GCal3 device: 4 - Command was: curl -ksL "http://code.mios.com/trac/mios_google_calendar_ii_plugin/raw-attachment/wiki/WikiStart/json.lua"
2016-03-24 21:04:56.527   luup_log:4: GCal3 device: 4 - local function: os_command
2016-03-24 21:04:56.527   luup_log:4: GCal3 device: 4 - Command was: curl -ksL -w "%{http_code} %{url_effective}\n" "http://code.mios.com/trac/mios_google_calendar_ii_plugin/raw-attachment/wiki/WikiStart/json.lua" -o /dev/null
2016-03-24 21:04:56.924   luup_log:4: GCal3 device: 4 - Writing file /usr/lib/lua/json.lua
2016-03-24 21:04:56.925   luup_log:4: GCal3 device: 4 - local function: writetoFile
2016-03-24 21:04:56.926   openLuup.context_switch::  ERROR: [string "[4] I_GCal3.xml"]:291: /usr/lib/lua/json.lua: No such file or directory
2016-03-24 21:04:56.926   openLuup.scheduler:: job aborted : [string "[4] I_GCal3.xml"]:291: /usr/lib/lua/json.lua: No such file or directory
2016-03-24 21:04:56.926   luup_log:3: ALTUI: startupDeferred, called on behalf of device:3


Code: [Select]

pi@raspberrypi:/etc/cmh-ludl/GCal3 $ ls -l
-rw-r--r-- 1 pi pi 1238 Nov 17  2014 GCal3.json


[Stuart]

@Cudenet

I'll need more of a clue as to what the issue is.  I've never tested on other than a stock vera unit -- so the directories and commands are all tailored around that.

If what is needed is some logical "or" statements to look in different directories then I can do that (and am happy to) if you can provide the locations (although this could be a never ending list depending on distro etc)

Similarly - the command used for looking for open ssl is "opkg list-installed | awk '/openssl-util/ {print $3}'"

Does that not work on your distro ?

I suppose I could search with something like "find -name etc etc" but that can take quite a while - especially on low power cpu's

[CudaNet]

Stuart,

Yes, unless I were running OpenWRT, opkg will fail. In my case, I'm running Debian which is apt/dpkg. It appears that the failed attempts package check & install eventually lead to the process stopping all together (which is to be expected). There are probably 10x more reasons not to change anything simply because your intent was to run on a Vera, not openLuup. Today I'm still running the original 1.x version you developed and it's running flawlessly so I'll continue to use that.

One of these days I'm going to talk you into giving openLuup a try. I even have turn-key images ready to load for the Raspberry Pi 2, 3 and Odroid XU4's. Unfortunately I'm not the best at coding (ok I'm dangerous at best) so Akbooer would be the best resource for anything openLuup related. Anyways, thanks again for everything and have great weekend !

@Cudenet

I'll need more of a clue as to what the issue is.  I've never tested on other than a stock vera unit -- so the directories and commands are all tailored around that.

If what is needed is some logical "or" statements to look in different directories then I can do that (and am happy to) if you can provide the locations (although this could be a never ending list depending on distro etc)

Similarly - the command used for looking for open ssl is "opkg list-installed | awk '/openssl-util/ {print $3}'"

Does that not work on your distro ?

I suppose I could search with something like "find -name etc etc" but that can take quite a while - especially on low power cpu's

[Stuart]

Let me take a look. I run Debian on my home server - so I'll see if a little additional logic can help out here ...  I'm thinking check to see if distribution is not OpenWRT then resort to time consuming searches .... 
One of the things I did in 2.0 was indeed to STOP the startup if "things" were not as expected since this made basic setup easier and errors more obvious.....

[CudaNet]

Yes, you could definitely check within the /etc/*release... I use this often and it's pretty safe across most all distros.

Code: [Select]

cat /etc/*release

DISTRIB_ID='OpenWrt'
DISTRIB_RELEASE='15.05'
DISTRIB_REVISION='r46767'
DISTRIB_CODENAME='chaos_calmer'
DISTRIB_TARGET='x86/64'
DISTRIB_DESCRIPTION='OpenWrt Chaos Calmer 15.05'
DISTRIB_TAINTS=''


Let me take a look. I run Debian on my home server - so I'll see if a little additional logic can help out here ...  I'm thinking check to see if distribution is not OpenWRT then resort to time consuming searches .... 
One of the things I did in 2.0 was indeed to STOP the startup if "things" were not as expected since this made basic setup easier and errors more obvious.....

[CudaNet]

Whoa, I was wiped out last night... I re-read this and figured I'd see if there was a method that could be used strictly within Lua.

Code: [Select]

function loadrequire(module)
    local function requiref(module)
        require(module)
    end
    res = pcall(requiref,module)
    if      not(res) then
            print("((Module))::Cannot locate the following module [" .. module .. "], halting.)")
    else    print("((Module))::Located [" .. module .. "] module, proceeding.)")
    end
end

loadrequire('dkjson')
loadrequire('ssl')

So checking to see what you're running on can steer how it checks or use this regardless. Basically if it's not OpenWrt then just write a log entry (keeping it simple) describing which modules aren't loaded on the system and halt.

[Stuart]

@ CudaNet

Try this.  All I have done is:  if the distro is not OpenWrt -- bypass the check for dkjson.lua / json.lua / openssl.
 Since we know these programs  are already on your system - this should allow your test to proceed.

[Stuart]

Whoa, I was wiped out last night... I re-read this and figured I'd see if there was a method that could be used strictly within Lua.

Code: [Select]

function loadrequire(module)
    local function requiref(module)
        require(module)
    end
    res = pcall(requiref,module)
    if      not(res) then
            print("((Module))::Cannot locate the following module [" .. module .. "], halting.)")
    else    print("((Module))::Located [" .. module .. "] module, proceeding.)")
    end
end

loadrequire('dkjson')
loadrequire('ssl')

So checking to see what you're running on can steer how it checks or use this regardless. Basically if it's not OpenWrt then just write a log entry (keeping it simple) describing which modules aren't loaded on the system and halt.

I'll take a closer look in a few days -- I like this approach for testing to see what's there (independent of path).
In the meantime - I took the opposite approach to stopping ... i.e. bypass and keep going.  That way you can see if there are any additional "gotcha's" with openLuup/AltUI

[CudaNet]

Just got back so I'll load this and let you know...

@ CudaNet

Try this.  All I have done is:  if the distro is not OpenWrt -- bypass the check for dkjson.lua / json.lua / openssl.
 Since we know these programs  are already on your system - this should allow your test to proceed.

[CudaNet]

Sweet... I'll run some additional tests and report back if I discover any issues.

Code: [Select]

2016-03-26 12:28:49.982   luup_log:4: GCal3 device: 4 - local function: GCalStartup
2016-03-26 12:28:49.982   luup.variable_set:4: 4.urn:srs-com:serviceId:GCalIII.gc_NextEvent was: Restarting ... now: Initial Startup #hooks:0
2016-03-26 12:28:49.982   luup.variable_set:4: 4.urn:srs-com:serviceId:GCalIII.gc_NextEventTime was: Changing Calendar ID now:  #hooks:0
2016-03-26 12:28:49.989   luup_log:4: GCal3 device: 4 - Command /bin/ls /etc/cmh-ludl/GCal3/ returned 0
2016-03-26 12:28:49.995   luup_log:4: GCal3 device: 4 - Command /bin/ls /etc/cmh-ludl/GCal3/GCal3.sem returned 512
2016-03-26 12:28:50.002   luup_log:4: GCal3 device: 4 - Command /bin/ls /etc/cmh-ludl/GCal3/*.token returned 512
2016-03-26 12:28:50.009   luup_log:4: GCal3 device: 4 - Command /bin/ls /usr/lib/lua/googlejwt.sh returned 512
2016-03-26 12:28:50.015   luup_log:4: GCal3 device: 4 - Command /bin/ls /etc/cmh-ludl/D_GCal37.json.lzo returned 512
2016-03-26 12:28:50.022   luup_log:4: GCal3 device: 4 - Command cat /etc/*release | grep OpenWrt returned 256
2016-03-26 12:28:50.022   luup_log:4: GCal3 device: 4 - OpenWrt test returned: 256
2016-03-26 12:28:50.022   luup_log:4: GCal3 device: 4 - Not a standard Vera Distro
2016-03-26 12:28:50.022   luup_log:4: GCal3 device: 4 - local function: makejson
2016-03-26 12:28:50.024   luup.variable_set:4: 4.urn:srs-com:serviceId:GCalIII.gc_Variables was: [{"NextCheck":"2016-03-26T12:28:04","TrippedID":"","CredentialFile":"GCal3.json","Version":" V2.0.1:","addCalendar":"true","LastCheck":"2016-03-26T12:28:04","gCal":"true","UI7Check":"true","CalendarID":"{removed}"}] now: [{"NextCheck":"2016-03-26T12:28:04","TrippedID":"","CredentialFile":"GCal3.json","Version":" V2.0.1:","addCalendar":"true","LastCheck":"2016-03-26T12:28:04","gCal":"true","UI7Check":"true","CalendarID":"{removed}"}] #hooks:0
2016-03-26 12:28:50.024   luup_log:4: GCal3 device: 4 - local function: setupVariables
2016-03-26 12:28:50.026   luup_log:4: GCal3 device: 4 - local function: makejson
2016-03-26 12:28:50.026   luup.variable_set:4: 4.urn:srs-com:serviceId:GCalIII.gc_Variables was: [{"NextCheck":"2016-03-26T12:28:04","TrippedID":"","CredentialFile":"GCal3.json","Version":" V2.0.1:","addCalendar":"true","LastCheck":"2016-03-26T12:28:04","gCal":"true","UI7Check":"true","CalendarID":"{removed}"}] now: [{"NextCheck":"2016-03-26T12:28:04","TrippedID":"","CredentialFile":"GCal3.json","Version":" V2.0.1:","addCalendar":"true","LastCheck":"2016-03-26T12:28:04","gCal":"true","UI7Check":"true","CalendarID":"{removed}"}] #hooks:0
2016-03-26 12:28:50.026   luup.variable_set:4: 4.urn:srs-com:serviceId:GCalIII.gc_NextEvent was: Initial Startup now:  #hooks:0
2016-03-26 12:28:50.026   luup.variable_set:4: 4.urn:srs-com:serviceId:GCalIII.gc_NextEventTime was:  now:  #hooks:0
2016-03-26 12:28:50.027   luup_log:4: GCal3 device: 4 - local function: makeBoolean
2016-03-26 12:28:50.027   luup.variable_set:4: 4.urn:srs-com:serviceId:GCalIII.gc_exactKeyword was: true now: true #hooks:0
2016-03-26 12:28:50.027   luup_log:4: GCal3 device: 4 - local function: makeBoolean
2016-03-26 12:28:50.027   luup.variable_set:4: 4.urn:srs-com:serviceId:GCalIII.gc_ignoreKeyword was: false now: false #hooks:0
2016-03-26 12:28:50.027   luup_log:4: GCal3 device: 4 - local function: makeBoolean
2016-03-26 12:28:50.027   luup.variable_set:4: 4.urn:srs-com:serviceId:GCalIII.gc_triggerNoKeyword was: false now: false #hooks:0
2016-03-26 12:28:50.027   luup_log:4: GCal3 device: 4 - local function: makeBoolean
2016-03-26 12:28:50.027   luup.variable_set:4: 4.urn:srs-com:serviceId:GCalIII.gc_ignoreAllDayEvent was: false now: false #hooks:0
2016-03-26 12:28:50.027   luup_log:4: GCal3 device: 4 - local function: makeBoolean
2016-03-26 12:28:50.027   luup.variable_set:4: 4.urn:srs-com:serviceId:GCalIII.gc_retrip was: true now: true #hooks:0
2016-03-26 12:28:50.027   luup_log:4: GCal3 device: 4 - local function: makejson
2016-03-26 12:28:50.028   luup.variable_set:4: 4.urn:srs-com:serviceId:GCalIII.gc_jsonEvents was: [] now: [] #hooks:0
2016-03-26 12:28:50.028   luup_log:4: GCal3 device: 4 - local function: makejson
2016-03-26 12:28:50.028   luup.variable_set:4: 4.urn:srs-com:serviceId:GCalIII.gc_jsonActiveEvents was: [] now: [] #hooks:0
2016-03-26 12:28:50.028   luup.variable_set:4: 4.urn:srs-com:serviceId:GCalIII.gc_ActiveEvents was:  now:  #hooks:0
2016-03-26 12:28:50.028   luup.variable_set:4: 4.urn:srs-com:serviceId:GCalIII.gc_displaystatus was: 0 now: 0 #hooks:0
2016-03-26 12:28:50.028   luup_log:4: GCal3 device: 4 - Checking CalendarID: {removed}
2016-03-26 12:28:50.028   luup.variable_set:4: 4.urn:srs-com:serviceId:GCalIII.gc_NextEvent was:  now: CalendarID is set #hooks:0
2016-03-26 12:28:50.028   luup_log:4: GCal3 device: 4 - Checking for Credential file: GCal3.json
2016-03-26 12:28:50.053   luup.variable_set:4: 4.urn:srs-com:serviceId:GCalIII.gc_NextEventTime was:  now: Credentials Set #hooks:0
2016-03-26 12:28:50.053   luup_log:4: GCal3 device: 4 - Checking for access-token
2016-03-26 12:28:50.290   luup.variable_set:4: 4.urn:srs-com:serviceId:GCalIII.gc_NextEventTime was: Credentials Set now: Got new token #hooks:0
2016-03-26 12:28:50.290   luup_log:4: GCal3 device: 4 - Running Plugin ...


[chocorem]

Hello,

I've performed all the steps for the calendard ID, installed the V2. Vera is saying "Startup Lua Failed' in the GCAL3 .....

when I'm trying to set the Calendar ID "https://calendar.google.com/calendar/embed?src=ddg3tl7u19n13256d35hm06grc%40group.calendar.google.com%26ctz=Europe/Paris"

It says Device not Ready

What am I doing Wrong ?

Thanks

Greg