OnScreenClock: permission problem

Several users have brought to my attention that the clock app no longer works after a device factory reset due to a permission problem. The dreaded message appears:

Unfortunately, your device’s rom does not support a user interface for granting the overlay permission :/

These are my findings so far:

Not brand or model specific problem
When I received the first report, I uninstalled the app on my Nexus Player (Oreo) and reinstalled it and guess what: same problem. In the meantime I received reports on Sony TV and Nvidia Shield TV.

Android TV Marshmallow and higher
Yes, all trouble started with Marshmallow. 😉 I received reports on Nougat and Oreo and tested the app also on a Nexus 7 tablet (KitKat) and Pie phone. Unsurprisingly on KitKat no problem since no permissions exist yet, on Pie my phone showed a permission dialog to grant the permission (which is good, but still abnormal).

Unfortunately, I have not encountered an Android TV rom yet which provides a user interface for granting the overlay permission (it’s not in stock android TV and none of the vendors seem to care).

Fresh installs
All reports were due to device resets, but as I already pointed out: simply uninstalling and reinstalling the clock app will exhibit the problem. Updating the app is no problem since it will preserve the previously granted permission. By taking advantage of this, I can test out a fix without impacting existing users.

I did a quick search on the interwebs but no luck so far, it’s kind of a niche topic after all. 😉 I suspect a google Play Services or Play store update is the culprit but didn’t find any news yet. If it’s part of a security fix, information will be sparse anyhow.

I see several options:

google did a small fuckup in the play services or store (as they did with the remote not responding, deep sleeping) and might fix it (can take several months)
google intentionally changed the way the overlay permission is granted (and no longer wants to grant it automagically)
the permission not being granted anymore might be a side effect of some other policy change

Overlay permissions have always been tricky on android (tv), so let’s hope there is a solution possible.. (or a hack at least)

Quick KDE KWin hack: Ignore global shortcuts except alt-tab

From time to time a pet peeve can reach a critical level of annoyance at which it starts to irritate. You google it, hoping to find a solution or at least some allies, but if none turns up and you’re out of time, you simply give up and hope a solution will magically manifest itself one other day. But what if that day never comes? Luckily for us, open source enthousiasts, we can do something about it ourselves 🙂

Context

I’m quite an avid shortcut user. Linux/KDE is an environment where this type of user feels right at home. Not only do most actions in a software have a shortcut, you can also define lots of system-wide or global shortcuts. Kwin, KDE’s window manager defines quite a few out of the box.

Problem

This is all fine and dandy until you want to use this cross-platform software which you got used to in a different environment (i.e. windows at work) and you want to start using it at home. When the application is not (KDE) shortcut friendly or you are so used to the windows keymap that it’s a pain to learn a new scheme, you start to feel frustrated. Learning a new shortcut scheme seems the best solution but this doesn’t work well when you have to maintain 2 shortcut schemes in your head, depending on your environment (home vs work, your computer at work versus a colleague’s computer).

A typical example here would be some IDE, for example Intellij IDEA. Although the IDE does provide a KDE keymap by now, I started using it ages ago when there simply was no alternative keymap (nor a community edition or even official linux version, I guess). Also, I used linux only occasionally back then. These shortcuts are hard-wired by now and it is already hard enough to keep in mind whether I’m using Intellij or Visual Studio.

Another possibility is that the application really needs that much shortcuts that it just can’t bother with the shortcuts which may or may not be defined by the environment or workspace. One example of such complex application is Blender. Try it out and you’ll see what I mean. 😉

KDE, being the configurable desktop environment, does offer some configuration settings. For each window, Kwin allows you to define rules which match certain windows / applications which creates a config. Using this config, you can configure special window attributes. One setting is to block all global shortcuts.

This allows you to use your favorite application with its favorite native shortcuts. Of course, most people use more than 1 application at the same time. However, since all global shortcuts are blocked, the alt+tab shortcut to switch windows is blocked too. Alt+tab is not a special shortcut to KDE, it simply is a shortcut that happens to be assigned globally to the “Walk through windows” action exposed by the KWin component to the System Settings. Lacking any window switching shortcut, you adapt and start using the mouse to switch applications. There are times you cope just fine. But there are other times it is just one thing too much that annoys you. 🙂

Solution

So how to fix this? Just like any other problem you have with an open source application: fetch the source, fix and build! 😉

Since I just wanted a quick fix (I thought, let’s check this out for 30 mins, which turned out to be a lot longer of course), I chose to patch the behavior of the “Ignore global shortcuts” setting to exclude the Alt+Tab shortcut (more precisely, the shortcut currently assigned to the “Walk through windows” action). I briefly investigated KWin’s scripting abilities but it turns out the required functionality is not exposed.

I will now document the process, which took place on opensuse leap 42.3. You may want to tune out now if you don’t care about the gritty tech details. 😉

Where is the source?

I thought it had to be kwin, right? So I checked out the source from github and started searching for “Ignore global shortcuts”. It turned out to just call something in kglobalaccel which is not part of kwin.

Getting my hands dirty, I first fired up a VM I had lying around in order not to f*ck up my currently running system. Next I installed Qt Creator to easily navigate the code. Also, never forget you have to checkout the tag for the version you are currently running! (can be a treasure hunt by itself)

Browsing the source, I finally find a possible tweaking point in the src/runtime/component.cpp -> deactivateShortcuts method. It disables all shortcuts when a window is entered which has a “Ignore global shortcuts” rule. So I simply added an exclusion for the “switch windows” action.

Building a patched rpm

Time to try it out! In order to build it, you will need to install its dependencies. On an rpm based system, the easiest way to go forward is to do a “sourceinstall” of the rpm in case. For that, we first have to know which rpm contains the affected shared object (.so file).

You can take different routes to pin point the rpm. I checked the CMakeLists.txt to see what the output targets were. When you build, you can also see the class pass by and check into which library it gets linked into. Using that .so file, you can find the rpm using “rpm -qf <.so>”

An alternative way would be to find a kde related rpm matching “global” and then checking its contents (using “rpm -q –filesbypkg <rpm>”) for the .so file.

Once you know the rpm, you can easily install all required build dependencies by issueing a zypper sourceinstall command: “zypper si <rpm>”

The hard part: logging

I wanted to add some debug output to see if I was looking at the right place in the source base and how frequently the code got called. Now things got messy. You’d think patching the code was the hard part, guess what.. 😉

I had played with logging before in KDE, but of course things have changed with KDE 5. Nowadays, KDE uses Qt’s default logging mechanism. It meant figuring out the logging category used by kglobalaccel and finding a way to activate it. The logging category was nicely tucked away in logging_p.h: “kglobalaccel-runtime”. Activating it seemed to be trickier. According to the Qt documentation, you can alter the log level through an env var QT_LOGGING_RULES. Since I was adapting a core component, I needed to set the env var before logging in. On opensuse you can create a file /etc/profile.local which gets sourced by /etc/profile.

I added the env var and it seemed to work (=showed up), unfortunately still no logging. More precisely, nothing showed up in ~/.xsession-errors-:0. I started adding log statements all over the place, analysing the .so file to check whether my modified method was actually included in it (readelf -Ws libKF5GlobalAccelPrivate.so.5.32.0 | grep deactivate), etc.. Until I realised I didn’t check the journal (journalctl -b -0) yet, there it was. 🙂 Bonus points: the code change had its intended effect!

End result

So there I had it, a working patch WITH logging ⇒ pure SATISFACTION 🙂

Feel free to share this satisfaction by downloading the .so and installing it in /usr/lib64 (use at own risk). I’ve been using it for 2 months now without problems. It’s a good hack until a more fundamental solution is in place. First steps have been taken! 😉

OnScreenClock v0.5.0

Customizable colors

Yes, you read this well! The most requested feature is finally there, although I wouldn’t call it finished just yet. As of 0.5.0 you can configure the clock and outline color by selecting a color from a predefined palette. You might have noticed I’m not that savvy with colors (a lot complained about the default green color, or was it yellow? I blame the color blindness) so I wasn’t sure which colors to include in the preconfigured palette. I found some nice material design style colors but figured people probably don’t care about material design and would want to tweak the colors anyway. 🙂 So I simply configured the colors from a standard 12-color color wheel and added black and white. The idea is that in a future version, you will be able to select a color from the palette and then tweak it to your liking. At first I didn’t want to release without the tweaking functionality but since this feature has been delayed for so long, I realized it’s better to release, put already quite some people out of their eyestrain misery and bear the negative reviews bitching about the fixed palette. 😉

The reason it took so long is because I wasn’t sure how to provide a user-friendly user interface for selecting a color on a tv (using a D-pad). I now settled on the fixed palette and found peace with it since I will be offering a fine-tune option later on.

So with the big fancy feature out of the way, what else is new?

Better gamepad support

As a Nexus Player user, I was harassed by this nasty bug Google rolled out a few weeks ago which causes the remote to go to sleep at the worst possible moments, i.e. when you’re using it. 😉 It’s really frustrating when you’re trying to watch tv, but it’s even more frustrating when you’re developing an app. 😉 I tried using an emulator but my pc hardware is officially unsupported now by google’s emulator (missing some instructions). It’s time to replace this 8 year old rig, it served me well.. Anyway, I’m digressing.

As a replacement for the bluetooth LE remote, I started using a Logitech gamepad. Unfortunately, it turned out to not emit DPAD_CENTER events required for adjusting size and position. From now on, gamepads should work just as well (use the “A” button to press “enter”).

Positioning the clock

While I was at it, I added a notice about keeping the button pressed to reposition the clock faster. It seems some people have trouble moving the clock around and complain having to move it a pixel at a time. The input dialogs for repositioning and scaling the clock were originally stubs for a nicer UI but as a lot of things, it just stayed.. Might improve in some future version.

Android 4.0.3+ required

The original plan was to have the clock run on as many devices as possible (also smart phones). So for a long time, the min api level was 10 (Gingerbread or 2.3.3+). I remember having a gingerbread device and often being annoyed by developers bumping the android version too early, effectively abandoning me from a functioning app (due to backend changes). I looked at the app stats and it turns out there are only 3 users below 4.0.3. I apologize, but no new version anymore for them. The bump is due to support lib requirements.

Changelog

+ customize clock and outline color
* fix pressing buttons with gamepad
* size and position pref: hint about keeping the button pressed to go faster
* min api level bumped from 10 to 15 (4.0.3) due to support lib requirement
* renamed main activity title because Oreo shows it in the launcher

OnScreenClock v0.4.1

This is a small update before the big changes come in 😉

I noticed sometimes the clock disappears due to high system load. Normally, the android system automatically relaunches the service afterwards, but sometimes this does not seem to happen. I’m not sure what’s the cause, but I added a trigger: when the service is destroyed by the system, it sends out a broadcast event just before dying to restart the service. Not sure if it will help..

v0.4.1 2017-11-30
=================
* try restarting service when it is killed by the system to free resources
* fix NPE crash when back button is pressed before webview is fully loaded

ADB over Ethernet v0.8.3

So I finally managed to release a new version of the ADBoverEthernet app. I noticed the ip detection stopped working on Marshmallow and higher. Rather than using command-line script-fu, I now use the official android api’s to query network interfaces and stuff. 🙂 I don’t recall why I didn’t use those in the first place, but in its current state, it definitely seems to work on all tested devices.

A nice-to-have is the new “autostart” feature. Check the box to enable the ethernet adb at boot time. This also explains the new permission RECEIVE_BOOT_COMPLETED.

v0.8.3  2017-04-28 
================== 
+ autostart at boot option (requires NEW permission RECEIVE_BOOT_COMPLETED) 
* use better ip detection logic (fixes ip detection on marshmallow and higher) 
* fix NPE when root access is denied

OnScreenClock: back to 1 app

When google released Android Marshmallow, they introduced a new permission model. I jumped on the band wagon too eagerly and as a result, the OnScreenClock app had become unusable on Marshmallow and up due to an unimplemented feature on Android TV. Complaining to google did not help.

So I did the only thing I could do: release a new app targeting an older api. This app is known as “OnScreenClock Marshmallow” (a bit of a paradox, i know 😉 ). Recently, however, just when I was about to migrate all users of the old app to the new one, I discovered Google has relaxed their policy, so now the “old” app is good to go again! I guess it was causing too much trouble for users everywhere.. This is undocumented so I guess Google wants to keep that change low profile. 😉 This also means the future is uncertain, but that’s life, right?

To all users using the “OnScreenClock Marshmallow” app, please migrate to the “OnScreenClock” app as I will not be further developing the “OnScreenClock Marshmallow” app anymore.

Apologies for the inconvience.

OnScreenClock v0.3.3

After a long hiatus, finally an (unfortunately minor) release 🙂 Let’s hope for more in the near future.

Since on Android TV, you never know when an app has been updated, I added an optional notification on upgrade so you know when the app has been upgraded. You can choose how you get notified:

Either a web page pops up right after the upgrade (this usually doesn’t bother since apps get updated at night when you don’t use the tv) where you can read up on the changes,
Or you get a visual indication in the clock widget. In the latter case, next time you open up the app, you will be presented with the changelog and the indicator disappears.

The latter option is default since it is less obtrusive.

In other news, you can finally change the text clock format to 12-hour notation if you wish so!

v0.3.3 2017-03-09
=================
+ optionally show clock notification after upgrade (default)
+ show D-pad instructions above release notes webpage
* fix 12 hour clock starting at 0

v0.3.2 2017-03-02 (not released)
=================
+ added 12-hour clock format preference
+ optionally show release notes after upgrade

OnScreenClock migration notice

For Android TV users

Use left/right on your D-pad to switch between scroll and link mode.
Use up/down to scroll the page/jump between links.

TL;DR

Please install the new OnScreenClock app as this one will no longer be developed and does not work on newer versions of Android (Marshmallow and up). The new app name is “OnScreenClock Marshmallow”. Sorry for the inconvenience.

More elaborate explanation

Marshmallow on Android TV does not support granting the overlay permission needed for this app, so I had to downgrade the targeted android version to Lollipop. However, google does not allow that for an app already published for Marshmallow, so I had to create a new app in the play store. For a more elaborate technical rant, read this post.

Android TV, Marshmallow and Overlay permission

I recently upgraded my Nexus Player to Marshmallow and guess what, the OnScreenClock app crashed. No big deal, I thought, let’s “just” fix this by properly requesting the overlay permission using the new permission system. Reading up on the topic, I learned there are 2 types of permissions: normal and dangerous permissions. Dangerous permissions need explicit permission from the user while normal permissions are automatically granted by the system. However, a small info panel (like for tidbits) in the documentation grabbed my attention, stating (parafrased): “oh, and btw, you also have some special permissions”. So actually there are 3 kinds of permissions! Why not say that up-front 😉

Anyway, you might have guessed, the OSD clock app uses a special permission called SYSTEM_ALERT_WINDOW which allows it to create windows that draw on top of anything else. You might think (as I did at first): no problem, let’s just ask the user for this OVERLAY permission. This is a nice idea in theory, and it actually works on your average Marshmallow phone. Android TV would not be Android TV if they did not add something extra to the challenge. Indeed, all Marshmallow Android TV roms that I have encountered (to be honest, so far only Nexus Player and Nvidia Shield, there are not much more at the moment), do NOT provide a GUI for allowing the user to allow the app this permission. They do have a gui for dangerous permissions but not for “special” permissions! I guess we finally know now what makes them special. 😉 A bug has been reported (originally for Android Wear which suffers the same problem) but no response from google.

So yeah, in case of my OSD clock app: no permission = no app. But wait, no problem, let’s just revert back to Lollipop target sdk and everything’s fine like Marshmallow never happened, right? Well, except if you already published a version of your app targeted for Marshmallow. In that case, you are not allowed by Google’s Play store to downgrade your target SDK. That’s really too bad.

That’s why I had no other choice than to create a new app with a similar name and update the current app to show a deprecation notice, inform the users of the new app and hope they understand what to do. So, goodbye nice reviews and stars, goodbye established user-base… Congratulations Google, you win.