Patent Alert
Warning. There's a patent on that behaviour
There's a patent war going on. Day by day we're seeing new black and white sketches on our social media timelines revealing inventions for our future world. The general response is often wildly enhusiastic. In the imagination people have the inventions in the hands already, even though the announced products are far from being a consumer product yet and some never make it to being released at all. Companies argue that the mechanism is a requirement to do bussiness, either selling the product or selling the patent. Companies labeled "patent trolls" focus on dealing fictional patents only, never with the intention to release an actual product.
It's a playing field of big bussinesses and it seems to be relevant for those working in the hi-tech world only, but the current stream of patents is different. They apply to us, humans, in a radical new way. Many of the recent patent filing deal with augmented reality and are meant for a future in which we're wearing augmented reality headsets, not viewing AR through a smartphone screen. AR might move on from being "the gimmick" it has been called for a decade, but it is going to impact the way we see the world and interact with objects and other people. Devices like the abandoned Google Glass, the Microsoft Hololens and the upcoming Apple AR product all connect to the cloud to show us annotations on what we're see and instructions on what to do.
But are these devices also going to tell us what -not- to do? Because the cloud we're connected to doesn't own the right patents or is not willing to pay for them? Human behaviour cannot, but robot behaviour can be patented. What about the semi-digital beings we're slowly becoming with the increasing amount of technical devices we're keeping close to our body? Our smartphones, wearables and smartwatches alerting us when there's an occasion to put on our AR or VR headsets. We are the robot of the future. But what about patents applicable to our semi-digital behaviour, steered by instructions appearing in augmented reality? Perhaps our behaviour will not be blocked, but certain features or instructions could be missing from the interface. I once looked up the reason why I cannot search for an app on the homescreen of my Samsung: a patent by Apple. Losing some features is not what the big companies will worry about, and it's only slightly annoying for us as consumers. But the current series of patents in augmented reality will define the limitations we'll experience during our interactions in the real world.
To make this discussion more tangible and visible, I've created a HoloLens app that uses a cloud-based Computer Vision library to analyse what's in the surrounding of the user. Using the found object tags a patent database is queries and the resulting patents are displayed in augmented reality, with an audible warning and explanation of what's not allowed because it is covered by a patent that's not owned by the supplier of the AR device the user is wearing. A few screenrecordings of the app in action are below.
HoloLens app
Patents appearing in the video:
US20150269783A1(Samsung)
Method and wearable device for providing a virtual input interface
US20170109929 (Apple)
Method for representing a virtual object in a real environment
US20150016777A1 (Magic Leap)
Planar Waveguide Apparatus with diffraction element(s) and system employing same
Brief technical overview of the functionality of the app:
The development of the app was kick-started by the sample code in this Microsoft blog-post
Be Your Own Robot
Sander Veenhof - SNDRV.COM