When iPhone 5S launched, the new word entered the discussion pretty soon. Many early users and reviewers agreed that the new Apple smartphone was “futureproof”. They were talking about the new 64-bit A7 CPU powering iPhone 5S, and M7 coprocessor that continuously tracks motion data. Value of both of those chips is pretty limited today, but they will enable a whole class of new apps and user experiences in the future.
But A7 and M7 are not the only iPhone 5S parts that are future proof. Its new Touch ID fingerprint sensor, is just a biometric ID for now. It makes locking and unlocking your smartphone, and shopping on iTunes a tad easier. But Apple has much bigger plans for Touch ID.
The near term Apple Touch ID designs include transforming the fingerprint sensor under the Home button into a trackpad, adding new way to navigate and control iPhone UI. It should come in very handy with the launch of iPhones with 5”+ sized displays next year.
If you can control them by sliding your thumb around Home button, just as well as swiping your finger around iPhone 4S display – your big iPhone 6 might be as easy to navigate around with a single hand. And there goes one of Apple’s main stated reasons for not building larger iPhones until now – one hand usability.
That’s the near term plan.
But Apple has even bigger designs for Touch ID in a more distant future. I may be oversimplifying a bit here, but Touch ID sensor in iPhone 5S is the same capacitive touchscreen sensor Apple uses for Multi-touch. Only with much higher (500 dpi) resolution. So enabling all iPhone display four touch ID is a matter of scaling up the technology Apple already made to work on a Home button.
An iPhone or iPad with Touch ID display will be able to recognize not just multi touch gestures, but also which finger from which hand you use to touch the screen. Which opens up a possibility to significantly upgrade Multi-touch UI with new gestures and user experiences.
Apple’s plans for Touch ID sensor have been described in detail in a patent application called “Device, Method, and Graphical User Interface for Manipulating User Interfaces Based on Fingerprint Sensor Inputs”, that showed up in Wolrd Intellectual Property Organization database last week.
Here are some of the ways Apple intends to use Touch ID, described in a patent app.
iPhone Home button as an advanced trackpad
The key to your Home button’s capability to act as a trackpad is the extreme sensitivity of a fingerprint sensor. At 500 dpi, the Touch ID sensor is able to read not just overall outline of your fingerprint, but the smallest details of a fingerprint ridges, and it can recognize multiple prints in any orientation, real fast.
All that is needed to transform Touch ID from identification device into a trackpad – is adding the ability to follow changes of scanned fingerprint (or even just small part of it, and its ridges) over time – i.e. tracking its movement over the surface of the button. It might require a lot of processing power to work well, but that’s where 64-bit A7 and future Apple CPUs come in. The rest is software magic, similar to what Apple did with Multi-Touch in 2007. Which, I am sure, engineers in Cupertino are busy perfecting now for Touch ID.
This will allow your iPhone to recognize ‘Revolving” and “Twisting” finger movements on a Home button:
And will enable you to control the iPhone by “twisting” or “revolving” your thumb around the home button. E.g. slide your finger left <-> right and the map displayed on the screen scrolls in the same direction, or your browser window scrolls up and down. Move your finger upwards, and the multitasking window pops up, making the map underneath inactive, and allowing you to scroll through recent apps.
In addition to tracking your fingerprint movement, the Home button can measure time – how long you press it, and force – how hard you press it. Combine these three readings and you made Home into an advanced joystick, allowing for the whole new type of game controls:
Measure the pressure and time while moving your finger around the Home button, and you get a scroll wheel to navigate around the homescreen full of icons:
Integrating Touch ID into iPhone and iPad touchscreens
Touch ID in iPhone’s Home button is also a capacitive touch sensor, in its basics pretty similar to what Apple is using today in the Multi-touch devices. Capacitive plate array is much denser, but the operating principle is the same. So making iPhone display into a capacitive touchscreen with enough resolution to read fingerprints, will mostly involve an upscaling of the technology Apple already made to work in iPhone 5S.
But having such dense capacitor array active whenever the screen is on, may consume too much power. To avoid that iPhones and iPads with Touch ID displays will operate in two modes. Reduced sensitivity – acting like a traditional Multi-touch display of today, and enhanced sensitivity – performing fingerprint recognition.
Furthermore, Apple says that they should be able to activate only separate small screen areas in enhanced sensitivity mode. In such case, only places above certain Apps which require protection, like Banking or Mail, could be Touch ID sensitive, while the rest of the display works as normal touchscreen.
Fingerprint reading display will also allow you to create gesture shortcuts for apps and functions you need. E.g., pinch with the right pinky and thumb may launch an e-mail app, while the long-press with the right thumb could launch a camera from any app you are currently in, snap a picture, and return you to that same app.
Recognizing which fingers are touching the display could also enable more intuitive and sophisticated game controls, as well as development of completely new types of applications. E.g. touch-typing or piano playing.
Touch-typing assigns each of the keys on a QWERTY keyboard to one of your 8 fingers (except thumbs), and then builds-up your muscle memory so each finger “knows” where each letter is without you looking. But since your iPad has no idea which finger you used to press a letter, training in touch-typing on it was nearly impossible.
With a fingerprint sensor integrated into the screen, your iPad will “know” whether you press the letter “P” with the left pinkie or index finger. And let you know when you are doing things wrong, so effective touch-typing apps can now be designed for it. Similar techniques can be used for piano playing training exercises.
Touch ID enabled touchscreens are still probably several years away. There’s simply too much of technology integration and development to be done, to make it work seamlessly.
But I have little doubt that Apple is already working on it. And the rumors about iPhones with pressure sensitive displays, Apple’s investment in Sapphire manufacturing tech, etc; are probably related to that.
As for the trackpad on a Home button? Most of the pieces of that technology are already in iPhone 5S. And adding such functionality for bigger screen iPhones makes so much sense, I will be surprised if we don’t see it at iPhone 6 launch.
The patent application itself, is a whopper of the document – 612 pages of text&drawings, and 464 patent claims. The only applications of this size that I’ve seen from Apple before, were for the main Multi-touch patent and SIRI. Which is a good indication how seriously Apple is taking its Touch ID tech.