Friday, August 13, 2004

Why no pen-centric UIs?

Microsoft has released an updated version of the TabletPC OS in the latest Windows XP SP2 (it is downloadable now if you want to try it out). The major change you can see for the TabletPC is the improvement of the TIP (Text Input Panel). This is better, but hardly a radical improvement to the UI.

At a recent TabletPC workshop put on by Microsoft and the University of Washington, many researchers complained about Microsoft refusing to release a pen-centric UI for the TabletPC. Instead they have relied on a UI that is simply incremental over the existing desktop UI. The pen is little more than a mouse pointer in the existing GUI. The TIP is just one such example of a hack that is necessary to let the pen input text into existing applications without the apps having to know anything about it.

The reason Microsoft has given for this decision to many of us privately in the past is that they actually did develop a purely pen-centric UI and it tested quite well. The problem was that people either said or exhibited problems when moving between that novel UI and their existing GUI apps, which they would still have to do in the future. This feels like a bad compromise or a great research problem.

How do you allow someone to learn an entirely new UI that takes advantage of the unique input characteristics of the TabletPC platform, yet still allow them to easily use their existing GUI metaphor when it is more appropriate or necessary? Can you make this new UI easy to learn and allow it to become more pen-centric as the user gets more expertise? Think about how Marking Menus start out as a simple pie menu (a fairly traditional GUI interaction -- except for the circular part) and then moves to a gestural interface as the user become more expert with it. You could push this research even further if you consider multimodal UIs that use pen, gesture, and speech rather than just pens.

What should Microsoft do? What is the interesting research here for those of us who want to push harder? What do you think?


2 comments:

Richard C. Davis said...

How can Microsoft promote a new pen-based user interface, yet still allow the use of the GUI metaphor? I think the question is moot until Microsoft provides a compelling reason to use its pen UI. What is the “killer app” for the TabletPC? The app has existed in all of our dreams for years, and yet it still has not been realized. It’s an electronic notebook that extends the capabilities of existing notebooks in four ways. First it allows notes to be edited in some simple ways, because we all occasionally run out of space where we need it. Second, it allows scraps of information from the outside world, such as pictures, scanned documents, or nearby people’s notes to be put on the page. Third, it allows notes to be archived and retrieved easily and reliably. And finally, it allows searching through the notes for recognized text.

I thought this killer app might have been a pipe dream until I saw the “Journal” application that shipped with the TabletPC. Its UI was something of a GUI/Pen UI bastard, and it didn’t work perfectly in either situation. The tool buttons, for example, were clumsy to use with the pen, and when using a mouse and keyboard, there wasn’t a good way to enter typed text. It had bigger problems, such as sluggish performance with large collections of notes, but none of these overshadowed its strengths. Journal provides the best DRAWING experience of any ink application I have ever used, and its search capabilities are surprisingly good. Since the other problems looked easy to solve, I had hopes that Journal would evolve into the killer app I had been waiting for.

But when will it come? Microsoft seems to be investing all of its energy in OneNote, and this is becoming more tailored to keyboard/mouse input than pen input. I have not heard of any new versions planned for Journal. Why on earth not? Perhaps Microsoft believes that people are content to find a level surface and pull out a laptop computer whenever they want to write something down. I think they are very mistaken, but perhaps they will not be convinced until another company comes along and produces the application that they should have. The beauty of it all is that this application could run under Windows, regardless of what other applications are installed.

Once this electronic notebook begins to gain acceptance, then it is time to talk about how to transition between PenUIs and GUIs. I think that there’s a lot of interesting work to be done in beautification of sketched documents and incorporation of beautified sketches into “Office” documents. But that work can’t really begin until people start taking electronic notes on a large scale.

In any case, I think Microsoft should get off its butt and finish what it started.

Unknown said...

Wow, I really must have been spacing when I read that... I worked at Go! There are many interesting things in the Go model that we still don't see in most mainstream OSs (e.g., embedded applications -- it was very cool). They were simply too early (expensive, slow hardware). Run the GO OS on today's tablet PC and it would rock and be way better than Windows XP.