I’ve had this idea in my head for over ten years now. I’m not an expert in the field of user interface design—I have no training whatsoever, but I can’t help but bemoan the fact that blind people must have this extra layer of software—the screen reader—in between them and the computer. If you think of human-computer interaction as a stack with the hardware on the bottom, and the human at the top, input goes from the human to the operating system, to the hardware, while output goes from the hardware to the OS to the human. Every OS ever is designed with sighted output, so blind people must cram a screen reader in between themselves and the OS. I’m sure this isn’t going to go away because we want to interact with our sighted peers and use the same types of software they do, but surely, I’m not the only one to get annoyed at the kludgy way in which we sometimes need to get things done. Reddit’s current design is a perfect example. There’s no straightforward way to navigate between comments or posts. We have to find the downvote button, then move once more down to get to the username, and then skip a bunch of stuff to read the actual post.

There’s also the fear of stuff just breaking. Every time there’s an update, there may be anything from an unlabeled button to a completely unreadable window. Imagine if a similar thing befell sighted users. What if Microsoft released a patch that completely broke video output to people using a certain type of graphics card, or maybe a software update that accidentally made the start button unclickable. That sort of thing wouldn’t go live, yet we got to deal with this stuff all the time.

So here’s my stupid idea: what if we tore that stack down to the very first elements that could possibly interact with the user, and redesigned them with the blind as first-class citizens, not a graphical user interface, maybe not even a command line interface, but an audio/braille/haptic user interface. Think of it as an old school notetaker on steroids. Even the old notetakers were built on top of mobile OS’s like Windows CE but were heavily modified so that they could be efficiently used by the blind.

How would this look in practice? I don’t know. Like I said, I’m not an expert, just a grumpy blind man, but maybe it’d be useful if we looked at how things work now. Almost every GUI on desktops and laptops is based on the desktop metaphor. Your desktop is supposed to be the surface of a table or desk, with open applications representing things like papers you’re working on or a calculator you use to balance the budget, or a waste basket to throw away junk. You can move your attention to any one of these “papers” and work on it as you please. We’re used to the idea that an application is confined to one or more discrete windows, but our hypothetical new UI doesn’t use a screen at all. There’s no “full-screen” or “minimized.” Instead, we could have all currently open apps on a stack, with the currently focused app in the foreground, and other open apps receding into the “background.” You could use a keyboard shortcut to switch the focus among them. So far this is sounding a lot like alt-tabbing between open apps on Windows, but what if there were modifier keys on the keyboard, which could dynamically interact with background apps without having to bring them to the foreground. If you’re familiar with those ultra-compact keyboards that use layers to access things like function and arrow keys, that’s sort of what I have in mind. If we had three modifiers, that would give us eight layers (including the one where no modifier keys are pressed.) When you start an app, it is assigned a modifier by the OS, and when in the background, you can press that modifier key along with other keys to interact with it. The prototypical example is probably pausing music coming from a media player in the background, but I’m sure there are other good examples. You could have a terminal emulator open in the foreground, with a web browser open below it. The terminal gets most of your attention, you could hold down a modifier key and press ‘h’ to cycle through the headings on the web browser, all without “leaving” the terminal.

So yeah, probably never gonna happen, but I though I’d throw it out there. Most of this is probably because I’m frustrated as I move from low vision to non visual ways of doing things, but maybe someone else can pick this up and make something of it.