I don't think you're going to get a smart version that works 100% of the time. I prefer the ability to switch quickly and have the command as close to the video card as possible, so that it will work in all applications and menus universally. The iPhone invert does this pretty well, there's nothing it doesnn't invert and it's quick to switch between them. The only annoying part is that I also want to switch between voiceover and no voiceover, but if you add that to the triple tap you get a menu popup which is really slow and fiddly. So I kind of have to choose between voiceovering and inverting.

On Linux I have a key bound to a utility called xrander-invert, which does a good job of inverting everything across applications. The Windows magnifier does a pretty poor job in comparison, since it's bound to the Windows window manager (not even sure if it has a name) and so doesn't work in games and some other programs.