The most annoying aspect that come to mind are touch-screen based payment terminals to input a PIN, which are not really usable for most blind people in any reliable way. Online payment is usually straightforward enough, and when it isn't it's more of an issue of electronic banking problems (using virtual keyboards for example).

Calling the bank is always a pain for me. They always want me to read back the last 3 transactions on my previous statement, for security reasons, before they'll even talk to me about anything. I can't read paper statements, of course, so that means logging in to my banks online website, and listening to text to speech read out the transactions, then relaying that to them over the phone. Of course, if I'm calling because I need an online password reset, or I am having trouble with the website, I just can't. I have to go into a local branch and present ID in-person, no other option.

Agreed. Payment kiosks pose the biggest problem for me because there is usually no way to use them without assistance. A small one, but getting all of the information for a new credit card is also something that is not as easy to do independently.