The question should be: do you want a sweaty back, a wonky (unbalanced) bike or bad aerodynamics? Truth be told, it is a pain either way and if we could avoid carrying anything on a bicycle, we would.
Here is a brief précis of the options, their pros and their cons as I have found them:
Two panniers on your back wheels are a pain in the neck. You work so much harder cycling with that amount of drag; it feels like you're dragging a toddler on a skateboard behind you. It's fine on a peachy day when there is no wind but as soon as the slightest breeze starts stirring, you really do feel it. This is, however, the only option for touring. The good ones are also not cheap (£60+ for a pair of Ortlieb panniers). You do get some lovely retro panniers which are beautiful (but heavy and not waterproof).
A single pannier on the left side of your bicycle (the pavement side) is better than two panniers on your back wheels, however it does leave you wonky and lopsided if you overload it. There is less drag but be mindful of the uneven load when you get off or stop at a traffic light. You can buy single panniers in groovy fabric that double up as shopping bags and they're so cute I might try one...
Bags across the chest are not an option for busty dames. So I've never tried a full-sized version. I tried a small version and it kept on sliding forward (so the heavy bit sits on my lap) which is mightily annoying and not very ergonomic.
My personal choice. I have a dirt-cheap backpack and a waterproof cover that I bought from a hiking shop. Total cost? £18. Okay, I get a sweaty back - even in mid winter - but it's easy to carry, my trousers lie flat and don't get creased and there is no drag involved when you're cycling into the wind. That's the main pro!