You can cut down on a lot of the bloat by coding your own themes and plugins or being careful whose you use.
99% of the bloat in WP is WP core. It loads everything, each time. And there's a LOTof shite in there! For example, loading the front-end loads half the admin functionality, and vice-versa. Has no option but to!
It's a product of it's time, based on 2000ish procedural php4. I worked with the initial db class (ez sql) donkeys years ago... it's primarily why WP is GPL because it's core functionality was, not because of the Mullenwegs bleeding-heart 'everything wp should be gpl'. That's why they had to provide hosted version. Gotta make a buck out of it somehow!
It's got NO framework, no dynamic class loading as you'd get with an MVC framework (and the cms's based on them), NO integrated caching - without the caching plugins and opcode caching it would be unusable for a large site. As it is it's just about usable with both of these.
Saying that, it does have good points. You can throw just about any kind of shit at it and it will have a go. The admin panel is excellent, esp since 3.5.
I still use it, still develop for it, and feel it sucking away my soul each time I do!
There's quite a few mvc cms's springing up which unless WP has a total core redevelopment to an MVC / HMVC or MVCL architecture will eventually take over.
Unless you need all the extra crap or need serious customized functionality then products like fuelcms are better.
If you want a basic cms with standard pages/posts when God help us WP is still the default go to.