Scalability Notes
Jon Udell has an article on the Scalability Myth. Scalability is like optimization, premature scalability is the root of all evil. As Jon mentions, there are a lot of axes to scale along. Scalability usually brings up discussions of Amazon or Yahoo!, big websites with high loads. But I've seen discussions of scalability completely kill a project that never will see that kind of volume. Yahoo! and Amazon weren't always the size they are now, they grew into what they are. But people still go around wasting time on ill-advised caching schemes and what have you, when I think it's more important to get something working, get some good measurements, and let them guide you.
There are a few principles that generally hold true, however. When you're talking about scalability in the commonly understood sense, statelessness is definitely an important feature of these architectures. My last project was a set of web services running moderately high message rates (on the order of 7 digits / day), and the (mostly) stateless nature of the system was a definite asset, both in performance and management, but the killer was the message existence time on the server. It's one thing to scale up with messages that get processed in a second or two, it's something else with messages that can take 20 - 30 seconds (that's not a misconfigured DB or anything, we deal with a lot of external data sources that can be slow). Caching slow data sources helped out, but what we really discovered is that the problem doesn't really go away until you have a really decoupled architecture. You can't just block on a socket for 30 seconds and hope your thread pool doesn't fill up. The thing is, we did extensive performance testing, but never anticipated that we'd have 20-30 second waits for messages.