Performance testing for Web 2.0
Submitted by Donald Foss on Thu, 2006-11-02 13:09.Performance testing for Web 2.0 is similar in some ways to regular web testing, but quite different in other ways. This blog will discuss some of the ways that Web 2.0 is different in testing aspects from pre-Web 2.0 sites, and how they should be tested differently. I'll also talk about traditional web load testing, common problems, and load testing pitfalls.
Software Test and Performance Conference East - 2007
Submitted by Donald Foss on Sat, 2007-09-29 18:50.See me at STPConEast2007
continue reading "Software Test and Performance Conference East - 2007"
The worst thing you can do to your home page -- don't slow it down on purpose!
Submitted by Donald Foss on Thu, 2007-03-01 13:43. FundamentalsThis one will be short, because there simply isn't that much to say. Your home page is one of the most important pages on your site in terms of the visitor's experience. If your site requires registration, authentication or identification, nearly all users must go through this page. It is the proverbial front door to your site and application.
On a recent load test, the test had to be aborted after 9 minutes, while they were only at 25% of the planned total load level of 385,000 sessions per hour. They were using a LAMJ architecture, and each home page hit generated a long running SQL query. Even very patient users, who are tolerant to slowdowns and errors, will not stick around if the home page takes several minutes. However, this site didn't even do that! After 2 minutes into the test, the pages simply said "
continue reading "The worst thing you can do to your home page -- don't slow it down on purpose!"
Goal metrics - one of the worst ever
Submitted by Donald Foss on Mon, 2006-12-04 00:21.If you've ever heard me speak, you've probably heard me talking about how we need to stop thinking about concurrent users, except for a very small number of circumstances. However, a few weeks ago I came across a goal metric much, much worse than concurrent users. This customer was still using BroadVision and wanted to achieve a certain number of Active Sessions in BroadVision (shudder).
CMG 2006
Submitted by Donald Foss on Sun, 2006-12-03 12:25.I'm here at CMG 2006 in Reno, Nevada. Below are links to presentations that I am either presenting, or my colleagues are presenting.
Non-linear scaling
Submitted by Donald Foss on Sun, 2006-11-19 20:26.Non-linear scaling
Although I take it as a given, not everyone understands that applications do not scale linearly, no matter how much we wish that they did. If the application performs well with 10,000 user sessions per hour in the QA environment with 2 application servers, it may not work well in production with 100,000 user sessions per hour and 20 application servers. One example of this behavior has recently come to my attention.
Fundamentals - load test in a production-like environment
Submitted by Donald Foss on Mon, 2006-11-13 08:24. FundamentalsOne of the keys to a realistic load test is to test on the same infrastructure and equipment that you will be utilizing in production. The corollary is to make sure that any scheduled jobs that run during production run during the test, and jobs that only run at off-peak times do NOT run during the test. Friday night we were testing a site that started to fail at the 50% load level. Their web servers were not especially busy, and their application servers were only moderately loaded. The culprit? Rsync and other batch jobs running to replicate content and perform routine site maintenance. The system administrators had to kill a number of processes and put the cron jobs on hold. The data for that load level was largely useless.
continue reading "Fundamentals - load test in a production-like environment"
Fundamentals - basic networking skills are still important
Submitted by Donald Foss on Fri, 2006-11-03 16:24. FundamentalsOn Monday night this week, I performed a considerably large load test for a customer. They eventually found that anything that can break, will. The test was configured in a standard stepped ramp configuration. The test begins at 25% load, maintains that for 30 minutes, then goes to the next. Within 5 minutes of the beginning of the test, response time starts climbing dramatically. TCP connect time is normal, but first byte time is hitting near 5 seconds. This is a site that normally has a page response time of less than 2 seconds across all pages, which is quite an accomplishment. This is an Alexa Top 500 site. These guys know what they are doing.
continue reading "Fundamentals - basic networking skills are still important"
Fundamentals - having enough bandwidth
Submitted by Donald Foss on Thu, 2006-11-02 19:50. FundamentalsI work with many large retail customers, and this is the time of year when many of them are scrambling to make sure that they can handle their anticipated peak traffic days ahead--Black/Green Friday, the day after the US Thanksgiving holiday, and on until Christmas. One of the fundamentals that seems so 2000 is making sure you have enough bandwidth. Forget the 8-, 4- or 2-second rules--if you don't have enough bandwidth to serve up the transactions, even with a Content Delivery Network handling all of your static content, then you have major problems. At least this customer realized that internal testing does not give you the full picture, and that one must conduct end-to-end testing to really know what is going to happen. Unless your customers are shopping from your data center, then it does not make sense to test from there. My mantra "unrealistic testing yields unrealistic results" is entirely true here.