Here's an excellent presentation (PDF) on how Amazon measures their Web site performance, with an emphasis on A/B testing -- running experiments where one element on a page is changed to see which performs significantly (statistically speaking) better. My favorite bit (paraphrased from slide 22):
- Automation replaces intuition:
- home page is made up of slots
- anyone can submit content for a slot
- best-performing content (based on A/B testing) wins
For those of you considering an e-commerce system (or upgrading the one you currently have), you should ask if/how the system supports A/B testing. Of course, this requires that you are already generating significant traffic to your Web site, which means people can find it, and that they are effective when using it (i.e. they reach the goal that they had in mind when visiting).
I'd be interested in hearing from Cultivate, eWinerySolutions, or Inertia on the topic of A/B testing and how their customers might do it using their product offerings. Measure and test is the heart of Web site success.
(PS - A simple way to start tackling this issue is to have a "Most Popular Pages" list in the sidebar of your home page above the fold. At least this points visitors to pages that other visitors find interesting. Bottom line: you need to respond to how users are voting with their clicks.)