I think about this list every time I start working on a new website
- Google Analytics, because if you can't measure something, you can't improve it.
- A custom 404 page (or handler), to help visitors find what they're looking for.
- A robots.txt file, to manage how your website is crawled by search engine spiders
- A favicon.ico file, so your site stands out in browser favorite/bookmark lists.
- Uptime notification, so you know if your web site is unresponsive.
- Google Webmaster Tools, to see how Google sees your site.
- An XML sitemap for Google, to make sure than Google sees your site correctly.
- A reboot notification (for hosted sites), so you know when your hosting company reboots your server.