This is a shorthand summary of web server performance and caching systems as of March 2016.
General disclaimer: I try to link benchmark results that seem relatively neutral, but as with any survey or study or benchmark roundup, the most pertinent question is cui bono? Ultimately the only reliable benchmarks are those you run yourself in your own environment.
Choosing a Web Server
Apache >= 2.4 with the Event MPM and Nginx are equivalent in typical environments. Nginx wins if you’re serving only a bunch of static files. If you’re running a personal blog, wiki, or typical commerce site, just use whatever’s most comfortable for you.
Nginx scales better if you’re attempting to run the next Reddit.
Lighttpd has few if any real-world advantages over Nginx.
PHP Engine
HHVM outperforms PHP-FPM by a comfortable margin. HHVM is newer technology, though, so you may want to stick with the old warhorse or run them concurrently.
Unless you’re running something that requires thread-safe mod_php (why?), at least install PHP-FPM.
Running a PHP interpreter that is separate from the web server offers additional advantages, like the ability to restart the web server without dumping the PHP cache.
Caching Systems
PHP: APC is now unsupported and obsolete. Use the Zend OPcache for opcode caching.
APCu was split from APC to serve as a userland cache for PHP. It is simple to use for single servers or those wanting a basic, in-memory, key-value data store. For distributed environments run Redis (actually more of a data structure setup as opposed to simple key-value pairs) or Memcached; Redis can outperform Memcached if tuned carefully but is more hands-on. Note that Nginx has an included Memcached module which may be advantageous here.
Python/Ruby/Other: Most frameworks like Django or Rails have their own caching systems that you configure independently.
General caching systems like Varnish, Memcached or Redis apply here too.
If you plan to run Varnish and you use SSL, a proxy or load balancer like Pound or Nginx can decrypt SSL traffic before passing it to Varnish.
Databases
In the usual LAMP type of setup, consider migrating to MariaDB from MySQL, or else migrating MySQL to TokuDB. Obviously there are many factors involved, but the XtraDB and TokuDB storage engines generally outperform InnoDB either in MariaDB or MySQL, particularly in heavy usage environments.
The Percona Toolkit offers some optimization and management software that may be of use.
Percona Server may be preferable as well.
