We're trying to launch our Django site, and we're coming into severe load issues.
The site right now has 2 SQL servers, clustered, and 3 web servers all running with 4 cores and 4 gigs of memory.
Our biggest problem is everything seems ok on the development site, load times are ok, the SQL queries are quick, and there doesn't seem to be any slow down. We use caching *everywhere* that it hits the db and have tested both memcached and disk caching.
Once we push it to the live server, immediately all web servers end up averaging 10-20 load. We should be able to handle this on 1, maybe 2 servers, the 3rd server was to handle *any* kind of load we would have within the next few months, we've been able to handle peak traffic days with just the 2 servers using PHP/MySQL.
We are running lighttpd with fastcgi in prefork mode. We tried using threaded but django spits out an error about a weakly-referenced object, related to sessions I believe, no longer existing. Yes I submitted this as a ticket and as far as I know it still exists.
If anyone has any experience with debugging this kind of problem it'd be a great help.