I'm running a server with ubuntu 9.10 set up according to the perfect server guide for ISPConfig 3. There's very little running on the server besides a couple of smaller websites. The box is fairly overpowered and is running just fine usually...
But every few days or so something happens that causes the system load to rise indefinitely, up to the point where the system gets completely unresponsive (I've seen it as high as 60.0).
A look at 'top' doesn't reveal any suspicious process, nothing is using a lot of CPU or memory. The amount of processes in the "waiting" state rises significantly though. My first guess was disk IO, but "iotop" doesn't reveal anything special either.
I have also installed munin, but so far I couldn't make any use of it's output. The spikes are clearly visible in the graphs, but they aren't higher than at other points of the day when the hardware has zero problems handling it.
When I'm quick enough a simple restart of mysqld solves the problem temporarily, the load goes back to normal instantly, but since so many services depend on mysql that's not necessarily the source of the problem.
Some help in hunting down this issue would be very welcome!
Thanks for any tips