When I see high load averages the first thing I always check is disk IO.
Your load average should be less than or equal to the number of CPU cores you have. If the load is higher than the number of cores then you have a bottleneck somewhere and it is usually disk IO.
When this issue is happening run: vmstat 1
Ctrl+C will exit
This will print lots of data including blocks read/written each second.
You will see that IO in/out will be rather high.
Till and Turbanator are right, you need more RAM.
You were using nearly 1GB of SWAP and your CPU's were spending 61% of their time waiting on disk IO:
Cpu(s): 26.4%us, 11.2%sy, 0.2%ni, 0.0%id, 61.7%wa, 0.0%hi, 0.4%si, 0.0%st
Mem: 2063384k total, 1940664k used, 122720k free, 12108k buffers
Swap: 1951856k total, 932880k used, 1018976k free, 142528k cached