i wanted to throw in my 2 cents when it came to your mysql usage. If you're having high traffic on a few wordpress sites, I'm guessing you're having large load in mysql too.
Can you try increasing your key_buffer_size from 16M to something closer to your index size...like 128M (now that you're at 4G Ram) and report back after a restart and 2 days usage? I like to start higher and drop it down if it's overkill.
My other thought is your temp tables. Poor queries (and any query running over fulltext searches), will end up in temp tables which are tables written to disk. A major drain on resources. Decreasing your slow query time from 10 sec to 5 sec will probably show more of those queries so you can analyze and tweak them.