Unstable server - high CPU wait time

Discussion in 'ISPConfig 3 Priority Support' started by philgardner, Feb 12, 2019 at 4:50 PM.

  1. philgardner

    philgardner Member HowtoForge Supporter

    Server: Ubuntu server 18.04 64-bit
    Memory: 1.5 GB
    Swap: 1.5 GB
    Hard disk: 70 GB (80% used)
    ISPConfig version: 3.1.13

    Recently my server has become unstable: individual services frequently restart, especially amavis, clamd, apache2 and PHP-FPM, and sometimes the whole server becomes unresponsive or reboots spontaneously. I have not made any major changes that might account for this, although I do update the packages regularly.

    The top line of the monit webpage now frequently shows CPU usages like this: 7.3%us, 7.1%sy, 75.2%wa
    If I understand this correctly, the very high wait percentage indicates a problem with disk I/O.

    I have also noticed frequent occurrences of the following message in the HTML5 Serial Console:
    A stop job is running for /dev/xvda2 (56s / no limit)
    xvda2 is the swap partition. As there is 'no limit' on the failing stop job (why?), the server has to be manually restarted.

    Do these symptoms imply a problem with the swap partition, as I suspect? How should I troubleshoot it? I'd be very grateful for any help.
     
  2. till

    till Super Moderator Staff Member ISPConfig Developer

    Check your disk health with smartmontools.
     
  3. philgardner

    philgardner Member HowtoForge Supporter

    It's a professionally hosted XenPV VPS, so I don't think smartmontools would produce meaningful results, would it? I don't have access to the host machine, although I could raise a support ticket in case the host hard drive has a problem. That seems unlikely, though, as I would expect them to be running diagnostic tools and to be aware of disk errors.
     
  4. Taleman

    Taleman Well-Known Member HowtoForge Supporter

    Is the host running out of memory? Is swap full?
     
  5. philgardner

    philgardner Member HowtoForge Supporter

    As it's a professional VPS, hosted on a large server, it's highly unlikely that the host is running out of memory, as this would affect all the VPSs on the host and they would have to fix it immediately.

    Swap does get quite full at times, occasionally reaching over 80% for a short time, but I haven't seen it reach 100% and it certainly isn't full when I'm seeing the high CPU wait percentages.

    Perhaps I should try disabling swap, formatting it with mkswap (keeping the same UUID) and enabling it again?
     
  6. till

    till Super Moderator Staff Member ISPConfig Developer

    If it's a vps and not a dedicated server, it might be that another VM on the same system uses too much resources. Did you talk with the support of your hoster about the problem already?
     
  7. philgardner

    philgardner Member HowtoForge Supporter

    Thanks, Till. I think all the VPSs on the host are limited in the resources they can use. However, I will open a support ticket and see what they advise.

    Meanwhile, do you think my suggestion of disabling swap, formatting it with mkswap and enabling it again is worth a try?
     
  8. till

    till Super Moderator Staff Member ISPConfig Developer

    I never had a broken swap partition, but you I guess it will not harm, so try it out.
     

Share This Page