Go Back   HowtoForge Forums | HowtoForge - Linux Howtos and Tutorials > Linux Forums > Technical

Do you like HowtoForge? Please consider supporting us by becoming a subscriber.
Reply
 
Thread Tools Display Modes
  #1  
Old 22nd November 2011, 12:37
ZeroEnna ZeroEnna is offline
Member
 
Join Date: Aug 2010
Posts: 71
Thanks: 9
Thanked 1 Time in 1 Post
Default Unexplained Load increases

Hi everyone,

for over two months, I am facing unexplained load increases on my server.

Take a look at the screenshot.

Cacti-Screenshot

At about 2.00, no login is possible, and the remote service board fails to get a video signal as well, because no input is possible.

Well, is there a tool like Microsoft's perfmon to caputre load, and what application causes it? Maybe I can isolate the cause and fix the problem.

Thank you in advance

Kind Regards

ZeroEnna
Reply With Quote
Sponsored Links
  #2  
Old 23rd November 2011, 13:14
falko falko is offline
Super Moderator
 
Join Date: Apr 2005
Location: Lneburg, Germany
Posts: 41,701
Thanks: 1,900
Thanked 2,740 Times in 2,575 Posts
Default

Might have to do with a cron job and/or hard disk I/O. munin might give you more insight.
__________________
Falko
--
Download the ISPConfig 3 Manual! | Check out the ISPConfig 3 Billing Module!

FB: http://www.facebook.com/howtoforge

nginx-Webhosting: Timme Hosting | Follow me on:
Reply With Quote
  #3  
Old 23rd November 2011, 20:10
ZeroEnna ZeroEnna is offline
Member
 
Join Date: Aug 2010
Posts: 71
Thanks: 9
Thanked 1 Time in 1 Post
Default

Concerning the crontab, this is output of

Code:
for user in $(cut -f1 -d: /etc/passwd); do echo $user; crontab -u $user -l; done
Code:
root
0 1 * * * /var/www/kibo.fm/web/sctrans/sc_trans_040/sc_trans stop
45 * * * * MAILTO=""; lynx --dump "http://kibo.fm/meinkibo/bot.php?action=log&password=d6iKpB$"
00 01 * * 0 /root/logs_backup.sh

00 02 * * * rsync -rltDv -e ssh /backup/ hikarimedia@rsync.hidrive.strato.com:/users/hikarimedia/
00 03 * * 0 /root/spamtrain.sh
30 01 * * * /root/mysqlback.sh

00 03 * * 6 /root/reboot.sh
23 4 */2 * * /usr/bin/sa-update --no-gpg &> /dev/null
* * * * * /usr/local/ispconfig/server/server.sh > /dev/null 2>> /var/log/ispconfig/cron.log
30 00 * * * /usr/local/ispconfig/server/cron_daily.sh > /dev/null 2>> /var/log/ispconfig/cron.log
daemon
no crontab for daemon
bin
no crontab for bin
sys
no crontab for sys
sync
no crontab for sync
games
no crontab for games
man
no crontab for man
lp
no crontab for lp
mail
no crontab for mail
news
no crontab for news
uucp
no crontab for uucp
proxy
no crontab for proxy
www-data
no crontab for www-data
backup
no crontab for backup
list
no crontab for list
irc
no crontab for irc
gnats
no crontab for gnats
nobody
no crontab for nobody
libuuid
no crontab for libuuid
Debian-exim
no crontab for Debian-exim
statd
no crontab for statd
zeroenna
no crontab for zeroenna
sshd
no crontab for sshd
ntp
no crontab for ntp
mysql
no crontab for mysql
postfix
no crontab for postfix
clamav
no crontab for clamav
amavis
no crontab for amavis
vmail
no crontab for vmail
getmail
*/5 * * * * /usr/local/bin/run-getmail.sh > /dev/null 2>> /var/log/ispconfig/cron.log
ispapps
no crontab for ispapps
ispconfig
no crontab for ispconfig
web1
no crontab for web1
shoutcast
no crontab for shoutcast
web3
no crontab for web3
postgrey
no crontab for postgrey
unreal
no crontab for unreal
anope
no crontab for anope
eggdrop
# m h  dom mon dow   command
*/1 * * * * /home/eggdrop/eggdrop/scripts/botchk > /dev/null &
web5
no crontab for web5
web6
no crontab for web6
web7
no crontab for web7
web8
no crontab for web8
web9
# m h  dom mon dow   command
#00 16 * * * /var/www/kibo.fm/web/sctrans/sc_trans_040/sc_trans start
web11
no crontab for web11
web12
no crontab for web12
web13
no crontab for web13
web14
no crontab for web14
web15
no crontab for web15
web16
no crontab for web16
web17
no crontab for web17
web18
no crontab for web18
davfs2
no crontab for davfs2
dellroot
no crontab for dellroot
web19
no crontab for web19
ircservice
no crontab for ircservice
web21
no crontab for web21
marcel
no crontab for marcel
sctrans
no crontab for sctrans
web24
no crontab for web24
munin
no crontab for munin
messagebus
no crontab for messagebus
snmp
no crontab for snmp
web26
no crontab for web26
tcpdump
no crontab for tcpdump
web29
no crontab for web29
paranormal
# Edit this file to introduce tasks to be run by cron.
#
# Each task to run has to be defined through a single line
# indicating with different fields when the task will be run
# and what command to run for the task
#
# To define the time you can provide concrete values for
# minute (m), hour (h), day of month (dom), month (mon),
# and day of week (dow) or use '*' in these fields (for 'any').#
# Notice that tasks will be started based on the cron's system
# daemon's notion of time and timezones.
#
# Output of the crontab jobs (including errors) is sent through
# email to the user the crontab file belongs to (unless redirected).
#
# For example, you can run a backup of all your user accounts
# at 5 a.m every week with:
# 0 5 * * 1 tar -zcf /var/backups/home.tgz /home/
#
# For more information see the manual pages of crontab(5) and cron(8)
#
# m h  dom mon dow   command


28,58 * * * * /home/paranormal/bots/psi-bot/scripts/botchk



web31
no crontab for web31
minecraft
no crontab for minecraft
avahi
no crontab for avahi
Reply With Quote
  #4  
Old 25th November 2011, 17:49
falko falko is offline
Super Moderator
 
Join Date: Apr 2005
Location: Lneburg, Germany
Posts: 41,701
Thanks: 1,900
Thanked 2,740 Times in 2,575 Posts
Default

I guess this cron job is the problem:

Code:
00 02 * * * rsync -rltDv -e ssh /backup/ hikarimedia@rsync.hidrive.strato.com:/users/hikarimedia/
It could cause high disk I/O. Can you deactivate it and see if that changes anything?
__________________
Falko
--
Download the ISPConfig 3 Manual! | Check out the ISPConfig 3 Billing Module!

FB: http://www.facebook.com/howtoforge

nginx-Webhosting: Timme Hosting | Follow me on:
Reply With Quote
  #5  
Old 29th November 2011, 09:37
ZeroEnna ZeroEnna is offline
Member
 
Join Date: Aug 2010
Posts: 71
Thanks: 9
Thanked 1 Time in 1 Post
Default

Hi,
unfortunately, this is my backup job, so disabling it for about 3 weeks might cause a problem. Also, the problem occured at about 3 p.m, the task finished at about 2:40 a.m., so I don't see any relations between the increasing load and the task.

A few days ago I remembered that this had happened before. But in July or August (I think) there has been an update, I don't know what it was anymore, maybe the logs will tell.
Anyway, after this, I had uptimes of about 60 to 70 days, when a kernel update arrived. After installing it, my uptimes crashed! So either, one of my installed packages has some problems, or something else is going wrong with it, as a patch fixed it, and another patch caused it to happen again.

Regards

ZeroEnna
Reply With Quote
  #6  
Old 30th November 2011, 18:53
falko falko is offline
Super Moderator
 
Join Date: Apr 2005
Location: Lneburg, Germany
Posts: 41,701
Thanks: 1,900
Thanked 2,740 Times in 2,575 Posts
Default

Quote:
Originally Posted by ZeroEnna View Post
Also, the problem occured at about 3 p.m, the task finished at about 2:40 a.m., so I don't see any relations between the increasing load and the task.
In your first post you said it happens at 2.00h. (Is it possible that this has something to do with daylight savings time? Do you and your server have the same time?)
__________________
Falko
--
Download the ISPConfig 3 Manual! | Check out the ISPConfig 3 Billing Module!

FB: http://www.facebook.com/howtoforge

nginx-Webhosting: Timme Hosting | Follow me on:
Reply With Quote
  #7  
Old 2nd December 2011, 08:16
ZeroEnna ZeroEnna is offline
Member
 
Join Date: Aug 2010
Posts: 71
Thanks: 9
Thanked 1 Time in 1 Post
Default

I'm sorry, I was misleading.

I am currently (at this very moment) having issues again. It started at about noon yesterday and increased slowly to a peak of 7 and staying!

Please have a look at the munin graphs (They are accessable)

I have no access to SSH anymore, The connection is established, I get a liogon prompt but then the session freezes with no possibility of entering commands. Also, the Console Redirection (DELL iDRAC) is not returning a display, it stays black.

The only option I have is a reboot, but I am afraid, I will never ever find out what is causing this! Is there a possibility (as FTP and HTTP are working fine) to write a script that returns the output as a Webpage (e.g. something with exec())?

Regards

Zero
Reply With Quote
  #8  
Old 3rd December 2011, 15:08
falko falko is offline
Super Moderator
 
Join Date: Apr 2005
Location: Lneburg, Germany
Posts: 41,701
Thanks: 1,900
Thanked 2,740 Times in 2,575 Posts
 
Default

According to http://www.hikari-media.de/monitorin...ops/index.html you have some disk IO. That could be the problem, but I'm not sure. If it's possible to change the HDDs, you should try that.
__________________
Falko
--
Download the ISPConfig 3 Manual! | Check out the ISPConfig 3 Billing Module!

FB: http://www.facebook.com/howtoforge

nginx-Webhosting: Timme Hosting | Follow me on:
Reply With Quote
Reply

Bookmarks

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
ISPConfig 3.0.3.3 Autoresponder, adding subdomain problem BlackHat Installation/Configuration 4 18th October 2011 14:24
High Availability (Load Balancing) behind a firewall geek.de.nz Server Operation 7 4th January 2011 13:58
can't send/recieve email postfix/smtpd archbird Installation/Configuration 6 8th November 2010 14:48
libWand.so.10 error Taxick Installation/Configuration 8 3rd May 2009 01:27
Howto suggestion suse PhP ver 4 + Ver 5 wwparrish Suggest HOWTO 11 7th August 2006 13:29


All times are GMT +2. The time now is 22:56.


Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2014, vBulletin Solutions, Inc.