Go Back   HowtoForge Forums | HowtoForge - Linux Howtos and Tutorials > ISPConfig 3 > General

Do you like HowtoForge? Please consider supporting us by becoming a subscriber.
Reply
 
Thread Tools Display Modes
  #1  
Old 26th September 2012, 12:49
vilhena vilhena is offline
Junior Member
 
Join Date: Apr 2012
Posts: 18
Thanks: 4
Thanked 2 Times in 1 Post
Default Urgent: Job stuck

Hello.

I have a multi server Ispconfig configuration.

One master and one slave (webservers), and a DBserver. Each server has its own local ispconfig database.

Today I tried to change the PHP open_basedir configuration in one website, and that job stayed queued for like 30 minutes, after that I erased it from the master database (table sys_datalog).

And now everytime I make a change on a website, that jobs stays stuck in the job queue, which means I can't make any change!

I really really need some help here, this ispconfig is hosting more or les 60 websites.

Thanks in advance.

Kind regards,
Ricardo Vilhena
Reply With Quote
Sponsored Links
  #2  
Old 26th September 2012, 13:12
till till is offline
Super Moderator
 
Join Date: Apr 2005
Location: Lüneburg, Germany
Posts: 34,647
Thanks: 794
Thanked 5,003 Times in 3,912 Posts
Default

Quote:
after that I erased it from the master database (table sys_datalog).
Bad idea, never edit or change any data in that table manually as it might destroy the data integrity of the system.

Quote:
And now everytime I make a change on a website, that jobs stays stuck in the job queue, which means I can't make any change!
http://www.howtoforge.com/forums/showthread.php?t=58408
__________________
Till Brehm
--
Get ISPConfig support and the ISPConfig 3 manual from ispconfig.org.
Reply With Quote
The Following User Says Thank You to till For This Useful Post:
vilhena (1st October 2012)
  #3  
Old 26th September 2012, 13:22
vilhena vilhena is offline
Junior Member
 
Join Date: Apr 2012
Posts: 18
Thanks: 4
Thanked 2 Times in 1 Post
Default

Damn :/

Linux Distribution and version: Red Hat Enterprise Linux Server release 6.2 (Santiago)
ISPConfig Version: 3.0.4.3

I can't change the log level because the job stays stuck as the others do.

Any clue on how to solve this?

Thanks.

Regards,
Ricardo Vilhena
Reply With Quote
  #4  
Old 26th September 2012, 13:58
till till is offline
Super Moderator
 
Join Date: Apr 2005
Location: Lüneburg, Germany
Posts: 34,647
Thanks: 794
Thanked 5,003 Times in 3,912 Posts
Default

Quote:
I can't change the log level because the job stays stuck as the others do.
Thats not related as the loglevel is not part of the jobs in the jobqueue. Please change the loglevel as described in the faq by loggin into the interface of the master server and setting it to debug for the affected slave server.
__________________
Till Brehm
--
Get ISPConfig support and the ISPConfig 3 manual from ispconfig.org.
Reply With Quote
  #5  
Old 26th September 2012, 15:34
vilhena vilhena is offline
Junior Member
 
Join Date: Apr 2012
Posts: 18
Thanks: 4
Thanked 2 Times in 1 Post
Default

I did what you said, and them I run the server.sh in the two servers (master and slave)

Here's the command output:

Master:

[root@ul-clr-srv01 log]# /usr/local/ispconfig/server/server.sh
26.09.2012-13:27 - DEBUG - Set Lock: /usr/local/ispconfig/server/temp/.ispconfig_lock
26.09.2012-13:27 - DEBUG - Found 3 changes, starting update process.
26.09.2012-13:27 - DEBUG - Calling function 'ssl' from plugin 'apache2_plugin' raised by event 'web_domain_update'.
26.09.2012-13:27 - DEBUG - Calling function 'update' from plugin 'apache2_plugin' raised by event 'web_domain_update'.
26.09.2012-13:27 - DEBUG - exec: chmod 755 /var/www/clients/client6/web29
26.09.2012-13:27 - DEBUG - exec: chmod 755 /var/www/clients/client6/web29/cgi-bin
26.09.2012-13:27 - DEBUG - exec: chmod 755 /var/www/clients/client6/web29/log
26.09.2012-13:27 - DEBUG - exec: chmod 755 /var/www/clients/client6/web29/ssl
26.09.2012-13:27 - DEBUG - exec: chmod 755 /var/www/clients/client6/web29/web
26.09.2012-13:27 - DEBUG - exec: chmod 777 /var/www/clients/client6/web29/tmp
26.09.2012-13:27 - DEBUG - exec: chown root:root /var/www/clients/client6/web29
26.09.2012-13:27 - DEBUG - exec: chown web29:client6 /var/www/clients/client6/web29/cgi-bin
26.09.2012-13:27 - DEBUG - exec: chown root:root /var/www/clients/client6/web29/log
26.09.2012-13:27 - DEBUG - exec: chown web29:client6 /var/www/clients/client6/web29/tmp
26.09.2012-13:27 - DEBUG - exec: chown web29:client6 /var/www/clients/client6/web29/ssl
26.09.2012-13:27 - DEBUG - exec: chown web29:client6 /var/www/clients/client6/web29/web
26.09.2012-13:27 - DEBUG - exec: chown web29:client6 /var/www/clients/client6/web29/log/error.log
26.09.2012-13:27 - DEBUG - Enable SSL for: teste-joomla.ul.pt
26.09.2012-13:27 - DEBUG - Writing the vhost file: /etc/httpd/conf/sites-available/teste-joomla.ul.pt.vhost
26.09.2012-13:27 - DEBUG - Apache status is: 1
26.09.2012-13:27 - DEBUG - Calling function 'restartHttpd' from module 'web_module'.
[Wed Sep 26 14:27:20 2012] [warn] The Alias directive in /etc/httpd/conf.d/phpmyadmin.conf at line 11 will probably never match because it overlaps an earlier Alias.
[Wed Sep 26 14:27:20 2012] [warn] The Alias directive in /etc/httpd/conf.d/phpmyadmin.conf at line 12 will probably never match because it overlaps an earlier Alias.
[Wed Sep 26 14:27:20 2012] [warn] NameVirtualHost 10.100.25.101:80 has no VirtualHosts
[Wed Sep 26 14:27:20 2012] [warn] NameVirtualHost 10.100.25.101:443 has no VirtualHosts
26.09.2012-13:27 - DEBUG - Apache online status after restart is: 1
26.09.2012-13:27 - DEBUG - Processed datalog_id 1017
26.09.2012-13:27 - DEBUG - Calling function 'update' from plugin 'apps_vhost_plugin' raised by event 'server_update'.
26.09.2012-13:27 - DEBUG - Calling function 'update' from plugin 'network_settings_plugin' raised by event 'server_update'.
26.09.2012-13:27 - DEBUG - Network configuration disabled in server settings.
26.09.2012-13:27 - DEBUG - Calling function 'update' from plugin 'postfix_server_plugin' raised by event 'server_update'.
26.09.2012-13:27 - DEBUG - Processed datalog_id 1018
26.09.2012-13:27 - DEBUG - Calling function 'update' from plugin 'apps_vhost_plugin' raised by event 'server_update'.
26.09.2012-13:27 - DEBUG - Calling function 'update' from plugin 'network_settings_plugin' raised by event 'server_update'.
26.09.2012-13:27 - DEBUG - Network configuration disabled in server settings.
26.09.2012-13:27 - DEBUG - Calling function 'update' from plugin 'postfix_server_plugin' raised by event 'server_update'.
26.09.2012-13:27 - DEBUG - Processed datalog_id 1019
26.09.2012-13:27 - DEBUG - Calling function 'restartHttpd' from module 'web_module'.
[Wed Sep 26 14:27:24 2012] [warn] The Alias directive in /etc/httpd/conf.d/phpmyadmin.conf at line 11 will probably never match because it overlaps an earlier Alias.
[Wed Sep 26 14:27:24 2012] [warn] The Alias directive in /etc/httpd/conf.d/phpmyadmin.conf at line 12 will probably never match because it overlaps an earlier Alias.
[Wed Sep 26 14:27:24 2012] [warn] NameVirtualHost 10.100.25.101:80 has no VirtualHosts
[Wed Sep 26 14:27:24 2012] [warn] NameVirtualHost 10.100.25.101:443 has no VirtualHosts
26.09.2012-13:27 - DEBUG - Remove Lock: /usr/local/ispconfig/server/temp/.ispconfig_lock
finished.



Slave:
]# /usr/local/ispconfig/server/server.sh
26.09.2012-13:27 - DEBUG - Set Lock: /usr/local/ispconfig/server/temp/.ispconfig_lock
26.09.2012-13:27 - DEBUG - No Updated records found, starting only the core.
26.09.2012-13:27 - DEBUG - Remove Lock: /usr/local/ispconfig/server/temp/.ispconfig_lock
finished.



I've looked at the job queue and it was empty, so I tried to test a change in a website configuration (changed the php-mode from Mod-php to fast-cgi), and the job is still in queue.

Thanks.

Regards,
Ricardo Vilhena
Reply With Quote
  #6  
Old 26th September 2012, 15:39
till till is offline
Super Moderator
 
Join Date: Apr 2005
Location: Lüneburg, Germany
Posts: 34,647
Thanks: 794
Thanked 5,003 Times in 3,912 Posts
Default

Please rerun the server.sh on the slave if there is a job in the queue. ensure that you commented out the cronjob before you do that as described in the faq.
__________________
Till Brehm
--
Get ISPConfig support and the ISPConfig 3 manual from ispconfig.org.
Reply With Quote
  #7  
Old 26th September 2012, 15:42
vilhena vilhena is offline
Junior Member
 
Join Date: Apr 2012
Posts: 18
Thanks: 4
Thanked 2 Times in 1 Post
Default

I've rerun the job on the slave, here's the output:

26.09.2012-13:40 - DEBUG - Set Lock: /usr/local/ispconfig/server/temp/.ispconfig_lock
26.09.2012-13:40 - DEBUG - No Updated records found, starting only the core.
which: no tw_cli in (/sbin:/usr/sbin:/bin:/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/X11R6/bin)
which: no fail2ban-client in (/sbin:/usr/sbin:/bin:/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/X11R6/bin)
which: no fail2ban in (/sbin:/usr/sbin:/bin:/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/X11R6/bin)
/sbin/iptables
/sbin/ip6tables
26.09.2012-13:40 - DEBUG - Remove Lock: /usr/local/ispconfig/server/temp/.ispconfig_lock
finished.


The job is still queued (on the master), should I run the server.sh in the master as well?

Thanks.

Regards,
Ricardo Vilhena
Reply With Quote
  #8  
Old 26th September 2012, 15:51
till till is offline
Super Moderator
 
Join Date: Apr 2005
Location: Lüneburg, Germany
Posts: 34,647
Thanks: 794
Thanked 5,003 Times in 3,912 Posts
Default

Quote:
should I run the server.sh in the master as well?
No.

Ok, so there are several possible resaons for this:

1) take a look at the sys_datalog table with phpmyadmin and find the job, it must be the last job, the one with the highest datalog_id. The table has a column for the server_id, does this server_id match with the server_id in the /usr/local/ispconfig/server/lib/config.inc.php file on the slave server?

2) If 1) is ok, then write down the datalog_id of the record, open the "server" table in the ispconfig db on the master server, select the record for the affected slave server and compare the number in the updated column with the datalog_id of the record, the number in updated has to be lower then the datalog_id, as only jobs were datalog_id > value in updated column get executed on the slave.
__________________
Till Brehm
--
Get ISPConfig support and the ISPConfig 3 manual from ispconfig.org.
Reply With Quote
The Following User Says Thank You to till For This Useful Post:
vilhena (1st October 2012)
  #9  
Old 26th September 2012, 16:05
vilhena vilhena is offline
Junior Member
 
Join Date: Apr 2012
Posts: 18
Thanks: 4
Thanked 2 Times in 1 Post
Default

The sys_datalog table in the slave is empty. In the master there are multiple entries in the sys_datalog table. I've found the last one and server_id is 1,
but the server_id in the /usr/local/ispconfig/server/lib/config.inc.php on the slave is 3, and in the master is 1.

Thanks.

Regards,
Ricardo Vilhena
Reply With Quote
  #10  
Old 26th September 2012, 17:25
vilhena vilhena is offline
Junior Member
 
Join Date: Apr 2012
Posts: 18
Thanks: 4
Thanked 2 Times in 1 Post
 
Default

I've just found out that a colleague of mine, yesterday by mistake, erased the crontab of the root user in the master server, do you think that this could be the reason, in the first place, of the stuck jobs?

Thanks.

Regards,
Ricardo Vilhena
Reply With Quote
Reply

Bookmarks

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Cron Job php with '.....' in the ending Imam86 Installation/Configuration 1 24th September 2012 02:36
New Server - stuck job queue mmidgett Installation/Configuration 3 2nd February 2012 18:29
How to prepare for a technical job after 3 years of gap? Rodericksmith Technical 0 16th October 2010 13:13
Problem getting cron job to run and in need of help :) MrCompTech General 7 27th July 2010 09:50
Experts Please i need help urgent ia m stuck kadi Server Operation 11 13th February 2008 22:33


All times are GMT +2. The time now is 03:13.


Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2014, vBulletin Solutions, Inc.