HowtoForge Forums | HowtoForge - Linux Howtos and Tutorials

HowtoForge Forums | HowtoForge - Linux Howtos and Tutorials (http://www.howtoforge.com/forums/index.php)
-   Installation/Configuration (http://www.howtoforge.com/forums/forumdisplay.php?f=27)
-   -   After upgrade to 3.0.5.2, getting: libz.so.1: cannot open shared object file: No such (http://www.howtoforge.com/forums/showthread.php?t=61818)

cbj4074 12th May 2013 00:20

After upgrade to 3.0.5.2, getting: libz.so.1: cannot open shared object file: No such
 
Hi, everyone,

I upgraded from ISPC 3.0.4.6 (I think) to 3.0.5.2. Everything seemed to go smoothly and I was able to access the ISPC interface.

But some 10 to 15 minutes later, all hell seemed to break loose on this server. In particular, I began to notice several permissions-related problems, such as "immutable" flags being set on files within some of the websites stored in the /var/www directory. I ran "chattr -R -i /", which seemed to fix the immutable problems, but then I noticed others.

The first symptom I observed was this, in /var/log/apache2/error.log:

Code:

Warning: SuexecUserGroup directive requires SUEXEC wrapper.
I found the thread at http://www.howtoforge.com/forums/showthread.php?t=29912 and ran "apt-get install apache2-suexec", which fixed this problem.

Everything seemed to be well, until 10 to 15 minutes later when the same problem occurred again. So, again, I installed apache2-suexec, and it fixed the problem. How is this software being uninstalled? This server is a mediaTemple VPS, and I assume that some of the core system image components are shared among several instances, and they were doing maintenance on some servers (although, supposedly not this one) during this time. (I wonder if they broke something during the maintenance on other servers.)

But now, whenever I try to access the ISPC interface, I receive a server 500 response and see the following in /var/log/apache2/error.log:

Code:

[Sat May 11 14:43:52 2013] [notice] mod_fcgid: call /var/www/ispconfig/index.php with wrapper /var/www/php-fcgi-scripts/ispconfig/.php-fcgi-starter
/usr/bin/php-cgi: error while loading shared libraries: libz.so.1: cannot open shared object file: No such file or directory
[Sat May 11 14:43:52 2013] [warn] [client 74.75.234.205] (104)Connection reset by peer: mod_fcgid: error reading data from FastCGI server
[Sat May 11 14:43:52 2013] [error] [client 74.75.234.205] Premature end of script headers: index.php
[Sat May 11 14:43:58 2013] [notice] mod_fcgid: process /var/www/ispconfig/index.php(20179) exit(communication error), terminated by calling exit(), return code: 127

It seems that libz.so.1 is part of the zlib package, which is definitely installed. And, when I perform a "find" on that file, the same two files exist that exist on my other servers that don't have this problem:

Code:

find / -name "libz.so.1"
/var/www/clients/client1/web9/lib/libz.so.1
/var/www/clients/client1/web4/lib/libz.so.1
/usr/lib32/libz.so.1
/lib/libz.so.1

I also see the following, which I assume to occur whenever ISPC tries to do its scheduled monitor check:

Code:

[Sat May 11 14:45:01 2013] [error] [client 127.0.0.1] client denied by server configuration: /htdocs
I'm wondering if anybody might know what could have happened, and more importantly, how to fix it.

Thanks for any help here!

EDIT:

I was able to get back into the ISPC interface by commenting-out the SuexecUserGroup directive in /etc/apache2/sites-enabled/000-ispconfig.vhost:

Code:

<IfModule mod_fcgid.c>
    DocumentRoot /var/www/ispconfig/
    #SuexecUserGroup ispconfig ispconfig
    <Directory /var/www/ispconfig/>
      Options Indexes FollowSymLinks MultiViews +ExecCGI
      AllowOverride AuthConfig Indexes Limit Options FileInfo
      AddHandler fcgid-script .php
      FCGIWrapper /var/www/php-fcgi-scripts/ispconfig/.php-fcgi-starter .php
      Order allow,deny
      Allow from all
    </Directory>
    IPCCommTimeout  7200
        MaxRequestLen 15728640
  </IfModule>

Now I can log into ISPC. But I hesitate to change anything until I can figure out what happened to "break" ISPC in the first place.

Might anyone know how to fix this situation, which seems to be suexec-related?

EDIT 2:

I should mention that I have several other sites whose PHP mode is set to Fast-CGI with the SuExec box checked, and these sites always functioned, even when the ISPC virtual host went tits-up. And they continue to function now, yet their vhost files also contain the "SuexecUserGroup" directive:

Code:

<IfModule mod_suexec.c>
        SuexecUserGroup web14 client1
</IfModule>

Ultimately, I am wondering why using this directive causes the libz.so.1 failure when used in the ISPC vhost configuration, but seems not to cause problems anywhere else.

cbj4074 21st May 2013 01:13

Hate to do it, but:

Bump... nobody?

Why does the ISPConfig virtual host suffer from this problem after upgrading from 3.0.4.6 to 3.0.5.2 (in /var/log/apache2/error.log):

Code:

[Sat May 11 14:43:52 2013] [notice] mod_fcgid: call /var/www/ispconfig/index.php with wrapper /var/www/php-fcgi-scripts/ispconfig/.php-fcgi-starter
/usr/bin/php-cgi: error while loading shared libraries: libz.so.1: cannot open shared object file: No such file or directory
[Sat May 11 14:43:52 2013] [warn] [client 74.75.234.205] (104)Connection reset by peer: mod_fcgid: error reading data from FastCGI server
[Sat May 11 14:43:52 2013] [error] [client 74.75.234.205] Premature end of script headers: index.php
[Sat May 11 14:43:58 2013] [notice] mod_fcgid: process /var/www/ispconfig/index.php(20179) exit(communication error), terminated by calling exit(), return code: 127

Again, I'm able to "resolve" this problem by commenting-out this line in /etc/apache2/sites-enabled/000-ispconfig.vhost

Code:

#SuexecUserGroup ispconfig ispconfig
Do I need to copy the file libz.so.1 to some location other than those at which the file already exists (see my first post)?

And, again, I also see the following (in /var/log/apache2/error.log), which I assume to occur whenever ISPC tries to do its scheduled monitor check:

Code:

[Sat May 11 14:45:01 2013] [error] [client 127.0.0.1] client denied by server configuration: /htdocs
Any help is greatly appreciated.

till 21st May 2013 10:15

Quote:

In particular, I began to notice several permissions-related problems, such as "immutable" flags being set on files within some of the websites stored in the /var/www directory. I ran "chattr -R -i /", which seemed to fix the immutable problems, but then I noticed others.
Dont remove the immutable attribute on servers that allow access by ftp client to sites for your customers. The immutable bit is required for security reasons.

The php-fcgi problem that you have is not ispconfig related, the php binary /usr/bin/php-cgi requires libz and thats not installed or it does not has the correct version or similar. As this binary is not from ispconfig, it is from your linux distribution, there must be packages missing or dependency problems. You should consider to reinstall the php cgi / fcgi packages and their dependencies.

Quote:

I should mention that I have several other sites whose PHP mode is set to Fast-CGI with the SuExec box checked, and these sites always functioned, even when the ISPC virtual host went tits-up. And they continue to function now, yet their vhost files also contain the "SuexecUserGroup" directive:
Most likely suexec is not working in these sites then, the ifmodule means that suexec is used optionally when it is installed while the ispconfig vhost requires it. In oher wors, your sites may work, but with wrong permissions, while the ispconfig vhost can not work without suexec as its required for security reasons.

cbj4074 23rd May 2013 18:49

Hi, Till, and thanks for the thorough response.

Quote:

Dont remove the immutable attribute on servers that allow access by ftp client to sites for your customers. The immutable bit is required for security reasons.
I see. I didn't realize that this feature was added with ISPConfig 3.0.5. It's a relief to know that ISPC set the immutable flags intentionally. I won't remove them in the future.

Is there any way to re-apply these settings (such as toggling the feature on and off in the ISPC interface)?

Quote:

You should consider to reinstall the php cgi / fcgi packages and their dependencies.
What process do you recommend? Something like this?

Code:

apt-get install --reinstall php5 php5-dev php5-common php5-cli php5-cgi libapache2-mod-fcgid apache2-suexec libapache2-mod-suphp
From what I understand, using --reinstall doesn't touch any of the dependencies, though. And frankly, I'm a little hesitant to uninstall any dependencies, as some of them are critical, core packages (right?).

Maybe the first thing to try (before trying to reinstall packages) is to look at the phpinfo() output for the ISPC vhost and see what (if anything) appears related to zlib.

Thoughts? Thanks again for your help!

cbj4074 23rd May 2013 19:24

I created a file with a long, random name in the ISPC vhost, within the directory /usr/local/ispconfig/interface/web, and called phpinfo() therein. Not only is there a section for zlib, but it looks identical to the output on a nearly-identically-configured server where this issue does not occur:

Code:

zlib

ZLib Support        enabled
Stream Wrapper support        compress.zlib://
Stream Filter support        zlib.inflate, zlib.deflate
Compiled Version        1.2.1.1
Linked Version        1.2.3.3

Directive        Local Value        Master Value
zlib.output_compression        Off        Off
zlib.output_compression_level        -1        -1
zlib.output_handler        no value        no value

As I said earlier, zlib is definitely installed:

Code:

# aptitude search '~i' | grep "zlib"
i A libcompress-raw-zlib-perl      - low-level interface to zlib compression li
i  zlib1g                          - compression library - runtime
i A zlib1g-dev                      - compression library - development
i  zlibc                          - An on-fly auto-uncompressing C library

Or am I looking at/for the wrong package name(s) here?

And here is the filesystem information for the libz.so.1 file that is mentioned in the Apache error messages:

Code:

# find / -name "libz.so.1"
/var/www/clients/client1/web9/lib/libz.so.1
/var/www/clients/client1/web4/lib/libz.so.1
/usr/lib32/libz.so.1
/lib/libz.so.1

# ls -lah /var/www/clients/client1/web9/lib/libz.so.1
lrwxrwxrwx 1 web9 client1 15 Jan 26  2011 /var/www/clients/client1/web9/lib/libz.so.1 -> libz.so.1.2.3.3

# ls -lah /var/www/clients/client1/web4/lib/libz.so.1
lrwxrwxrwx 1 root root 15 Nov 16  2010 /var/www/clients/client1/web4/lib/libz.so.1 -> libz.so.1.2.3.3

# ls -lah /usr/lib32/libz.so.1
lrwxrwxrwx 1 root root 15 May 24  2010 /usr/lib32/libz.so.1 -> libz.so.1.2.3.3

# ls -lah /lib/libz.so.1
lrwxrwxrwx 1 root root 15 Oct  5  2010 /lib/libz.so.1 -> libz.so.1.2.3.3

# find / -name "libz.so.1.2.3.3"
/var/www/clients/client1/web9/lib/libz.so.1.2.3.3
/var/www/clients/client1/web4/lib/libz.so.1.2.3.3
/usr/lib32/libz.so.1.2.3.3
/lib/libz.so.1.2.3.3

Is there any possibility that the ISPC vhost is looking for this file in some other location? I see that a symbolic link to this file is created in each vhost's directory (for example, /var/www/clients/client1/web9/lib/libz.so.1.2.3.3, above). Does a symbolic link to this file also need to exist somewhere within the ISPC vhost's directory?

Is my only option to start reinstalling packages?

till 23rd May 2013 19:47

Quote:

Is there any way to re-apply these settings (such as toggling the feature on and off in the ISPC interface)?
Please see resync tool in the tools module.

Quote:

From what I understand, using --reinstall doesn't touch any of the dependencies, though. And frankly, I'm a little hesitant to uninstall any dependencies, as some of them are critical, core packages (right?).
yes.

Quote:

Is there any possibility that the ISPC vhost is looking for this file in some other location?
this is aphp dependency, not a ispconfig dependency. The locations are compiled into the php packages of the linux distribution, so nithing that ispconfifg is or even can influence.

cbj4074 23rd May 2013 20:32

Thanks, Till. I understand.

I will wait until the risk of downtime is acceptable on the server in question and try to troubleshoot further. In case others run into this problem, I'll post my findings here.


All times are GMT +2. The time now is 00:14.

Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2014, vBulletin Solutions, Inc.