Go Back   HowtoForge Forums | HowtoForge - Linux Howtos and Tutorials > ISPConfig 3 > General

Do you like HowtoForge? Please consider supporting us by becoming a subscriber.
Reply
 
Thread Tools Display Modes
  #1  
Old 3rd August 2010, 07:35
FeraTechInc FeraTechInc is offline
Senior Member
 
Join Date: Feb 2007
Posts: 138
Thanks: 3
Thanked 5 Times in 5 Posts
Default Server Scaling Issue

So I am having several issues which are getting worse as we expand the number of sites and users on our servers.

We currently have 3 servers with about 3-10 sites per server and about 30-40 e-mail accounts.

All of our servers are Ubuntu 10.4 with Raid 5 or Raid 1 configurations.

With almost all of our accounts the number of e-mails has quickly exceeded past 3000. The e-mails are simply listed in the directory. Then we have several file uploading sites with the same issues which we are working on.

The problem now is all of our servers are taxing our drives and everything is slowing down.

With a linux file system it is recommended not to have more than 1000 files or folder listed per directory. With our sites this is an easy fix and files uploaded are categorized by month. However, with e-mails this is a growing problems and size limits do not easily fix this.

Is there any way to have e-mails organized into folders as well?
Reply With Quote
Sponsored Links
  #2  
Old 3rd August 2010, 09:25
till till is offline
Super Moderator
 
Join Date: Apr 2005
Location: Lüneburg, Germany
Posts: 36,405
Thanks: 834
Thanked 5,496 Times in 4,326 Posts
Default

Not sure if this is really your issue. The maildir format handles large amounts of email very well as new email get always put into a separate folder until they get fetched and also all email are indexed and this index is used by the pop3 / imap daemon to get information about the email, so the files dont get listed in the filesystem when someone accesses the mailbox. Additionally, your setup is quite small with just 3000 email per mailbox, so I doubt that this is the issue

1) Which filesystem do you use for the email partition?
2) Which pop3 / imap daemon do you use?
3) How high is the system load (use top or uptime)
4) Do you use a virtualisation software on this server?
5) Is this a software or hardware raid?
__________________
Till Brehm
--
Get ISPConfig support and the ISPConfig 3 manual from ispconfig.org.
Reply With Quote
  #3  
Old 8th August 2010, 11:18
FeraTechInc FeraTechInc is offline
Senior Member
 
Join Date: Feb 2007
Posts: 138
Thanks: 3
Thanked 5 Times in 5 Posts
 
Default

The server uses software raid 5. There is no virtualization on the server. However, basic graphic desktop packages are installed and courier is used for mail. The "Ubuntu Perfect Server 9.10" with all values set to what the recommended setup should be.

The file system is broken up in the following way.

3 drives

1gig boot (raid1) | 10 gig swap (raid1) | 480 gig / root (raid5)
1gig boot (raid1) | 10 gig swap (raid1) | 480 gig / root (raid5)
1gig boot (raid1) | 10 gig swap (raid1) | 480 gig / root (raid5)

However, we have figured out the main cause of our issues.

It seems to be a problem stemming from the setup of our mysql database.

For those interested I feel like this script is a must!


Download MySQLTuner using the following command
wget http://mysqltuner.com/mysqltuner.pl
To run the script, simply make it executable and run it:
chmod +x mysqltuner.pl
sudo ./mysqltuner.pl

It will give you options for tuning your database.
We had queries that were so intensive that they completely took over the mysql server for 30 seconds at a time. I didn't actually realize each time a user logs in through IMAP their authentication needed to be queried through the mysql server.
Reply With Quote
Reply

Bookmarks

Tags
scaling, server

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Centos5.4/ISPConfig 3--Virtual site not working MichaelCaditz Installation/Configuration 25 25th March 2011 11:37
Email problem 'Cannot set my user or group id.' (using ISPConfig 3 + OpenSuSE 11.2) urosm Installation/Configuration 5 19th June 2010 22:41
Problem with mail server and subdomains MyKE General 7 26th February 2010 10:18
Can't start apache Musty Server Operation 12 9th March 2008 13:58
Administrator receive empty mail ??? Stanev General 16 7th April 2006 20:56


All times are GMT +2. The time now is 05:48.


Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2014, vBulletin Solutions, Inc.