Go Back   HowtoForge Forums | HowtoForge - Linux Howtos and Tutorials > ISPConfig 3 > Developers' Forum

Do you like HowtoForge? Please consider supporting us by becoming a subscriber.
Reply
 
Thread Tools Display Modes
  #1  
Old 18th June 2011, 17:09
erosbk erosbk is offline
Senior Member
 
Join Date: Mar 2011
Posts: 337
Thanks: 49
Thanked 36 Times in 30 Posts
Default FTP traffic - Help with php code

I posted yesterday a piece of code for calculate ftp traffic in the tasklist (it will update ftp traffic every minute, like web traffic).

I need to know right now, which is the best way to read and show that information to user. I have a multiserver enviroment, and ftp log is in the same server that ISPConfig main is running...

But if my ftp/site server is in a different server (ispconfig in another), how can I read it remotely? what is the best way? do I have to update always the table of ispconfig main server, copying date from ftp server?

http://bugtracker.ispconfig.org/inde...otes&sort=desc

Thanks!
Reply With Quote
Sponsored Links
  #2  
Old 20th June 2011, 12:01
till till is offline
Super Moderator
 
Join Date: Apr 2005
Location: Lüneburg, Germany
Posts: 35,504
Thanks: 813
Thanked 5,265 Times in 4,129 Posts
Default

Data can not be read from the master remotely on the slave. For that reason, you will have to run a script on the slave that updates the traffic data in the mysql database of the master server.
__________________
Till Brehm
--
Get ISPConfig support and the ISPConfig 3 manual from ispconfig.org.
Reply With Quote
  #3  
Old 20th June 2011, 17:52
erosbk erosbk is offline
Senior Member
 
Join Date: Mar 2011
Posts: 337
Thanks: 49
Thanked 36 Times in 30 Posts
Default

Ok till, is should be easy. How can I detect in a slave which is the master server?, or in ispconfig, which ones are the variables used to connect to master server?
Reply With Quote
  #4  
Old 20th June 2011, 18:15
till till is offline
Super Moderator
 
Join Date: Apr 2005
Location: Lüneburg, Germany
Posts: 35,504
Thanks: 813
Thanked 5,265 Times in 4,129 Posts
Default

Please see ispconfig configuration file /usr/local/ispconfig/server/lib/config.inc.php. The mysql master login details are in there. Please be aware that the mysql user has only write access to the monitor_data and web traffic table. If you need access to your table as well, you will have to grant specific permissions for that table in the installer. Do not grant access to all tables as this would cause security problems when a slave gets hacked!
__________________
Till Brehm
--
Get ISPConfig support and the ISPConfig 3 manual from ispconfig.org.
Reply With Quote
  #5  
Old 21st June 2011, 20:26
erosbk erosbk is offline
Senior Member
 
Join Date: Mar 2011
Posts: 337
Thanks: 49
Thanked 36 Times in 30 Posts
Default

Thanks Till. Please give me advise. I am coding this for ispconfig, not just for me. So, is any chance that the code become part of ispconfig or you will writte your own for this? I am trying to consider it for multiserver and for different distros (but I think that I will not have change to test in all of them...). The code will be posted in the tasklist of course.

thank you
Reply With Quote
  #6  
Old 22nd June 2011, 11:58
till till is offline
Super Moderator
 
Join Date: Apr 2005
Location: Lüneburg, Germany
Posts: 35,504
Thanks: 813
Thanked 5,265 Times in 4,129 Posts
Default

You code can be added to ispcongig main version if it works well. Here a few things that might be worth to consider.

1) Shall the statistics aggregated by FTP user or by website? If we want to show them by website, then it might be better to add a column to the web_stats table and insert the traffic data there, so we dont need a new database table then and dont have to deal with the extra permissions during setup. This would also enable is to show the statistics as part of the website traffic stats easily without the need of a separate list.

2) In your script you read the ftp log in one chunk with the file() function. This function loads the complete file into memory in one step. If you have a lrge ftp file (e.g. 1 GB, then this will fail. I thin its better to use this approach to go trough all log lines which can deal with any file size:

$handle = fopen ("/tmp/inputfile.txt", "r");
while (!feof($handle)) {
$buffer = fgets($handle, 4096);
echo $buffer;
}
fclose ($handle);

so you process the lines inside the while loop.

3) It might be nescessary to add some code that ensures that you dont read the ftp traffic in twice e.g. by storing the date / time of the last processed ftp log line in a file and when the script is executed again, the skip all ftp log lines where the date is older then the date of that last log line.
__________________
Till Brehm
--
Get ISPConfig support and the ISPConfig 3 manual from ispconfig.org.
Reply With Quote
  #7  
Old 22nd June 2011, 16:51
erosbk erosbk is offline
Senior Member
 
Join Date: Mar 2011
Posts: 337
Thanks: 49
Thanked 36 Times in 30 Posts
Default

Thanks till, I have little knowledge in programming (well... not so little), but I am new in php.

Addin ftp traffic to web site traffic will be easy. But in this case, with ftp traffic by user, if there are two or more users per site, clients could see who is using the most traffic. If you limit traffic per month, I think that this approach would be usefull for clients.

It is up to you to decide which one is the better approach! I will wait your answer before starting coding. I want to make this for ispconfig, not just for me!

Anyway, in my script I am saving in table every last timestamp per using, calculating the max in next run and comparing it with each log line to know if that line must be added or not to the table... it is not the best... I think that using a file should be the best... but, where to save that file? is a directory in ispconfig for this? does it have a table for this?

thanks again

edit: I think that adding a new table will be the best... you can get a lot of information with it... is good for both clients and users, in every approach... site traffic is site traffic... ftp traffic is ftp traffic...

Last edited by erosbk; 22nd June 2011 at 16:56.
Reply With Quote
  #8  
Old 22nd June 2011, 18:01
erosbk erosbk is offline
Senior Member
 
Join Date: Mar 2011
Posts: 337
Thanks: 49
Thanked 36 Times in 30 Posts
Default

Add ispconfig system table (multiuse to store values of different types, for different modules that could need it in a future)
- With this, you can save a variable value, needed to be updated from time to time:
- ( ftp_traffic, traffic_timestamp, "", 0, 1234512345 ) (instead of saving data to a file if needed, or saving a value

multiple times in differents columns as it was my original idea...)

sys_vars
* var_module (char)
* var_name (char)
* var_string (varchar)
* var_integer (bigint)
* var_timestamp (timestamp)

Add ftp_traffic table

ftp_traffic
* hostname
* ftp_user
* traffic_date
* traffic_type
* traffic_bytes

Basically, this should be the program (in pseudo-pseudo code :P)

If server = FTP server (where to obtain this from a slave??)

read last update from sys_vars.var_timestamp and save in lasttimestamp
read ftplogfile

while !eof(ftplogfile)

if ftplogfile.line.timestamp > lasttimestamp

updatearray[] = ftplogfile.line.values, hostname of ftplogfile.line.ftp_user
x = x + 1
endif

if x = 10
call addtraffic
x = 0
end if
end while

if x > 0
call addtraffic
end if


function addtraffic
begin transaction
if exists in ftp_traffic
update ftp_traffic (values and hostname)
else
insert ftp_traffic (values and hostname)
endif
update sys_vars.var_timestamp with updatearray[].timestamp
end transaction
end function

end if

I hope you understand my idea, sorry for my poor and limited english...
Reply With Quote
  #9  
Old 22nd June 2011, 18:45
till till is offline
Super Moderator
 
Join Date: Apr 2005
Location: Lüneburg, Germany
Posts: 35,504
Thanks: 813
Thanked 5,265 Times in 4,129 Posts
Default

ISPConfig has already a config system to store values, so please do not duplicate that.

Using a separate ftp traffic table is ok, even if I would prefer to use the existing web_traffic table.
__________________
Till Brehm
--
Get ISPConfig support and the ISPConfig 3 manual from ispconfig.org.
Reply With Quote
  #10  
Old 22nd June 2011, 20:05
erosbk erosbk is offline
Senior Member
 
Join Date: Mar 2011
Posts: 337
Thanks: 49
Thanked 36 Times in 30 Posts
 
Default

Ok, where in ispconfig should I save the last timestamp of the logfile? with your answer you will save me a lot of work xD
Reply With Quote
Reply

Bookmarks

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Amavisd not identfying spam properly Cracklefish Installation/Configuration 11 22nd December 2009 11:05
Freebsd 6.1 support misterm Installation/Configuration 10 9th April 2009 09:29
ftp is very slow and asks for user/passwd continuously chico58 Installation/Configuration 7 8th October 2008 12:43
How to add a subdomain? malinens Installation/Configuration 2 10th March 2008 23:16
Apache2 Freezes celtic Server Operation 31 28th May 2007 17:18


All times are GMT +2. The time now is 09:04.


Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2014, vBulletin Solutions, Inc.