Go Back   HowtoForge Forums | HowtoForge - Linux Howtos and Tutorials > Linux Forums > HOWTO-Related Questions

Do you like HowtoForge? Please consider supporting us by becoming a subscriber.
Reply
 
Thread Tools Display Modes
  #1  
Old 5th May 2009, 22:36
wxman wxman is offline
Senior Member
 
Join Date: May 2007
Posts: 189
Thanks: 11
Thanked 2 Times in 2 Posts
Default HAProxy keeps switching between servers

I have HAProxy and Heartbeat running as my load balancer/fail over. I used the how-to here to set it up. I noticed that when I refresh a page, it reloads using the opposite server's file. I could tell because I temporarily shut off my rsync to test what was happening. I thought HAProxy would set a cookie (yes the browser allows them) to keep you on the server you started on as long as it was still running? I tested this in more than one browser, and it still happens.

On a similar note; why would I keep needing to restart heartbeat on the load balancer so I can see the HAProxy stats page? frequently, if I go to the stats page, it's not there. I check, and Heartbeat is running, and if I restart it, the the stats return. I don't see anything in the logs that would answer this except every time it happens, I see that it's looking for it on the web server, not the load balancer:
Code:
File does not exist: /var/www/apache2-default/haproxy

Last edited by wxman; 6th May 2009 at 15:18.
Reply With Quote
Sponsored Links
  #2  
Old 6th May 2009, 23:07
falko falko is offline
Super Moderator
 
Join Date: Apr 2005
Location: Lneburg, Germany
Posts: 41,701
Thanks: 1,900
Thanked 2,735 Times in 2,571 Posts
Default

I've never had such problems with HAProxy and heartbeat. Please double-check your setup - maybe you made a typo somewhere?
__________________
Falko
--
Download the ISPConfig 3 Manual! | Check out the ISPConfig 3 Billing Module!

FB: http://www.facebook.com/howtoforge

nginx-Webhosting: Timme Hosting | Follow me on:
Reply With Quote
  #3  
Old 7th May 2009, 13:50
wxman wxman is offline
Senior Member
 
Join Date: May 2007
Posts: 189
Thanks: 11
Thanked 2 Times in 2 Posts
Default

I've checked them many times now. I even went through and completely redid them. I've also done a lot of Googling the the two problems, and I haven't found anyone else talking about it.
Reply With Quote
  #4  
Old 8th May 2009, 22:26
wtarreau wtarreau is offline
Junior Member
 
Join Date: May 2009
Posts: 4
Thanks: 0
Thanked 0 Times in 0 Posts
Default

The fact that the stats page is forwarded to the servers happens because you forgot to set "option httpclose" in haproxy's config, so a connection to your web site is maintained between the client and the server and haproxy does not see subsequent requests for the stats in it. I'm not sure why restarting heartbeat fixes the issue, that may simply be a side effect of breaking the connection to the browser.
Reply With Quote
  #5  
Old 10th May 2009, 13:29
wxman wxman is offline
Senior Member
 
Join Date: May 2007
Posts: 189
Thanks: 11
Thanked 2 Times in 2 Posts
Default

Quote:
Originally Posted by wtarreau View Post
The fact that the stats page is forwarded to the servers happens because you forgot to set "option httpclose" in haproxy's config, so a connection to your web site is maintained between the client and the server and haproxy does not see subsequent requests for the stats in it. I'm not sure why restarting heartbeat fixes the issue, that may simply be a side effect of breaking the connection to the browser.
I just checked, and option httpclose is set in my cfg:
Code:
global
        log 127.0.0.1   local0
        log 127.0.0.1   local1 notice
        #log loghost    local0 info
        maxconn 4096
        #debug
        #quiet
        user haproxy
        group haproxy

defaults
        log     global
        mode    http
        option  httplog
        option  dontlognull
        retries 3
        redispatch
        maxconn 2000
        contimeout      5000
        clitimeout      50000
        srvtimeout      50000

listen webfarm 192.168.31.100:80
       mode http
       stats enable
       stats auth netadmin:5bgr+bdd1WbA
       balance roundrobin
       cookie JSESSIONID prefix
       option httpclose
       option forwardfor
       option httpchk HEAD /check.txt HTTP/1.0
       server web1 192.168.31.202:80 cookie w01 check inter 2000 rise 2 fall 2
       server web2 192.168.31.212:80 cookie w02 check inter 2000 rise 2 fall 2
       option persist
       redispatch
       contimeout 5000
Maybe that will help. I thought I went over it enough, and didn't see any problems.
Reply With Quote
  #6  
Old 17th May 2009, 15:00
wxman wxman is offline
Senior Member
 
Join Date: May 2007
Posts: 189
Thanks: 11
Thanked 2 Times in 2 Posts
Default

I've been doing more testing and research on what's going on here.
I use http://192.168.31.100/haproxy?stats to get to that stats page. The .100 is the shared address between the load balancers. If I use .201, which is LB1, I get the browser's 404 notice. If I use .100, it shows my apache generic 404 page. So somehow it stops seeing LB1, and goes to port 80 on my web server on the web1 node. That's where I see the apache error saying it can't find the HAProxy stats page.
I've never used tcpdump before, so I had to do some quick research first. I reset the node again to make things work. When I used the "tcpdump -q -i eth0 tcp port 80 and src host 192.168.31.100" command, it showed me looking at the stats, and the test web page:
Code:
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes
11:23:16.106664 IP 192.168.31.100.www > 192.168.30.64.2289: tcp 0
11:23:16.254209 IP 192.168.31.100.www > 192.168.30.64.2289: tcp 0
11:23:16.254409 IP 192.168.31.100.www > 192.168.30.64.2289: tcp 262
11:23:16.254501 IP 192.168.31.100.www > 192.168.30.64.2289: tcp 0
11:23:17.460534 IP 192.168.31.100.www > 192.168.30.64.2290: tcp 0
11:23:17.628385 IP 192.168.31.100.www > 192.168.30.64.2290: tcp 0 11:23:17.628590 IP 192.168.31.100.www > 192.168.30.64.2290: tcp 2712
11:23:17.839448 IP 192.168.31.100.www > 192.168.30.64.2290: tcp 2712 11:23:17.839460 IP 192.168.31.100.www > 192.168.30.64.2290: tcp 524
Once I couldn't see the stats page again, the tcpdump output stopped. I watched it on LB2 as well. It seems like it stops listing to the .100 IP address. If I use "tcpdump -q -i eth0 tcp port 80" I see LB1 checking web1 and web2, but nothing on the .100 address. If I was running a name server, I'd say that's the problem, but I'm not.
Reply With Quote
  #7  
Old 27th August 2009, 02:08
astroman1 astroman1 is offline
Junior Member
 
Join Date: Aug 2009
Posts: 4
Thanks: 0
Thanked 1 Time in 1 Post
Default

Quote:
Originally Posted by wxman View Post
I have HAProxy and Heartbeat running as my load balancer/fail over. I used the how-to here to set it up. I noticed that when I refresh a page, it reloads using the opposite server's file. I could tell because I temporarily shut off my rsync to test what was happening. I thought HAProxy would set a cookie (yes the browser allows them) to keep you on the server you started on as long as it was still running? I tested this in more than one browser, and it still happens.
Did you ever figure out why this was happening? I am seeing this. I have a page which loads 8 files (html, images, javascript and css), and the HTTP GET requests are split alternately between the two web servers?
Reply With Quote
  #8  
Old 31st August 2009, 19:27
astroman1 astroman1 is offline
Junior Member
 
Join Date: Aug 2009
Posts: 4
Thanks: 0
Thanked 1 Time in 1 Post
Default

I found that mine was not working because of the line
Quote:
cookie JSESSIONID prefix
I had to change it to
cookie JSESSIONID insert

as the JSESSIONID cookie was not previously sent from my site. The haproxy server data could not be prefixed to the cookie and the server data was not sent to the client, All traffic from the client did not have the server tag and every request went to alternate servers.

Note that the cookie name above (JSESSIONID) should be unique and not be the same as any cookie emitted by your server since haproxy will replace it with its own if cookie insert is used.
Reply With Quote
The Following User Says Thank You to astroman1 For This Useful Post:
sufehmi (20th October 2010)
  #9  
Old 20th October 2010, 13:49
sufehmi sufehmi is offline
HowtoForge Supporter
 
Join Date: May 2009
Posts: 4
Thanks: 2
Thanked 0 Times in 0 Posts
Default Thank You !

Quote:
Originally Posted by astroman1 View Post
I had to change it to
cookie JSESSIONID insert

as the JSESSIONID cookie was not previously sent from my site. The haproxy server data could not be prefixed to the cookie and the server data was not sent to the client, All traffic from the client did not have the server tag and every request went to alternate servers.
I logged on just to say THANKS - this is spot on

Reading all over HAproxy documentation at 3 am sure doesn't help me finding the information I needed.
Thankfully, Google never felt sleepy - and correctly pointed me to this post of yours.


Quote:
Note that the cookie name above (JSESSIONID) should be unique and not be the same as any cookie emitted by your server since haproxy will replace it with its own if cookie insert is used.
Another BULLSEYE, I was using PHPSESSID and wondered why it didn't work.

So I changed it to a unique cookie name, and voila -- session-aware load-balancer !

Thanks again !!


Cheers, HS
Reply With Quote
  #10  
Old 17th October 2011, 16:40
deepakpalmca deepakpalmca is offline
Junior Member
 
Join Date: Oct 2011
Posts: 1
Thanks: 0
Thanked 0 Times in 0 Posts
 
Default Not serving request in round robin fashion.

Hey,

I have confiured haproxy to work in a roundrobin fashion with two backend servers and enabled session persistence. But it's serving most of the requests from appserver1 only rarely it's going to appserver2 like once in a dozen.


my configuration is.

haproxy.cfg:

global
maxconn 50000
user haproxy
group haproxy
daemon
#number of processing core use 3
nbproc 3
pidfile /tmp/haproxy.pid
defaults
mode http
clitimeout 10000
srvtimeout 10000
contimeout 10000
#Disable keepalive
option httpclose
log global
log 127.0.0.1 local3
option httplog
# option http-server-close

listen http_proxy 127.0.0.1:80
#Balancing Algorithm
balance roundrobin
#Use this to chek the status of servers on stats
# option httpcheck HEAD /check.txt HTTP/1.0
#This option will set x-forward-for
option forwardfor
stats enable
#check this once again
stats uri /ha_stats
stats realm Global\ Haproxy
stats auth analytix:analytix
stats refresh 30s
# stats hide-version
# option httpclose
capture cookie JSESSIONID len 52
#check this
# stick-table type string len 52 size 2g expire 5h
cookie HTTPSERVERID insert nocache indirect
# appsession JSESSIONID len 52 timeout 5h mode query-string
appsession JSESSIONID len 52 timeout 5h

#Our Backend servers entry by command line arguments

server appServer1 204.232.206.201:80 weight 1 maxconn 15000 check inter 2000 rise 2 fall 3 cookie A

server appServer2 204.232.206.221:80 weight 1 maxconn 15000 check inter 2000 rise 2 fall 3 cookie A




Regards,
Deepak
Reply With Quote
Reply

Bookmarks

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
How to host a forum with multiple web servers? itwillcome Technical 3 11th February 2011 01:26
Apache handeling SSL requests and passing them through to HAproxy gamboni Installation/Configuration 5 3rd September 2009 04:48
switching servers, also want to go from ISPCFG2 to ISPCFG3 Ovidiu General 11 7th May 2009 13:54
Unable send receive emails vassilis3 Installation/Configuration 15 19th May 2007 14:34
No SPF record. beryl Installation/Configuration 6 17th May 2007 19:52


All times are GMT +2. The time now is 19:13.


Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2014, vBulletin Solutions, Inc.