HAProxy Load Balancer with SSL Offloading

In my previous blog, where I mentioned how to use HA Proxy(http://haproxy.1wt.eu/ – Open source Load balancing Solution) based load balancer for Exchange 2010 (Link to Post), I had a few requests on how to do the SSL offloading as well. If you have followed that guide, you have your SSL certificates hosted on the Exchange Hub CAS servers.
Microsoft does suggest to use the SSL on the boxes, but you can offload it as well, please remember that I am writing this for a generic https server and we will be using 2 open source products.
We will use HA Proxy for the Load balancing and the Pound for the SSL offloading. Now, for this purposes, I have used a single box, but for production machines, you may want to make a HA Pair of Linux box, which I will cover in a different post.
I have been running HA Proxy in the exchange environment and it is great, the only thing, I could ask for is source port based persistence Smile.
Anycase, back to the topic. Here is how the architecture will look like.
image

As you can see the Pound will do the SSL offload and send the traffic to HA Proxy and HA Proxy will do the load balancing.
Don’t get me wrong guys, pound is capable of sending the traffic directly to the servers, but then you will not be able to use the cool features of ha proxy like cookie persistence, so on and so forth.

In our test environment, here is what we will do.
Client Facing IP: 192.168.10.10
Server IP Address: 10.10.10.11, 10.10.10.12
The Linux Box is 192.168.10.9

First, install Ubuntu (http://www.ubuntu.com/download/ubuntu/download)  on the box.
Add the 192.168.10.10 as a Virtual IP on the box (If you are using a Active standby setup, this will be the floating IP)
Edit the interfaces script in Ubuntu)
vi /etc/network/interfaces

auto eth0
iface eth0 inet static
address 192.168.10.9
netmask 255.255.255.0
network 192.168.10.0
broadcast 192.168.10.255
gateway 192.168.10.1
auto eth0:1
iface eth0:1 inet static
address 192.168.10.10
netmask 255.255.255.0
network 192.168.10.0
broadcast 192.168.10.255
gateway 192.168.10.1



Restart the Networking subsystem
/etc/init.d/networking restart

Once that is done, the IP addresses will be shown when you execute “ifconfig –a” command.

Then install the following packages

   1: sudo apt-get install haproxy pound

This will install the haproxy and pound,

Now, lets roll

Configuring pound:


In order to configure pound, you will need to edit the config file using vi (or editor of your choice)

vi /etc/pound/pound.cfg

The file should look like the following. Please note, that I have installed the server certificate as PEM on the system.


## global options:
User		"www-data"
Group		"www-data"
#RootJail	"/chroot/pound"
## Logging: (goes to syslog by default)
##	0	no logging
##	1	normal
##	2	extended
##	3	Apache-style (common log format)
LogLevel	1
## check backend every X secs:
Alive		30
# Creating Listener
ListenHTTPS
Address 192.168.10.10
Port    443
Cert    "/etc/pound/testdom.pem"
Client  20
End
Service
HeadRequire "Host:.*testdomain.com*"
BackEnd
Address 127.0.0.2
Port 80
End



So, as you can see, it’s a generic configuration and then it is using the 127.x.x.x subnet to talk to the HA Proxy, the HAProxy will be listening on the particular IP and port.

After the configuration is done, go ahead and restart pound.

/etc/init.d/pound restart

The HA Proxy configuration will look like this


   1: global
   2:         #uid 99
   3:         #gid 99
   4:         daemon
   5:         stats socket /var/run/haproxy.stat mode 600 level admin
   6:         maxconn 40000
   7:         ulimit-n 81000
   8:         pidfile /var/run/haproxy.pid
   9: defaults
  10:         mode    http
  11:         contimeout      4000
  12:         clitimeout      3600000
  13:         srvtimeout      3600000
  14:         balance roundrobin
  15: listen  ServerSSL 127.0.0.2:80
  16:         mode    tcp
  17:         option  persist
  18:         balance leastconn
  19:         stick-table type ip size 10240k expire 15m
  20:         stick on src
  21:         server Server1 10.10.10.11 weight 1 check port 80 inter 5000 rise 2 fall 3
  22:         server Server2 10.10.10.11 weight 1 check port 80 inter 5000 rise 2 fall 3
  23:         option redispatch
  24:         option abortonclose
  25:         maxconn 40000
  26: listen  stats :7000
  27:         stats   enable
  28:         stats   uri /
  29:         option  httpclose
  30:         stats   auth username:password

This might not seem like worth it, as this is for a single HTTPs client, but let me go ahead and do the same thing for exchange.

Here is how the Pound configuration and HA Proxy configuration will look for Exchange 2010, with SSL offload.


Pound and HAProxy for Exchange 2010

Pound Configuration:


   1: ## Pound Configuration
   2: User        "www-data"
   3: Group        "www-data"
   4: ## Logging: (goes to syslog by default)
   5: ##    0    no logging
   6: ##    1    normal
   7: ##    2    extended
   8: ##    3    Apache-style (common log format)
   9: LogLevel    1
  10: ## check backend every X secs:
  11: Alive        30
  12:               ListenHTTP
  13:                   Address 192.168.10.10
  14:                   Port    80
  15:                   Client  10
  16:               End
  17:               ListenHTTPS
  18:                   Address 192.168.10.10
  19:                   Port    443
  20:                   Cert    "/etc/pound/mailserver.pem"
  21:                   Client  20
  22:               End
  23:               Service
  24:                   HeadRequire "Host:.*mail.domain.com*"
  25:                   BackEnd
  26:                       Address 127.0.0.5
  27:                       Port    80
  28:                   End

HA Proxy Configuration:


   1: global
   2:         #uid 99
   3:         #gid 99
   4:         daemon
   5:         stats socket /var/run/haproxy.stat mode 600 level admin
   6:         maxconn 40000
   7:         ulimit-n 81000
   8:         pidfile /var/run/haproxy.pid
   9: defaults
  10:         mode    http
  11:         contimeout      4000
  12:         clitimeout      3600000
  13:         srvtimeout      3600000
  14:         balance roundrobin
  15: listen  Exchange2010-HTTP-HTTPS 127.0.0.5:80
  16:         mode    tcp
  17:         option  persist
  18:         balance roundrobin
  19:         stick-table type ip size 10240k expire 30m
  20:         stick on src
  21:         server HC-CAS1 10.10.10.11:80 weight 1 check port 80 inter 5000 rise 2 fall 3
  22:         server HC-CAS2 10.10.10.12:80 weight 1 check port 80 inter 5000 rise 2 fall 3
  23:         option redispatch
  24:         option abortonclose
  25:         maxconn 40000
  26: listen  Exchange2010 192.168.10.10:25
  27:         bind 192.168.10.10:110,192.168.10.10:135
  28:         bind 192.168.10.10:139,192.168.10.10:443
  29:         bind 192.168.10.10:60000,192.168.10.10:60001
  30:         bind 192.168.10.10:6001-6004
  31:         bind 192.168.10.10:993-995
  32:         mode    tcp
  33:         option  persist
  34:         balance roundrobin
  35:         stick-table type ip size 10240k expire 30m
  36:         stick on src
  37:         server HC-CAS1 10.10.10.11 weight 1 check port 80 inter 5000 rise 2 fall 3
  38:         server HC-CAS2 10.10.10.12 weight 1 check port 80 inter 5000 rise 2 fall 3
  39:         option redispatch
  40:         option abortonclose
  41:         maxconn 40000
  42: listen  stats :7000
  43:         stats   enable
  44:         stats   uri /
  45:         option  httpclose
  46:         stats   auth username:password

Please note, that you will still have to make those changes on the Exchange server as noted in my earlier blog post.

If you need explanation on any of the configuration components, do ask (for the HA proxy they are all explained in my previous blog post)

I hope you find this informative and would like to thank you for reading.

Comments

  1. Thunder Emperor,

    Keep up the great work on your blog. I am particular interested in a blog post on HAProxy which you previously mentioned "I have used a single box, but for production machines, you may want to make a HA Pair of Linux box, which I will cover in a different post."

    When do you think you will find time in your busy schedule to make this blog post on a pair of Linux HAProxy servers ?

    Your blogs are clear and easy to understand.
    Thanks,

    ReplyDelete
  2. Sure Bruce ... I will write the HA Pair one next ... Once I have some time ...

    Thanks for reading and hope it helps

    ReplyDelete
  3. Amazing blogs, thanks for the time and effort that you have put in to these.

    If only you have made them sooner.
    I just made a lab a while ago and was force into using a HW loadbalancer on trail to make it work.

    Guess what im replacing them with when it expires.

    If you also do SQL i would love to see a post about distributed transactions servers using HAproxy.

    ReplyDelete
  4. Morfien .... Thanks, I will write one on MySQL that I created for multiple master.

    I dont know if you have noticed, this load balncing stuff is written in 3 parts for Exchange 2010 ... All of them focus on a different aspect

    Part 1: Basic
    http://3-4-5-6.blogspot.com/2011/03/ha-proxy-for-exchange-2010-deployment.html

    Part 2: Including SSL Offloading
    http://3-4-5-6.blogspot.com/2011/08/haproxy-load-balancer-with-ssl.html

    Part 3: Redundancy
    http://3-4-5-6.blogspot.com/2011/11/haproxy-for-exchange-2010part-2.html


    Hope this helps ...

    ReplyDelete
  5. I did notice thanks.

    I'm in the process of creating a MSSQL multi-master setup and am planning on using HAproxy as the load balancer.

    we'll see how that goes :)

    ReplyDelete
  6. I am trying to find your example for cookies in this article. You mention it as an option when using Pound, but I am not seeing any of the HAProxy cookie config options in your HA Proxy configuration example. Are they enabled by default if the data has been unencrypted?

    ReplyDelete
    Replies
    1. Hi Leon,

      In this example, I have used the source IP persistence

      If you want to use cookie persistence then change the lines 15 down


      15: listen Exchange2010-HTTP-HTTPS 127.0.0.5:80
      16: mode http
      17: option httpclose
      18: balance roundrobin
      19: cookie exchangecookie insert
      20: server HC-CAS1 10.10.10.11:80 cookie exchangecookie_HC-CAS1 weight 1 check port 80 inter 5000 rise 2 fall 3
      21: server HC-CAS2 10.10.10.12:80 cookie exchangecookie_HC-CAS2 weight 1 check port 80 inter 5000 rise 2 fall 3
      22: option redispatch
      23: option abortonclose
      24: maxconn 40000


      I guess its self explanatory ... let me know if you need any more assistance

      Delete
  7. Hi Thunder Emperor,

    You mentioned pem certificate is this a exchange server certificate or is it a self signed certificate.

    Thanks,
    Ramana

    ReplyDelete
    Replies
    1. Hi Ram,

      Sorry for the delay in response. This is the Exchange Server Certificate which needs to be offloaded.

      Delete
  8. This comment has been removed by a blog administrator.

    ReplyDelete

Post a Comment

Popular posts from this blog

HA Proxy for Exchange 2010 Deployment & SMTP Restriction

Juniper Aggregate Interfaces (LACP/No LACP)

Configuring Multicasting with Juniper EX switches (Part 1)