HAProxy Load Balancer with SSL Offloading

In my previous blog, where I mentioned how to use HA Proxy(http://haproxy.1wt.eu/ – Open source Load balancing Solution) based load balancer for Exchange 2010 (Link to Post), I had a few requests on how to do the SSL offloading as well. If you have followed that guide, you have your SSL certificates hosted on the Exchange Hub CAS servers.
Microsoft does suggest to use the SSL on the boxes, but you can offload it as well, please remember that I am writing this for a generic https server and we will be using 2 open source products.
We will use HA Proxy for the Load balancing and the Pound for the SSL offloading. Now, for this purposes, I have used a single box, but for production machines, you may want to make a HA Pair of Linux box, which I will cover in a different post.
I have been running HA Proxy in the exchange environment and it is great, the only thing, I could ask for is source port based persistence Smile.
Anycase, back to the topic. Here is how the architecture will look like.
image

As you can see the Pound will do the SSL offload and send the traffic to HA Proxy and HA Proxy will do the load balancing.
Don’t get me wrong guys, pound is capable of sending the traffic directly to the servers, but then you will not be able to use the cool features of ha proxy like cookie persistence, so on and so forth.

In our test environment, here is what we will do.
Client Facing IP: 192.168.10.10
Server IP Address: 10.10.10.11, 10.10.10.12
The Linux Box is 192.168.10.9

First, install Ubuntu (http://www.ubuntu.com/download/ubuntu/download)  on the box.
Add the 192.168.10.10 as a Virtual IP on the box (If you are using a Active standby setup, this will be the floating IP)
Edit the interfaces script in Ubuntu) 
vi /etc/network/interfaces

auto eth0
iface eth0 inet static
address 192.168.10.9
netmask 255.255.255.0
network 192.168.10.0
broadcast 192.168.10.255
gateway 192.168.10.1
auto eth0:1
iface eth0:1 inet static
address 192.168.10.10
netmask 255.255.255.0
network 192.168.10.0
broadcast 192.168.10.255
gateway 192.168.10.1



Restart the Networking subsystem 
/etc/init.d/networking restart

Once that is done, the IP addresses will be shown when you execute “ifconfig –a” command.

Then install the following packages

   1: sudo apt-get install haproxy pound

This will install the haproxy and pound,

Now, lets roll

Configuring pound:


In order to configure pound, you will need to edit the config file using vi (or editor of your choice)

vi /etc/pound/pound.cfg

The file should look like the following. Please note, that I have installed the server certificate as PEM on the system.


## global options:

User		"www-data"

Group		"www-data"

#RootJail	"/chroot/pound"

## Logging: (goes to syslog by default)

##	0	no logging

##	1	normal

##	2	extended

##	3	Apache-style (common log format)

LogLevel	1

## check backend every X secs:

Alive		30

# Creating Listener

ListenHTTPS

Address 192.168.10.10

Port    443

Cert    "/etc/pound/testdom.pem"

Client  20

End

Service

HeadRequire "Host:.*testdomain.com*"

BackEnd

Address 127.0.0.2

Port 80

End



So, as you can see, it’s a generic configuration and then it is using the 127.x.x.x subnet to talk to the HA Proxy, the HAProxy will be listening on the particular IP and port.

After the configuration is done, go ahead and restart pound.

/etc/init.d/pound restart

The HA Proxy configuration will look like this 


   1: global
   2:         #uid 99
   3:         #gid 99
   4:         daemon
   5:         stats socket /var/run/haproxy.stat mode 600 level admin
   6:         maxconn 40000
   7:         ulimit-n 81000
   8:         pidfile /var/run/haproxy.pid
   9: defaults
  10:         mode    http
  11:         contimeout      4000
  12:         clitimeout      3600000
  13:         srvtimeout      3600000
  14:         balance roundrobin
  15: listen  ServerSSL 127.0.0.2:80
  16:         mode    tcp
  17:         option  persist
  18:         balance leastconn
  19:         stick-table type ip size 10240k expire 15m
  20:         stick on src
  21:         server Server1 10.10.10.11 weight 1 check port 80 inter 5000 rise 2 fall 3
  22:         server Server2 10.10.10.11 weight 1 check port 80 inter 5000 rise 2 fall 3
  23:         option redispatch
  24:         option abortonclose
  25:         maxconn 40000
  26: listen  stats :7000
  27:         stats   enable
  28:         stats   uri /
  29:         option  httpclose
  30:         stats   auth username:password

This might not seem like worth it, as this is for a single HTTPs client, but let me go ahead and do the same thing for exchange.

Here is how the Pound configuration and HA Proxy configuration will look for Exchange 2010, with SSL offload.


Pound and HAProxy for Exchange 2010

Pound Configuration:


   1: ## Pound Configuration
   2: User        "www-data"
   3: Group        "www-data"
   4: ## Logging: (goes to syslog by default)
   5: ##    0    no logging
   6: ##    1    normal
   7: ##    2    extended
   8: ##    3    Apache-style (common log format)
   9: LogLevel    1
  10: ## check backend every X secs:
  11: Alive        30
  12:               ListenHTTP
  13:                   Address 192.168.10.10
  14:                   Port    80
  15:                   Client  10
  16:               End
  17:               ListenHTTPS
  18:                   Address 192.168.10.10
  19:                   Port    443
  20:                   Cert    "/etc/pound/mailserver.pem"
  21:                   Client  20
  22:               End
  23:               Service
  24:                   HeadRequire "Host:.*mail.domain.com*"
  25:                   BackEnd
  26:                       Address 127.0.0.5
  27:                       Port    80
  28:                   End

HA Proxy Configuration:


   1: global
   2:         #uid 99
   3:         #gid 99
   4:         daemon
   5:         stats socket /var/run/haproxy.stat mode 600 level admin
   6:         maxconn 40000
   7:         ulimit-n 81000
   8:         pidfile /var/run/haproxy.pid
   9: defaults
  10:         mode    http
  11:         contimeout      4000
  12:         clitimeout      3600000
  13:         srvtimeout      3600000
  14:         balance roundrobin
  15: listen  Exchange2010-HTTP-HTTPS 127.0.0.5:80
  16:         mode    tcp
  17:         option  persist
  18:         balance roundrobin
  19:         stick-table type ip size 10240k expire 30m
  20:         stick on src
  21:         server HC-CAS1 10.10.10.11:80 weight 1 check port 80 inter 5000 rise 2 fall 3
  22:         server HC-CAS2 10.10.10.12:80 weight 1 check port 80 inter 5000 rise 2 fall 3
  23:         option redispatch
  24:         option abortonclose
  25:         maxconn 40000
  26: listen  Exchange2010 192.168.10.10:25
  27:         bind 192.168.10.10:110,192.168.10.10:135
  28:         bind 192.168.10.10:139,192.168.10.10:443
  29:         bind 192.168.10.10:60000,192.168.10.10:60001
  30:         bind 192.168.10.10:6001-6004
  31:         bind 192.168.10.10:993-995
  32:         mode    tcp
  33:         option  persist
  34:         balance roundrobin
  35:         stick-table type ip size 10240k expire 30m
  36:         stick on src
  37:         server HC-CAS1 10.10.10.11 weight 1 check port 80 inter 5000 rise 2 fall 3
  38:         server HC-CAS2 10.10.10.12 weight 1 check port 80 inter 5000 rise 2 fall 3
  39:         option redispatch
  40:         option abortonclose
  41:         maxconn 40000
  42: listen  stats :7000
  43:         stats   enable
  44:         stats   uri /
  45:         option  httpclose
  46:         stats   auth username:password

Please note, that you will still have to make those changes on the Exchange server as noted in my earlier blog post.

If you need explanation on any of the configuration components, do ask (for the HA proxy they are all explained in my previous blog post)

I hope you find this informative and would like to thank you for reading.

Comments

  1. BruceNovember 8, 2011 at 7:52 AM

    Thunder Emperor,

    Keep up the great work on your blog. I am particular interested in a blog post on HAProxy which you previously mentioned "I have used a single box, but for production machines, you may want to make a HA Pair of Linux box, which I will cover in a different post."

    When do you think you will find time in your busy schedule to make this blog post on a pair of Linux HAProxy servers ?

    Your blogs are clear and easy to understand.
    Thanks,

    ReplyDelete
  2. Thunder EmperorNovember 8, 2011 at 11:16 PM

    Sure Bruce ... I will write the HA Pair one next ... Once I have some time ...

    Thanks for reading and hope it helps

    ReplyDelete
  3. morfienDecember 6, 2011 at 1:10 AM

    Amazing blogs, thanks for the time and effort that you have put in to these.

    If only you have made them sooner.
    I just made a lab a while ago and was force into using a HW loadbalancer on trail to make it work.

    Guess what im replacing them with when it expires.

    If you also do SQL i would love to see a post about distributed transactions servers using HAproxy.

    ReplyDelete
  4. Thunder EmperorDecember 7, 2011 at 12:58 AM

    Morfien .... Thanks, I will write one on MySQL that I created for multiple master.

    I dont know if you have noticed, this load balncing stuff is written in 3 parts for Exchange 2010 ... All of them focus on a different aspect

    Part 1: Basic
    http://3-4-5-6.blogspot.com/2011/03/ha-proxy-for-exchange-2010-deployment.html

    Part 2: Including SSL Offloading
    http://3-4-5-6.blogspot.com/2011/08/haproxy-load-balancer-with-ssl.html

    Part 3: Redundancy
    http://3-4-5-6.blogspot.com/2011/11/haproxy-for-exchange-2010part-2.html


    Hope this helps ...

    ReplyDelete
  5. morfienDecember 7, 2011 at 2:56 AM

    I did notice thanks.

    I'm in the process of creating a MSSQL multi-master setup and am planning on using HAproxy as the load balancer.

    we'll see how that goes :)

    ReplyDelete
  6. LeonApril 19, 2012 at 6:54 PM

    I am trying to find your example for cookies in this article. You mention it as an option when using Pound, but I am not seeing any of the HAProxy cookie config options in your HA Proxy configuration example. Are they enabled by default if the data has been unencrypted?

    ReplyDelete
    Replies
    1. Thunder EmperorApril 20, 2012 at 2:00 PM

      Hi Leon,

      In this example, I have used the source IP persistence

      If you want to use cookie persistence then change the lines 15 down


      15: listen Exchange2010-HTTP-HTTPS 127.0.0.5:80
      16: mode http
      17: option httpclose
      18: balance roundrobin
      19: cookie exchangecookie insert
      20: server HC-CAS1 10.10.10.11:80 cookie exchangecookie_HC-CAS1 weight 1 check port 80 inter 5000 rise 2 fall 3
      21: server HC-CAS2 10.10.10.12:80 cookie exchangecookie_HC-CAS2 weight 1 check port 80 inter 5000 rise 2 fall 3
      22: option redispatch
      23: option abortonclose
      24: maxconn 40000


      I guess its self explanatory ... let me know if you need any more assistance

      Delete
  7. ramFebruary 1, 2013 at 10:51 PM

    Hi Thunder Emperor,

    You mentioned pem certificate is this a exchange server certificate or is it a self signed certificate.

    Thanks,
    Ramana

    ReplyDelete
    Replies
    1. Thunder EmperorFebruary 8, 2013 at 1:00 PM

      Hi Ram,

      Sorry for the delay in response. This is the Exchange Server Certificate which needs to be offloaded.

      Delete
  8. UnknownAugust 8, 2016 at 3:07 AM

    This comment has been removed by a blog administrator.

    ReplyDelete

Post a Comment

Popular posts from this blog

Juniper Aggregate Interfaces (LACP/No LACP)

Image
Last week, I was implementing link aggregation on Juniper EX Series switches (EX4200). I was moving them off HP switches and putting them into Juniper. We were moving VMWare ESX boxes, NetApp and some physical boxes, though the configuration is very simple, I would like to mention how to do those with these boxes. Points to remember: You will explicitly need to create “Aggregated Ethernet” interfaces on juniper You cannot have more than 64 AE interfaces on a single Virtual Chassis You cannot change the algorithm for load balancing between the multiple links. (If you are interested in the Junipers LAG algorithm read this ) Now having said this, lets look at the common steps, you need to create the Aggregated Ethernet interfaces on the Juniper. root@EX4200#set chassis aggregated-devices ethernet device-count  4 The above command will create 4 “ae” interfaces, after you commit, you should see ae0 – ae 3 created. You can have any value between 1 and 64 in...
Read more

Exporting Contacts out of Blackberry with Windows Contacts

Image
So, you are trying to get your contacts out of your blackberry and sync it to your Android / iPhone / Windows Phone, or just want to sync it to your Google account for backup purposes. Here are a few tips that will get you started – I have just enumerated some methods. Option 1: Use Blackberry Desktop 6 (Manual) (Not the most updated software) – It can directly export out your contacts out in the CSV Format. Please read the official How-To here Option 2: Using Blackberry Desktop 7 (Manual) If you have a Blackberry Desktop 7 – Which most likely you may end up having, here is how you can do it – For people who don’t want to use the sync (Option 3) Step 1: Backup BB Contacts to Windows Contacts Connect your Phone to your Windows (Assuming you have Windows 7/8/8.1) Click On Organizer Select Configure Select “One-Way to Your Computer and Windows Contacts (You may Sync it to your outlook, but that’s not covered in this blog) Click on Sync Organizer (Sync All) on ...
Read more

Configuring Multicasting with Juniper EX switches (Part 1)

Image
Configuring multicast routing using Protocol Independent Multicast in a all Juniper environment is as simple as the Cisco environment, but when I searched, I dint find good articles or KB’s explaining this, so I decided to write one. I am fairly certain that if you have searched and arrived at this post, you know the basics of multicasting and IGMP, but yet, for the benefit of all readers, I will in brief describe the need for multicast. Please note, that this is just the IPv4 version of the story presented in a understandable manner and not all aspects of multicast are covered, but only the ones needed for this post. What is Multicast ? In IPv4 we have 3 kinds of packets, Unicast – One Source, One Destination Broadcast – One source and all destinations Multicast – One source and many destinations Now, all of us are familiar with the first one, we use it almost every minute we are on a network connected equipment. To, explain all this, lets choose different ...
Read more