Juniper Aggregate Interfaces (LACP/No LACP)

Last week, I was implementing link aggregation on Juniper EX Series switches (EX4200). I was moving them off HP switches and putting them into Juniper.

We were moving VMWare ESX boxes, NetApp and some physical boxes, though the configuration is very simple, I would like to mention how to do those with these boxes.

Points to remember:

• You will explicitly need to create “Aggregated Ethernet” interfaces on juniper
• You cannot have more than 64 AE interfaces on a single Virtual Chassis
• You cannot change the algorithm for load balancing between the multiple links. (If you are interested in the Junipers LAG algorithm read this )

Now having said this, lets look at the common steps, you need to create the Aggregated Ethernet interfaces on the Juniper.

root@EX4200#set chassis aggregated-devices ethernet device-count 4

The above command will create 4 “ae” interfaces, after you commit, you should see ae0 – ae 3 created. You can have any value between 1 and 64 in the place of the 4.

Please also remember that, the individual interfaces should have absolutely no configuration before they can be added to the aggregation. In our example, we have taken interfaces ge-0/0/0 and ge-0/0/1

root@EX4200#delete interfaces ge-0/0/0 unit 0

root@EX4200#delete interfaces ge-0/0/1 unit 0

You may chose to completely delete the interface itself, so all the configuration under the interfaces (like mtu) will be cleared.

At this point, you will have some thing similar

chassis {

aggregated-devices {
ethernet {
device-count 4;
}
}
}
interfaces
{
ge-0/0/0 {
}
ge-0/0/1 {
}
}

So, now that we have created the place holders for the aggregated interfaces, we will look into LACP in a little bit more detail.

Please remember that LACP is a negotiation protocol. It’s a standard protocol comparable to Cisco PAgP, which was later standardized into LACP. Also, remember that both ends must support LACP for the bundling to work using LACP.

ESX and juniper EX:

When creating the Aggregated interface, remember that the ESX (version 4) doesn’t support LACP, so we will be doing a generic aggregation. In this example, the ESX ports are vmnic2 and vmnic3 and the Switch ports are ge-0/0/0 and ge-0/0/1

On ESX

• Add the multiple networks in a vSwitch
• Set Load Balancing as “Route based on IP Hash”
• Network Failover Detection as “Link Status Only”
• Notify Switch as Yes

That’s all the change you need on VMWare.

 On Juniper EX

• Connect vmnic2 => ge-0/0/0
• Connect vmnic3 => ge-0/0/1

1: set interface ge-0/0/0 ether-options 802.3ad ae0

2: set interface ge-0/0/1 ether-options 802.3ad ae0
3: set ae5 unit 0 description "ESX - LAN Bundle"
4: set ae5 unit 0 family ethernet-switching port-mode access vlan members vlan1

So, with this, you see, we havent set any LACP properties, this is the way ESX likes is . We should have the interfaces up.

You can make sure by

1: root@EX4200> show interfaces terse | match ae0

2: ge-0/0/0.0 up up aenet --> ae0.0
3: ge-0/0/1.0 up up aenet --> ae0.0
4: ae0 up up
5: ae0.0 up up eth-switch

If the ae interface status is down, which means some thing went wrong and you need to reconfigure.

EDIT:
There is one thing I forgot to mention, some times you will lose connectivity while doing the change. In that case, you would have configured the Management Network differently, but the source Hash will not be working...

Please use the following to correct it ...

"# vim-cmd hostsvc/net/vswitch_setpolicy --nicteaming-policy=loadbalance_ip vSwitch0
# vim-cmd hostsvc/net/portgroup_set --nicteaming-policy=loadbalance_ip vSwitch0 

"

Link to VMWare KB

EDIT 2:
I was working with the ESX and ESXi and I was in for a shocking revelation... When I connected the ESXi ports which hosted the Managment network (even after making the Mgmt network to route based on IP Hash), the LACP bundle came up but there were 60% packet loss. On trying to find the reason, I found a weird work around.

Now, in my case, there is just one single subnet, so obviously, on the Juniper end, I had set to port mode access. Turns out ESX doesn't like it, though it is an access port, so to make the LACP bundle work, I converted the ae interface to trunk and set the access vlan as a native-vlan-id ... wallah !! it started working :)

Juniper EX configuration for the ESXi with Management Network

unit 0 {
    description "ESX1 - LAN";
    family ethernet-switching {
        port-mode trunk;
        native-vlan-id server_vlan1;
    }
}

Now, if you are using multiple vlans, then you may not have faced the problem in the first place as the port mode would have been trunk. 

Another thing, I would like to mention is that, in Juniper, if you have noticed, I haven't put anything under vlan members, that is because, if the port mode is trunk, anything that you put under vlan members becomes tagged ... Wierd I know ...

I wouldn't leave this to chance that it is working, so i did some more digging around, I again read this link. (Which I had read some 5 times before), but here is what i read ... 

"HP Switches Sample Configuration

The following configuration is specific to HP switches:

• HP switches support only two modes of LACP, ACTIVE, and PASSIVE while ESX/ESXi does not support either LACP mode currently.
• Set HP Switch port mode to TRUNK to accomplish static link aggregation with ESX/ESXi.
• TRUNK Mode of HP switch ports are the only supported aggregation method compatible with ESX 3.X NIC teaming mode IP hash. "

Now, if you will notice, the Juniper also supports only Active and Passive LACP modes ... I had ignored it as it was for HP switches and for ESX3.x (I am running 4.x), but I guess this is the only explanation as to why it works this way. 

Hopefully, you save some time digging around the info :)

NetApp and Juniper EX

NetApp supports LACP, so here we will see how to create a LACP vif on the NetApp and the corresponding configuration on Juniper EX.

In this case, on the netapp, we have 2 interfaces e0a and e0b (vif1), and on the Juniper side we have ge-0/0/2 and ge-0/0/3

On Net App:

If the vif is created, we will remove that and recreate.

the commands for NetApp are

1: ifconfig vif1 down

2: vif destroy vif1

3: vif create lacp vif1 -b ip e0a e0b

So, this is really very clear, we turned down an interface, then destroyed its configuration and then we said, we will create a virtual interface called vif1 with lacp and the load balancing method of “ip” , that’s what the “-b” switch stood for, and then we added the 2 interfaces.

The LACP is only available from version 7 and up, if you have a lower version, please go ahead and create a multi-mode vif and the Juniper switch configuration will like the ESX.

On Juniper EX

• Create the ae interfaces as mentioned in the beginning (if you already have configured them, then you don’t need to do it)
• Clean the configuration of the 2 interfaces.

The configuration on the EX switches will look like this

1: ae1 {

2:     aggregated-ether-options {

3:         link-speed 1g;

4:         lacp {

5:             active;

6:             periodic slow;

7:         }

8:     }

9:     unit 0 {

10:         description "NetApp1 - Controller 2 - LAN";

11:         family ethernet-switching {

12:             port-mode access;

13:             vlan {

14:                 members vlan1;

15:             }

16:         }

17:     }

18: }

19: interfaces

20: {

21:      ge-0/0/2 {

22:         ether-options {

23:         802.3ad ae1;

24:     }

25: }

26:     ge-0/0/3 {

27:     ether-options {

28:         802.3ad ae1;

29:     }

30: }



Once this is created, you can see the only difference is the LACP configuration that we put on the ae interface (this was not done for ESX/ESXi)

You can check if the interfaces are up, the very same way, but in addition, you also will have a LACP command to check.

1: root@PDX-SWIT-SVC02> show interfaces terse | match ae1.0

2: ge-0/0/2.0 up up aenet --> ae1.0
3: ge-0/0/3.0 up up aenet --> ae1.0
4: ae1.0 up up eth-switch


6: {master:0}
7: root@EX4200> show lacp interfaces ae1
8: Aggregated interface: ae1
9: LACP state: Role Exp Def Dist Col Syn Aggr Timeout Activity
10: ge-0/0/2 Actor No No Yes Yes Yes Yes Slow Active
11: ge-0/0/2 Partner No No Yes Yes Yes Yes Slow Active
12: ge-0/0/3 Actor No No Yes Yes Yes Yes Slow Active
13: ge-0/0/3 Partner No No Yes Yes Yes Yes Slow Active
14: LACP protocol: Receive State Transmit State Mux State
15: ge-0/0/3 Current Slow periodic Collecting distributing
16: ge-0/0/3 Current Slow periodic Collecting distributing
17:

You should be able to see if the LACP should be active for both partner and the Actor, if so, then the configuration works.

Hope you have found the guide useful. Please do comment if some thing needs to be corrected / altered.