Avaya Support Forums  

Go Back   Avaya Support Forums > Avaya Networking Products

Reply
 
Thread Tools Search this Thread Display Modes
  #1  
Old 05-30-2012, 06:52 AM
im12 im12 is offline
Member
 
Join Date: May 2012
Posts: 3
im12 has 10 reputation points
Default ERS5530 stack and linux bonding for high throughput

Hi

We've got 2 5530s in a stack with 10 dell r610 running debian squeeze with xen connected to them via 4x 1gig links each. 2 links in each switch of the stack.

sysDescr: Ethernet Routing Switch 5530-24TFD
HW:35 FW:6.0.0.6 SW:v6.1.1.017

Initially this was setup with bonding mode 0 ( round robin distribution between the 4 nics ). The throughput was fine, we could get speeds both tx/rx of just above 2gbit with nfs3. Meaning there is a single ip in both ends with just one connection.

However, allthough the throughput was good the stability was not. We'd loose connectivity to the nas randomly from either 1 server or all of them. As expected (I suppose ) you'd see the mac address of the bond interface jumping between all 4 connected ports when doing show mac-address-table. My initial thought was "thats not good" and when we shut 3 of the 4 connected switchport the stability issues disappeared.

So the question is, how do we set this up for best possible performance. From what I can understand, both MLT and LACP configs will hash based on mac or ip and in both cases that will result in only 1 nic getting used.

I really hope that someone on here can point us in a useful direction in regards to what needs to be done on the switches to support something like the bond mode 0 on the server side.

Regards and thanks in advance
Reply With Quote
  #2  
Old 05-30-2012, 06:53 AM
im12 im12 is offline
Member
 
Join Date: May 2012
Posts: 3
im12 has 10 reputation points
Default

bond config :
auto bond1
iface bond1 inet static
address 10.0.20.23
netmask 255.255.255.0
network 10.0.20.0
slaves eth2 eth3 eth4 eth5
bond_mode balance-rr
bond_arp_ip_target 10.0.20.12
bond_arp_interval 2000
mtu 8122
Reply With Quote
  #3  
Old 05-30-2012, 06:54 AM
im12 im12 is offline
Member
 
Join Date: May 2012
Posts: 3
im12 has 10 reputation points
Default

With MLT:
As the bond mode is balance-rr it seems we can utilize more than 1 link on outbound traffic:
root@x3:~# dd if=/dev/zero bs=8192 of=/mnt/xen_convirt/tmp/test3.img count=2500000
2500000+0 records in
2500000+0 records out
20480000000 bytes (20 GB) copied, 110.856 s, 185 MB/s

Inbound will still be limited by the hashing employed by the MLT, and I'm not even able to max out 1 link:
root@x3:~# dd if=/mnt/xen_convirt/tmp/test.img bs=8192 > /dev/null
1250000+0 records in
1250000+0 records out
10240000000 bytes (10 GB) copied, 185.452 s, 55.2 MB/s

With no LAG:
With "wrong" switch config, these are the results we get:
outbound:
root@x3:~# dd if=/dev/zero bs=8192 of=/mnt/xen_convirt/tmp/test3.img count=2500000
2500000+0 records in
2500000+0 records out
20480000000 bytes (20 GB) copied, 114.935 s, 178 MB/s

inbound:
root@x3:~# dd if=/mnt/xen_convirt/tmp/test.img bs=8192 > /dev/null
1250000+0 records in
1250000+0 records out
10240000000 bytes (10 GB) copied, 44.8598 s, 228 MB/s


So I'm not really looking to implement LACP or MLT with the restrictions you mention. If I can get around those restrictions somehow, then fine

But is there nothing I can do to make the switch behave like the balance-rr bonding mode? As you can see the throughput is sooo much better. But the catch is that the bond mac addr is jumping between the 4 switch ports and its causing (seemingly) some harm to the switch.

I'd really like to figure out what exactly is happening to the switch in this setup that is causing the timeouts. And maybe we can mitigate that somehow and just continue using balance-rr ?
Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -7. The time now is 12:55 AM.

This Forum is provided solely for the use and convenience of Avaya customers and partners. Use of the Forum is subject to the Terms and Use and Privacy Statement found at www.avaya.com. No other use is permitted. The Forum including all content posted is “AS IS” and Avaya expressly disclaims all warranties and/or guarantees as to its accuracy, reliability, usefulness, quality or non-infringement of intellectual property. Avaya reserves the right to remove any content posted on the Forum at any time and for whatever reason.

Avaya will not be liable for any content posted on this Forum, including, without limitation, any errors or omissions or for any losses or damages of any kind incurred as a result of use or reliance on any content, regardless of its origin.

You expressly understand and agree that you assume all risks associated with use or reliance on this content.