v13 - random words

While performing some tests with a congested 10Mbps link, a strange thing happened: The link was congested only on one direction and both endpoint queues were RED queues. Based on the parameters and the queue size, the delay between those two links should be something near 170ms. However, the delay was much larger: >300ms (!).

The “problem” was the ring-buffer of the underlying driver (e1000). This one used a buffer of 128 packets which when added to the average 150 packets in the queue, resulted in >300ms delay.

You can see this buffer by running:

#ethtool -g eth0

And you can modify this buffer by running:

#ethtool -G eth0 tx 80

This is the transmit buffer which (when filled) adds to the delay of the local queue.

Of course, in normal use, this buffer is a good thing as it will allow to get higher transfer rates easier (from the POV of the operating system). But when making experiments, this little thing gets in the way.

Another thing is that there seems to be a minimum value for this number. For example, on this card:

Marvell Technology Group Ltd. 88E8001 Gigabit Ethernet Controller

the minimum value is 80 (using e1000 driver from kernel 2.6.32)

So beware and don’t loose a week looking for this thing like I did.

NOTE: This is not related to the transmit queue length that is used on the interface, as shown by:

# ifconfig eth0
# ip link show eth0


Inside an Autonomous System, it is possible to move a machine inside a network, keeping its IP address even though it goes to a network segment that doesn’t serve the corresponding Network.

Something like this:

RouterA -------- Network Segment
                        |
         Host A (10.1.0.2/24, GW: 10.1.0.1)

It is possible to move Host A to another network segment, lets say to interface FastEthernet of RouterB. Of course this would require address and possibly other configuration changes to Host A. Changin the IP address of a server is not always a good idea ™.

Lets say that we move Host A to interface FastEthernet of Router B. Supposing that a routing protocol is setup and works in the Autonomous System, Host A may keep its IP address by configuring RouterB (cisco commands):

RouterB(config)# ip route 10.1.0.2 255.255.255.255
   FastEthernet 0/1

RouterB(config)# interface FastEthernet 0/1
RouterB(config-if)# ip proxy-arp      # This is the default

RouterB(config)# router eigrp 1
RouterB(config-router)# redistribute static

RouterA(config)# interface FastEthernet 0/1
RouterA(config-if)# ip proxy-arp

This will work because:

• Whenever Host A tries to reach a host in its subnet (ARP request), Router B will respond with its mac address. This is what Proxy ARP does.
• All routers within the A.S. will learn the 10.1.0.2/32 route. Even Router A will prefer to use this one instead of the directly connected 10.1.0.0/24 since it has a longest preffix (routing table lookups are longest matching preffix lookups)
• Since Router A will also learn this route and since it has Proxy ARP enabled, it will respond to ARP requests for 10.1.0.1 with its address