Pages

Tuesday 24 December 2013

Linux: iptables


Contents

To make a Bittorrent application work with Linux firewall iptables, you need to allow the Bittorrent client application to establish network connections to the network. Also, the Bittorent application need to accept connections from peer clients. In this section, we detail the iptables rules needed to achieve the above. These rules have been tested on a Linux Debian OS with the qBittorrent application. the same should apply to other torrent applications and Linux distributions with minor modifications if not any.




BitTorent ports
  • BitTorent client listens to one TCP port for incoming connections from peer torrent clients. This port is configurable and is normally set according to the standard to one value in the range 6881 to 6889. This range has then been extended from 6881 to 6999. It is however very common among torrent users to set this port outside the specified range in the standard, These users try to avoid Internet Service Providers (ISPs) bandwidth limitations set on these ports. Indeed, ISPs tend to limit Peer to Peer (P2P) applications due to the amount of internet usage they accounts for. 
  • BitTorent client should be allowed for TCP connections to peer torrent clients. For the reasons explained above, it very common for these ports to be outside the range 6881 to 6999.
  • If enabled, the DHT extension (peer2peer tracker) uses various UDP ports negotiated by the peers. At the client the UDP listening port is configurable.
  • The Bittorrent client should be allowed to reach the tracker server at port 6969. Note however, that most of the tracker servers use instead TCP port 80.
The table bellow depicts the list of UDP and TCP used by BitTorrent clients. [source: Wikipedia, List of TCP and UDP port numbers]:
Note that these ports are not registered IANA ports (1024 to 49151).

  Ports TCP   UDP
 6881-6968  TCP  UDP  BitTorrent part of full range ports used
 6969  TCP  BitTorrent Tracker
 6970-6999 TCP  UDP  BitTorrent part of full range ports used
 7000  TCP  Default for Vuze's built in HTTPS Bittorent Tracker
 10000 and above  TCP  UDP  Used by most BitTorrent applications.



Firewall Generic Rules for a BitTorrent application

The following table gives the generic rules you need to implement in your firewall to allow the BitTorrent client to work. In this table BitTorrent client IP is the IP of your network device that is connected to the internet.

 rule  Protocol Src. Port  Dest. Port  Src. IP  Dst. IP  Notes
 1  TCP  1024:65535  80  BitTorrent client IP  ANY Allow HTTP connections to web servers and to BitTorrent trackers.
 2  TCP  1024:65535  6881-65535  BitTorrent client IP  ANY  Allow BitTorrent client TCP connections to other peer clients.
 3  UDP  1024-65535  6881-65535  BitTorrent client IP  ANY Allow BitTorrent client UDP connection to other peer clients
 4  TCP 1024-65535   Configurable TCP port   ANY  BitTorrent client Allow peers to establish TCP connections to your BitTorrent application. 
 5  UDP  1024-65535  Configurable UDP port  ANY  BitTorrent client Allow you BitTorrent client to receive UDP packets from peers.
 6  TCP  1024-65535  53  BitTorrent Client IP  DNS server IP Allow TCP connections to the Domain Name Server (DNS) 



iptables Firewall Rules for a Standalone System

We need to translate the generic rules in the table above to iptables rules. Basically, the to set at the firewall should allow input and output TCP and UDP connections at specified ports and IP addresses. The Generic, iptable to allow output TCP connection is as follows:

iptables -A OUTPUT -o $ETH -p tcp \
   -s $TORRENT_CLIENT_IP --sport $UNPRIVPORTS \
     --dport $UNPRIVPORTS  \
    -m state --state NEW -j ACCEPT

iptables -A OUTPUT -o $ETH -p tcp ! --syn \
  -s $TORRENT_CLIENT_IP  --sport $UNPRIVPORTS \
   --dport $UNPRIVPORTS -j ACCEPT

iptables -A INPUT -i $ETH -p tcp ! --syn \
  -d $TORRENT_CLIENT_IP  --dport $UNPRIVPORTS \
  --sport $UNPRIVPORTS -j ACCEPT

In the above:

 -A OUTPUT Appends the rule to the 'output' chain.  
-o $ETH 
Specifies the network device name the rule applies applies to. If no device is specified, the rule is applied to all network devices.  

Generaly, the output network device is eth0. You may check your network devices on your machine using the command "ip add"

root@nemo:~# ip add
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:02:a5:22:8c:07 brd ff:ff:ff:ff:ff:ff
inet 192.168.153.51/24 brd 192.168.153.255 scope global eth0
inet6 fe80::202:a5ff:fe22:8c07/64 scope link
valid_lft forever preferred_lft forever


The output shows in our case two devices. the loopback device noted 'lo' and the 'eth0' network device with IP address 192.168.153.51. Thus we have:

ETH = eth0
TORRENT_CLIENT_IP = 192.168.153.51 
-p tcp   Specifies the TCP protocol that the rule applies to.   
-s $TORRENT_CLIENT_IP   The IP of the network device.
--sport $UNPRIVPORTS  
The source port allowed to establish a TCP connection. $UNPRIVPORTS are the unregistered private ports range 1024 to 65535.  Therefore:

UNPRIVPORTS = 1024:65535

 --dport $UNPRIVPORTS The destination port. 
 -m state --state NEW Rules in iptables are applied on a packet by packet basis. The 'state' filter option here is a performance enhancement that allows to bypass rule checking to packets that belongs to an established TCP or UDP connection. the option '--state NEW' is equivalent to the SYN TCP packet or the first UDP packet.

The rule bellow is necessary in case the state gets initialized (lost) in an ongoing TCP or UDP connection.

iptables -A OUTPUT -o $ETH -p tcp ! --syn \
  -s $TORRENT_CLIENT_IP  --sport $UNPRIVPORTS \
   --dport $UNPRIVPORTS -j ACCEPT 

 -j ACCEPT If the rule matches the received packet then accept it. 


Related Links

This wiki shows how to save iptables rules to a file and then use it to restore the rules at boot time. This is a handy way for iptables rules to survive a reboot.

You'll learn here how to configure programs to start up automatically at login. You may use that to start your torrent application without further action at boot time. This is useful if your BitTorrent client runs on a dedicated machine that you access remotely via the Torrent's application Web User Interface (Web UI).

A list of Internet service providers (ISPs) that are known to cause trouble for BitTorrent clients and the reason why.

The documentation of qBittorent.

You can use qBittorrent application through a Web User Interface (UI) only. In that case, your desktop do not need X server. This link describes how to achieve this, you need however to recompile the qBittorrent application.



Example
Bellow is an example of a script for iptables rules for the qBittorent application. The qBittorent application runs on HOME_DEBIAN (192.168.153.51) on the same subnet HOME_HP (192.168.153.21) is accessing the WEB UI of the qBittorent on port 5901.

#!/bin/bash

IPT="/sbin/iptables"


if [ "$1" = "status" ]
then
$IPT -L -v -n
exit 0
fi

ENABLE_LOGGING="1"
UDP_TORRENT="0"
ACCEPT_ICMP="1"
CONNECTION_TRACKING="1"
UNPRIVPORTS="1024:65535"
TORRENT_PORTS="6881"
TORRENT_PORTS_RANGE="6881:65535" 
SSH_PORTS="1020:65535"


HOMELAN="192.168.153.0/24"
HOMELAN_ADDR="192.168.153.1"
HOME_HP="192.168.153.21"
HOME_DEBIAN="192.168.153.51"
ETH="eth0"

ANY_ADDR="0.0.0.0/0"
LOOPBACK="127.0.0.0/8"

############################################
# Remove any existing rules from all chains#
############################################
$IPT --flush
$IPT -t nat --flush
$IPT -t mangle --flush

$IPT -X
$IPT -t nat -X
$IPT -t mangle -X

echo 1 > /proc/sys/net/ipv4/tcp_syncookies
echo 1 > /proc/sys/net/ipv4/icmp_echo_ignore_broadcasts
echo M1
###########################
# Reset the default policy#
###########################
$IPT --policy INPUT   ACCEPT
$IPT --policy OUTPUT  ACCEPT
$IPT --policy FORWARD ACCEPT
$IPT -t nat --policy PREROUTING  ACCEPT
$IPT -t nat --policy OUTPUT ACCEPT
$IPT -t nat --policy POSTROUTING ACCEPT
$IPT -t mangle --policy PREROUTING ACCEPT
$IPT -t mangle --policy OUTPUT ACCEPT

if [ "$1" = "stop" ]
then
echo "Firewall completely stopped!  WARNING: THIS HOST HAS NO FIREWALL RUNNING."
exit 0
fi

##############################################
# Unlimited traffic on the loopback interface#
##############################################

$IPT -A INPUT  -i lo -j ACCEPT
$IPT -A OUTPUT -o lo -j ACCEPT


#################################
# Set the default policy to drop#
#################################

iptables --policy INPUT DROP
iptables --policy FORWARD DROP
iptables --policy OUTPUT DROP

#$IPT -t nat --policy PREROUTING  DROP
#$IPT -t nat --policy OUTPUT DROP
#$IPT -t nat --policy POSTROUTING DROP

#$IPT -t mangle --policy PREROUTING DROP
#$IPT -t mangle --policy OUTPUT DROP

###############
#Stealth Scans#
###############
# Unclean
#$IPT -A INPUT -m unclean -j DROP
# All of the bits are cleared
$IPT -A INPUT -p tcp --tcp-flags ALL NONE -j DROP
# SYN and FIN are both set
$IPT -A INPUT -p tcp --tcp-flags SYN,FIN SYN,FIN -j DROP
# SYN and RST are both set
$IPT -A INPUT -p tcp --tcp-flags SYN,RST SYN,RST -j DROP
# FIN and RST are both set
$IPT -A INPUT -p tcp --tcp-flags FIN,RST FIN,RST -j DROP
# FIN is the only bit set, without the expected accompanying ACK
$IPT -A INPUT -p tcp --tcp-flags ACK,FIN FIN -j DROP
# PSH is the only bit set, without the expected accompanying ACK
$IPT -A INPUT -p tcp --tcp-flags ACK,PSH PSH -j DROP
# URG is the only bit set, without the expected accompanying ACK
$IPT -A INPUT -p tcp --tcp-flags ACK,URG URG -j DROP

##########################################
#Enable for Debuging purposes            #
##########################################
if [ "$ACCEPT_ICMP" = "1" ]; then
  iptables -A INPUT -p icmp -j ACCEPT
  iptables -A FORWARD -p icmp -j ACCEPT
  iptables -A OUTPUT -p icmp -j ACCEPT
fi
##########################################
#Connection State to bypass rule checking#
##########################################
if [ "$CONNECTION_TRACKING" = "1" ]; then
  iptables -A INPUT -m state --state ESTABLISHED -j ACCEPT
  iptables -A FORWARD -m state --state ESTABLISHED -j ACCEPT
  iptables -A OUTPUT -m state --state ESTABLISHED -j ACCEPT
fi


###############################################
# Refuse spoofed packets pretending to be from#
# the external interface's IP address         #
###############################################
$IPT -A INPUT  -i $ETH -s $HOME_DEBIAN -j DROP

############################################################
# Refuse packets claiming to be from the loopback interface#
############################################################
$IPT -A INPUT  -i $ETH -s $LOOPBACK -j DROP



####################################################
#Allow DNS lookup                                  #
####################################################
if [ "$CONNECTION_TRACKING" = "1" ]; then
  iptables -A OUTPUT -o $ETH -p udp \
   -s $HOME_DEBIAN --sport $UNPRIVPORTS \
     -d 195.170.0.1 --dport 53 \
    -m state --state NEW -j ACCEPT
fi

iptables -A OUTPUT -o $ETH -p udp \
  -s $HOME_DEBIAN --sport $UNPRIVPORTS \
  -d 195.170.0.1 --dport 53 -j ACCEPT

iptables -A INPUT -i $ETH -p udp \
  -d $HOME_DEBIAN --dport $UNPRIVPORTS \
  --sport 53 -s 195.170.0.1 -j ACCEPT

####################################################
#Allow TCP connection from LANA to STB for upgrade #
####################################################
if [ "$CONNECTION_TRACKING" = "1" ]; then
  iptables -A OUTPUT -o $ETH -p tcp \
   -s $HOME_DEBIAN --sport $UNPRIVPORTS \
     --dport 80 \
    -m state --state NEW -j ACCEPT
fi

iptables -A OUTPUT -o $ETH -p tcp \
  -s $HOME_DEBIAN --sport $UNPRIVPORTS \
   --dport 80 -j ACCEPT

iptables -A INPUT -i $ETH -p tcp ! --syn \
  -d $HOME_DEBIAN --dport $UNPRIVPORTS \
  --sport 80 -j ACCEPT


####################################################
#Allow TCP connection to torrents clients          #
####################################################
if [ "$CONNECTION_TRACKING" = "1" ]; then
  iptables -A OUTPUT -o $ETH -p tcp \
   -s $HOME_DEBIAN --sport $UNPRIVPORTS \
     --dport $TORRENT_PORTS_RANGE  \
    -m state --state NEW -j ACCEPT
fi

iptables -A OUTPUT -o $ETH -p tcp \
  -s $HOME_DEBIAN --sport $UNPRIVPORTS \
   --dport $TORRENT_PORTS_RANGE -j ACCEPT

iptables -A INPUT -i $ETH -p tcp ! --syn \
  -d $HOME_DEBIAN --dport $UNPRIVPORTS \
  --sport $TORRENT_PORTS_RANGE -j ACCEPT

if [ "$CONNECTION_TRACKING" = "1" ]; then
  iptables -A OUTPUT -o $ETH -p udp \
   -s $HOME_DEBIAN --sport $UNPRIVPORTS \
     --dport $TORRENT_PORTS_RANGE  \
    -m state --state NEW -j ACCEPT
fi

iptables -A OUTPUT -o $ETH -p udp \
  -s $HOME_DEBIAN --sport $UNPRIVPORTS \
   --dport $TORRENT_PORTS_RANGE -j ACCEPT

iptables -A INPUT -i $ETH -p udp \
  -d $HOME_DEBIAN --dport $UNPRIVPORTS \
  --sport $TORRENT_PORTS_RANGE -j ACCEPT


#####################################
# Allow SSH incoming access to debian #
# Allow SSH incoming accessto debian #
#####################################
if [ "$CONNECTION_TRACKING" = "1" ]; then
 iptables -A INPUT -i $ETH -p tcp \
   -s $HOME_HP --sport $SSH_PORTS \
    -d $HOME_DEBIAN --dport 22 \
    -m state --state NEW -j ACCEPT
fi

iptables -A INPUT -i $ETH -p tcp \
  -s $HOME_HP --sport $SSH_PORTS \
   -d $HOME_DEBIAN --dport 22 -j ACCEPT

iptables -A OUTPUT -o $ETH -p tcp ! --syn \
  -d $HOME_HP --dport $SSH_PORTS \
  -s $HOME_DEBIAN --sport 22 -j ACCEPT


if [ "$CONNECTION_TRACKING" = "1" ]; then
 iptables -A INPUT -i $ETH -p tcp \
   -s $HOME_HP --sport $SSH_PORTS \
    -d $HOME_DEBIAN --dport 5901 \
    -m state --state NEW -j ACCEPT
fi

iptables -A INPUT -i $ETH -p tcp \
  -s $HOME_HP --sport $SSH_PORTS \
   -d $HOME_DEBIAN --dport 5901 -j ACCEPT

iptables -A OUTPUT -o $ETH -p tcp ! --syn \
  -d $HOME_HP --dport $SSH_PORTS \
  -s $HOME_DEBIAN --sport 5901 -j ACCEPT



##################################
# Allow VNC connection
##################################

if [ "$CONNECTION_TRACKING" = "1" ]; then
 iptables -A INPUT -i $ETH -p tcp \
   -s $HOME_HP --sport $UNPRIVPORTS \
    -d $HOME_DEBIAN --dport 5900 \
    -m state --state NEW -j ACCEPT
fi

iptables -A INPUT -i $ETH -p tcp \
  -s $HOME_HP --sport $UNPRIVPORTS \
   -d $HOME_DEBIAN --dport 5900 -j ACCEPT

iptables -A OUTPUT -o $ETH -p tcp \
  -d $HOME_HP --dport $UNPRIVPORTS \
  -s $HOME_DEBIAN --sport 5900 -j ACCEPT


##################################
# Allow peer torrents connections 
##################################
if [ "$CONNECTION_TRACKING" = "1" ]; then
 iptables -A INPUT -i $ETH -p tcp \
    --sport $UNPRIVPORTS \
    -d $HOME_DEBIAN --dport $TORRENT_PORTS \
    -m state --state NEW -j ACCEPT
fi

iptables -A INPUT -i $ETH -p tcp \
   --sport $UNPRIVPORTS \
   -d $HOME_DEBIAN --dport $TORRENT_PORTS -j ACCEPT

iptables -A OUTPUT -o $ETH -p tcp ! --syn \
  --dport $TORRENT_PORTS \
  -s $HOME_DEBIAN --sport $TORRENT_PORTS -j ACCEPT

if [ "$UDP_TORRENT" = "1" ]; then 
if [ "$CONNECTION_TRACKING" = "1" ]; then
 iptables -A INPUT -i $ETH -p udp \
    --sport $UNPRIVPORTS \
    -d $HOME_DEBIAN --dport $TORRENT_PORTS \
    -m state --state NEW -j ACCEPT
fi

iptables -A INPUT -i $ETH -p udp \
   --sport $UNPRIVPORTS \
   -d $HOME_DEBIAN --dport $TORRENT_PORTS -j ACCEPT

iptables -A OUTPUT -o $ETH -p udp  \
  --dport $TORRENT_PORTS \
  -s $HOME_DEBIAN --sport $TORRENT_PORTS -j ACCEPT
fi




#drop everything and Log it
iptables -A INPUT -j LOG --log-prefix IN
iptables -A INPUT -j DROP
iptables -A OUTPUT -j LOG --log-prefix OUT
iptables -A OUTPUT -j DROP

#if [ "$ENABLE_LOGGING" = "1" ]; then
#for i in $STBLANS;do
#   iptables -A CDNCHAIN -s $i -j LOG --log-prefix STB_INVALID:
#done
#   iptables -A CDNCHAIN -s $STBLAN2 -j LOG --log-prefix STB_INVALID:
#   iptables -A CDNCHAIN -s $ANY_ADDR -j LOG --log-prefix UNKNOW_INVALID:
#fi

#for i in $STBLANS;do
#iptables -A CDNCHAIN -s $i -j DROP
#iptables -A CDNCHAIN -s $STBLAN2 -j DROP
#done
#iptables -A CDNCHAIN -s $ANY_ADDR -j DROP



Thursday 14 November 2013

Linux lPCI IDs: Unknown Device Issue

lspci is a utility for displaying information about PCI buses in the system and devices connected to them. The list of PCI IDs that lspci uses is contained in /usr/share/hwdata/pci.ids file.It might occur that the PCI id of a given device is not contained in that file. In that case, lspci will print out 'Unknown device' with the PCI id. To resolve this issue the pci.ids file must be updated or modified to include the missing ids.

In this example, we have a RedHat Linux EL5 installed as a virtual machine on VMware. The selected adapter type for the linux virtual machine is VMXNET3. If we run lspci on the Linux virtual machine, we get:
[root@localhost ~]# lspci
00:00.0 Host bridge: Intel Corporation 440BX/ZX/DX - 82443BX/ZX/DX Host bridge (rev 01)
00:01.0 PCI bridge: Intel Corporation 440BX/ZX/DX - 82443BX/ZX/DX AGP bridge (rev 01)
00:07.0 ISA bridge: Intel Corporation 82371AB/EB/MB PIIX4 ISA (rev 08)
00:07.1 IDE interface: Intel Corporation 82371AB/EB/MB PIIX4 IDE (rev 01)
00:07.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 08)
00:07.7 System peripheral: VMware Inc Unknown device 0740 (rev 10)
00:0f.0 VGA compatible controller: VMware Inc [VMware SVGA II] PCI Display Adapter
00:10.0 SCSI storage controller: LSI Logic / Symbios Logic 53c1030 PCI-X Fusion-MPT Dual Ultra320 SCSI (rev 01)
00:11.0 PCI bridge: VMware Inc Unknown device 0790 (rev 02)
00:15.0 PCI bridge: VMware Inc Unknown device 07a0 (rev 01)
00:15.1 PCI bridge: VMware Inc Unknown device 07a0 (rev 01)
00:15.2 PCI bridge: VMware Inc Unknown device 07a0 (rev 01)
00:15.3 PCI bridge: VMware Inc Unknown device 07a0 (rev 01)
00:15.4 PCI bridge: VMware Inc Unknown device 07a0 (rev 01)
00:15.5 PCI bridge: VMware Inc Unknown device 07a0 (rev 01)
00:15.6 PCI bridge: VMware Inc Unknown device 07a0 (rev 01)
00:15.7 PCI bridge: VMware Inc Unknown device 07a0 (rev 01)
00:16.0 PCI bridge: VMware Inc Unknown device 07a0 (rev 01)
00:16.1 PCI bridge: VMware Inc Unknown device 07a0 (rev 01)
00:16.2 PCI bridge: VMware Inc Unknown device 07a0 (rev 01)
00:16.3 PCI bridge: VMware Inc Unknown device 07a0 (rev 01)
00:16.4 PCI bridge: VMware Inc Unknown device 07a0 (rev 01)
00:16.5 PCI bridge: VMware Inc Unknown device 07a0 (rev 01)
00:16.6 PCI bridge: VMware Inc Unknown device 07a0 (rev 01)
00:16.7 PCI bridge: VMware Inc Unknown device 07a0 (rev 01)
00:17.0 PCI bridge: VMware Inc Unknown device 07a0 (rev 01)
00:17.1 PCI bridge: VMware Inc Unknown device 07a0 (rev 01)
00:17.2 PCI bridge: VMware Inc Unknown device 07a0 (rev 01)
00:17.3 PCI bridge: VMware Inc Unknown device 07a0 (rev 01)
00:17.4 PCI bridge: VMware Inc Unknown device 07a0 (rev 01)
00:17.5 PCI bridge: VMware Inc Unknown device 07a0 (rev 01)
00:17.6 PCI bridge: VMware Inc Unknown device 07a0 (rev 01)
00:17.7 PCI bridge: VMware Inc Unknown device 07a0 (rev 01)
00:18.0 PCI bridge: VMware Inc Unknown device 07a0 (rev 01)
00:18.1 PCI bridge: VMware Inc Unknown device 07a0 (rev 01)
00:18.2 PCI bridge: VMware Inc Unknown device 07a0 (rev 01)
00:18.3 PCI bridge: VMware Inc Unknown device 07a0 (rev 01)
00:18.4 PCI bridge: VMware Inc Unknown device 07a0 (rev 01)
00:18.5 PCI bridge: VMware Inc Unknown device 07a0 (rev 01)
00:18.6 PCI bridge: VMware Inc Unknown device 07a0 (rev 01)
00:18.7 PCI bridge: VMware Inc Unknown device 07a0 (rev 01)

03:00.0 Ethernet controller: VMware Inc Unknown device 07b0 (rev 01)

In the above example, devices with ids 07a0 and 07b0 are unknown. Indeed in the pci.ids file, under VMware Inc we have the following:

15ad  VMware Inc
        0405  [VMware SVGA II] PCI Display Adapter
        0710  Virtual SVGA
        0720  VMware High-Speed Virtual NIC [vmxnet]

/usr/share/hwdata/pci.ids file must be modified as follows:

15ad  VMware
0405  SVGA II Adapter
0710  SVGA Adapter
0720  VMXNET Ethernet Controller
0740  Virtual Machine Communication Interface
0770  USB2 EHCI Controller
0774  USB1.1 UHCI Controller
0778  USB3 xHCI Controller
0790  PCI bridge
07a0  PCI Express Root Port
07b0  VMXNET3 Ethernet Controller
07c0  PVSCSI SCSI Controller
0801  Virtual Machine Interface
15ad 0800  Hypervisor ROM Interface
1977  HD Audio Controller

After the modification, lspci gives the follwoing:

[root@localhost ~]# lspci
00:00.0 Host bridge: Intel Corporation 440BX/ZX/DX - 82443BX/ZX/DX Host bridge (rev 01)
00:01.0 PCI bridge: Intel Corporation 440BX/ZX/DX - 82443BX/ZX/DX AGP bridge (rev 01)
00:07.0 ISA bridge: Intel Corporation 82371AB/EB/MB PIIX4 ISA (rev 08)
00:07.1 IDE interface: Intel Corporation 82371AB/EB/MB PIIX4 IDE (rev 01)
00:07.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 08)
00:07.7 System peripheral: VMware Virtual Machine Communication Interface (rev 10)
00:0f.0 VGA compatible controller: VMware SVGA II Adapter
00:10.0 SCSI storage controller: LSI Logic / Symbios Logic 53c1030 PCI-X Fusion-MPT Dual Ultra320 SCSI (rev 01)
00:11.0 PCI bridge: VMware PCI bridge (rev 02)
00:15.0 PCI bridge: VMware PCI Express Root Port (rev 01)
00:15.1 PCI bridge: VMware PCI Express Root Port (rev 01)
00:15.2 PCI bridge: VMware PCI Express Root Port (rev 01)
00:15.3 PCI bridge: VMware PCI Express Root Port (rev 01)
00:15.4 PCI bridge: VMware PCI Express Root Port (rev 01)
00:15.5 PCI bridge: VMware PCI Express Root Port (rev 01)
00:15.6 PCI bridge: VMware PCI Express Root Port (rev 01)
00:15.7 PCI bridge: VMware PCI Express Root Port (rev 01)
00:16.0 PCI bridge: VMware PCI Express Root Port (rev 01)
00:16.1 PCI bridge: VMware PCI Express Root Port (rev 01)
00:16.2 PCI bridge: VMware PCI Express Root Port (rev 01)
00:16.3 PCI bridge: VMware PCI Express Root Port (rev 01)
00:16.4 PCI bridge: VMware PCI Express Root Port (rev 01)
00:16.5 PCI bridge: VMware PCI Express Root Port (rev 01)
00:16.6 PCI bridge: VMware PCI Express Root Port (rev 01)
00:16.7 PCI bridge: VMware PCI Express Root Port (rev 01)
00:17.0 PCI bridge: VMware PCI Express Root Port (rev 01)
00:17.1 PCI bridge: VMware PCI Express Root Port (rev 01)
00:17.2 PCI bridge: VMware PCI Express Root Port (rev 01)
00:17.3 PCI bridge: VMware PCI Express Root Port (rev 01)
00:17.4 PCI bridge: VMware PCI Express Root Port (rev 01)
00:17.5 PCI bridge: VMware PCI Express Root Port (rev 01)
00:17.6 PCI bridge: VMware PCI Express Root Port (rev 01)
00:17.7 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.0 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.1 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.2 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.3 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.4 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.5 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.6 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.7 PCI bridge: VMware PCI Express Root Port (rev 01)

03:00.0 Ethernet controller: VMware VMXNET3 Ethernet Controller (rev 01)




Saturday 9 November 2013

Socket Buffer Size

In network programming, we usually set the a buffer size to the sockets using the SO_RCVBUF and SO_SNDBUF socket options. In the code, we use the setsockopt function as follows:

int sockfd;                                     /* Socket descriptor */
int SockInBuffSize = 1024 * 512;  /* Size of the input socket buffer here 262144 bytes*/

setsockopt(sockfd, SOL_SOCKET, SO_RCVBUF, &SockInBuffSize, sizeof(SockInBuffSize));

For the above code to work, we need to make sure that the OS will allow us to set larger socket buffer sizes. We can achieve this using sysctl command as follows:

#sysctl -w net.core.rmem_max=524288 
#sysctl -w net.core.wmem_max=524288 
#sysctl -w net.core.wmem_default=524288 
#sysctl -w net.core.rmem_default=524288

Where rmem is the receive buffer and wmem the send buffer.

Failing to do the above, results in the socket buffer sizes that cannot be greater than the max values defined in /proc/net/core/

To view the current values of socket buffer sizes use the following command:

#sysctl -a | grep net.core


References

Working with NIC Ring Buffers
This post explains how to modify the NIC ring buffers which are important in fine tuning your network performance.
Source: www.scottalanmiller.com

Resolving Common Queuing/Frame Loss Issues
This chapter of Linux RedHat document explains how to use ethtool to fine tune your NIC for better network performance.
Source: redhat.com

Friday 8 November 2013

Find the Network Device Vendor ID (VID)

First find the devices that are connected to the all PCI buses in your system

# lspci

The output should be something similar to:

00:00.0 Host bridge: Intel Corporation X58 I/O Hub to ESI Port (rev 13)
00:01.0 PCI bridge: Intel Corporation X58 I/O Hub PCI Express Root Port 1 (rev 13)
00:02.0 PCI bridge: Intel Corporation X58 I/O Hub PCI Express Root Port 2 (rev 13)
00:03.0 PCI bridge: Intel Corporation X58 I/O Hub PCI Express Root Port 3 (rev 13)
00:04.0 PCI bridge: Intel Corporation X58 I/O Hub PCI Express Root Port 4 (rev 13)
00:05.0 PCI bridge: Intel Corporation X58 I/O Hub PCI Express Root Port 5 (rev 13)
00:06.0 PCI bridge: Intel Corporation X58 I/O Hub PCI Express Root Port 6 (rev 13)
00:07.0 PCI bridge: Intel Corporation X58 I/O Hub PCI Express Root Port 7 (rev 13)
00:08.0 PCI bridge: Intel Corporation X58 I/O Hub PCI Express Root Port 8 (rev 13)
00:09.0 PCI bridge: Intel Corporation X58 I/O Hub PCI Express Root Port 9 (rev 13)
00:0a.0 PCI bridge: Intel Corporation X58 I/O Hub PCI Express Root Port 10 (rev 13)
00:0d.0 Host bridge: Intel Corporation Unknown device 343a (rev 13)
00:0d.1 Host bridge: Intel Corporation Unknown device 343b (rev 13)
00:0d.2 Host bridge: Intel Corporation Unknown device 343c (rev 13)
00:0d.3 Host bridge: Intel Corporation Unknown device 343d (rev 13)
00:0d.4 Host bridge: Intel Corporation X58 Physical Layer Port 0 (rev 13)
00:0d.5 Host bridge: Intel Corporation Quickpath Interconnect Physical Layer Port 1 (rev 13)
00:0d.6 Host bridge: Intel Corporation Unknown device 341a (rev 13)
00:0e.0 Host bridge: Intel Corporation Unknown device 341c (rev 13)
00:0e.1 Host bridge: Intel Corporation Unknown device 341d (rev 13)
00:0e.2 Host bridge: Intel Corporation Unknown device 341e (rev 13)
00:0e.3 Host bridge: Intel Corporation Unknown device 341f (rev 13)
00:0e.4 Host bridge: Intel Corporation Unknown device 3439 (rev 13)
00:14.0 PIC: Intel Corporation X58 I/O Hub System Management Registers (rev 13)
00:14.1 PIC: Intel Corporation X58 I/O Hub GPIO and Scratch Pad Registers (rev 13)
00:14.2 PIC: Intel Corporation X58 I/O Hub Control Status and RAS Registers (rev 13)
00:1d.0 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #1
00:1d.1 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #2
00:1d.2 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #3
00:1d.3 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #6
00:1d.7 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #1
00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 90)
00:1f.0 ISA bridge: Intel Corporation 82801JIB (ICH10) LPC Interface Controller
00:1f.2 IDE interface: Intel Corporation 82801JI (ICH10 Family) 4 port SATA IDE Controller
01:03.0 VGA compatible controller: ATI Technologies Inc ES1000 (rev 02)
01:04.0 System peripheral: Compaq Computer Corporation Integrated Lights Out Controller (rev 03)
01:04.2 System peripheral: Compaq Computer Corporation Integrated Lights Out  Processor (rev 03)
01:04.4 USB Controller: Hewlett-Packard Company Proliant iLO2 virtual USB controller
01:04.6 IPMI SMIC interface: Hewlett-Packard Company Proliant iLO2 virtual UART
02:00.0 Ethernet controller: Broadcom Corporation NetXtreme II BCM5709 Gigabit Ethernet (rev 20)
02:00.1 Ethernet controller: Broadcom Corporation NetXtreme II BCM5709 Gigabit Ethernet (rev 20)
03:00.0 RAID bus controller: Hewlett-Packard Company Smart Array G6 controllers (rev 01)

The network adapters found in the above output are highlighted in yellow. In this particular case it is two Broadcom Ethernet controllers. To get the Vendor ID (VID) and the Device ID (DID) of these controllers, use the following command:

# lspci -n | grep x:x

The output is:

00:00.0 0600: 8086:3406 (rev 13)
00:01.0 0604: 8086:3408 (rev 13)
00:02.0 0604: 8086:3409 (rev 13)
00:03.0 0604: 8086:340a (rev 13)
00:04.0 0604: 8086:340b (rev 13)
00:05.0 0604: 8086:340c (rev 13)
00:06.0 0604: 8086:340d (rev 13)
00:07.0 0604: 8086:340e (rev 13)
00:08.0 0604: 8086:340f (rev 13)
00:09.0 0604: 8086:3410 (rev 13)
00:0a.0 0604: 8086:3411 (rev 13)
00:0d.0 0600: 8086:343a (rev 13)
00:0d.1 0600: 8086:343b (rev 13)
00:0d.2 0600: 8086:343c (rev 13)
00:0d.3 0600: 8086:343d (rev 13)
00:0d.4 0600: 8086:3418 (rev 13)
00:0d.5 0600: 8086:3419 (rev 13)
00:0d.6 0600: 8086:341a (rev 13)
00:0e.0 0600: 8086:341c (rev 13)
00:0e.1 0600: 8086:341d (rev 13)
00:0e.2 0600: 8086:341e (rev 13)
00:0e.3 0600: 8086:341f (rev 13)
00:0e.4 0600: 8086:3439 (rev 13)
00:14.0 0800: 8086:342e (rev 13)
00:14.1 0800: 8086:3422 (rev 13)
00:14.2 0800: 8086:3423 (rev 13)
00:1d.0 0c03: 8086:3a34
00:1d.1 0c03: 8086:3a35
00:1d.2 0c03: 8086:3a36
00:1d.3 0c03: 8086:3a39
00:1d.7 0c03: 8086:3a3a
00:1e.0 0604: 8086:244e (rev 90)
00:1f.0 0601: 8086:3a18
00:1f.2 0101: 8086:3a20
01:03.0 0300: 1002:515e (rev 02)
01:04.0 0880: 0e11:b203 (rev 03)
01:04.2 0880: 0e11:b204 (rev 03)
01:04.4 0c03: 103c:3300
01:04.6 0c07: 103c:3302
02:00.0 0200: 14e4:1639 (rev 20)
02:00.1 0200: 14e4:1639 (rev 20)
03:00.0 0104: 103c:323a (rev 01)

The Vendor ID (VID) and Device ID (DID) are highlighted in green. In the above output VID=14e4 and DID=1639.


References
You can find here PCI vendor list by name or by device ID, and other resources on the PCI bus.
Source: www.pcidatabase.com

Sunday 3 November 2013

Configuring the Flow Control

If a node on an Ethernet network sends packets faster than the receive node can handle, then the overwhelmed node can send pause frame to the sender pausing the sender for a short time. This process is called Flow Control.

If for any reason Flow Control is undesired it can be disabled using the ethtool. The example below is applied to the Intel e1000.

To list current status of flow control on the eth0 device, run the following command:

#ethtool –a eth0

The output:

Pause parameters for eth0:
Autonegotiate: on
RX: on
TX: on

To disable the Flow Control on the Tx and Rx, you need first to disable the auto-negotiations:

# ethtool --pause eth0 autoneg off

Then we need to disable the Flow Control for the send and receive traffic:

# ethtool --pause tx off rx off

Check again the status of Flow Control. All fields should be set to off:

# ethtool –a eth0
Pause parameters for eth0:
Autonegotiate: off
RX: off
TX: off


References

Ethernet flow control
Source: http://en.wikipedia.org


Saturday 2 November 2013

Find the Network Driver Version

Use the 'ethtool' command line as follows:

#ethtool -i eth0

The output should be something similar to the following:

driver: bnx2
version: 1.7.9-1
firmware-version: 4.6.4 NCSI 1.0.3
bus-info: 0000:02:00.0

Note the buf-info value of 02:00.0. Using that value with the lspci command, we can learn more on the vendor and type of the Ethernet device

#lspci | grep 02:00.0
02:00.0 Ethernet controller: Broadcom Corporation NetXtreme II BCM5709 Gigabit Ethernet (rev 20)

In conclusion, bnx2 is the Broadcom in-kernel Linux driver for the NetXtreme II 1G Ethernet device.

Friday 1 November 2013

Linux OS installed in 32-bit or 64-bit?

You can use the getconf linux command as follows:

$ getconf LONG_BIT

the will return the Number of bits in a type long int. If the value is 32 then you have a 32-bit Linux OS, otherwise it is a 64-bit OS

References


Apache Server Files Permissions

Files permissions on the Apache server is tightly linked with two configuration parameters in the Apache server configuration file (httpd.conf).  These are the User and Group configuration parameters that set the (User ID) UID and (Group ID) GID of the httpd binary. In the default configuration file (default /etc/httpd/conf/httpd.conf in RedHat Linux) we have following:    

#
# If you wish httpd to run as a different user or group, you must run
# httpd as root initially and it will switch.
#
# User/Group: The name (or #number) of the user/group to run httpd as.
#  . On SCO (ODT 3) use "User nouser" and "Group nogroup".
#  . On HPUX you may not be able to use shared memory as nobody, and the
#    suggested workaround is to create a user www and use that user.
#  NOTE that some kernels refuse to setgid(Group) or semctl(IPC_SET)
#  when the value of (unsigned)Group is above 60000;
#  don't use Group #-1 on these systems!
#
User apache
Group apache


In the above configuration, the User and Group are set to 'apache'. The httpd binary should start as root, since only a root user can change the user and group in runtime. httpd binary as user 'root' binds to port 80 (443 for https), or to an alternative port defined in the configuration file. Then, it creates child processes with the User and Group as defined in the configuration file. Running the service httpd, and listing the httpd processes we have:

[root@localhost ~]# ps -ef | grep httpd
root     22680     1  0 11:19 ?        00:00:00 /usr/sbin/httpd
apache   22682 22680  0 11:19 ?        00:00:00 /usr/sbin/httpd
apache   22683 22680  0 11:19 ?        00:00:00 /usr/sbin/httpd
apache   22684 22680  0 11:19 ?        00:00:00 /usr/sbin/httpd
apache   22685 22680  0 11:19 ?        00:00:00 /usr/sbin/httpd
apache   22686 22680  0 11:19 ?        00:00:00 /usr/sbin/httpd
apache   22687 22680  0 11:19 ?        00:00:00 /usr/sbin/httpd
apache   22688 22680  0 11:19 ?        00:00:00 /usr/sbin/httpd
apache   22689 22680  0 11:19 ?        00:00:00 /usr/sbin/httpd


and we have the following open tcp port:

[root@localhost ~]# netstat -na -tcp | grep httpd
tcp        0      0 :::80                       :::*                        LISTEN      22680/httpd
tcp        0      0 :::443                     :::*                        LISTEN      22680/httpd

The above shows a httpd daemon process with pid 22680 running as root, and a number of child httpd processes running owned by the 'apache' user. This is common practice in the Apache configuration, since we do not want the child processes to have the same privileges as root. Apache documentation states that the configured user and group ('apache' in the above example), should not own any content on the server.

Let us check that a script run by the httpd daemon has 'apache' user id. The following perl script is used, and placed /var/www/cgi-bin folder (default configuration):

#!/usr/bin/perl
my $whoiami=`whoami`;
print "\n$whoiami";
exit

Running this script from the web browser gives:

apache

The above means that the process that run this script on the Apache server is owned by 'apache'. This is in concordance with the configuration file we have.

cgi-scripts file permissions

Looking at the file permission of the Perl script file, we have:

[root@localhost cgi-bin]# ls -l -rwxr-xr-x 1 root   root   124 Oct 31 11:50 test.pl

This shows that the file is owned by root, and is part of the root group. The owner has read-write-execute privileges. The group and all users have read-execute privileges. Usually, we want server users to be able to modify script files without having 'root' privileges. Also, we need to disable read and execute privileges of the all users. We achieve this by using groups. Lets assume we want the user 'Alice' to be able to edit this file. For this purpose, we create a group 'webcontent' and add user 'Alice' to this group as follows:

[root@localhost]# groupadd webcontent

We need to add the 'apache' user to the 'webcontent' group as well, because the user 'apache' needs to be able to run this script

[root@localhost]# usermod -G web-content alice
[root@localhost]# usermod -G web-content apache

Now we need to set the correct permissions to the script file:

[root@localhost cgi-bin]# chmod 750 test.pl

If for any reason the script does not run, check Apache server error log file. If you notice something like the following:

[Fri Nov 01 10:24:11 2013] [error] [client xxx.xxx.xxx.xxx] (13)Permission denied: exec of '/var/www/cgi-bin/test.pl' failed

then the file permissions are set wrong for the given script. Review the steps above and set your file permission correctly.

References

Apache Tutorial: Dynamic Content with CGI