Hello!

I am trying to create my own QoS configuration, but I cannot make any filters to work.
I have problems with the most basic script as well:

tc qdisc del dev eth0.1 root
tc qdisc add dev eth0.1 root handle 1: htb default 13
tc class add dev eth0.1 parent 1: classid 1:1 htb rate 650kbit
tc class add dev eth0.1 parent 1:1 classid 1:13 htb rate 1kbit ceil 650kbit prio 3
tc qdisc add dev eth0.1 parent 1:13 handle 13: sfq
tc class add dev eth0.1 parent 1:1 classid 1:11 htb rate 1kbit ceil 650kbit prio 1
tc qdisc add dev eth0.1 parent 1:11 handle 11: sfq
tc filter add dev eth0.1 parent 1: prio 3 protocol ip handle 0x11 fw flowid 1:11 
tc filter add dev eth0.1 parent 1: protocol ip u32 match ip src 192.168.1.2/32 flowid 1:20

Basically, 2 classes, one is for 192.168.1.2's traffic, other is everything else. I create some traffic, but no packets arrive in the top priority class:

qdisc sfq 11: limit 128p quantum 1518b flows 128/1024
 Sent 0 bytes 0 pkts (dropped 0, overlimits 0)
qdisc sfq 13: limit 128p quantum 1518b flows 128/1024
 Sent 105252494 bytes 275034 pkts (dropped 6582, overlimits 0)
 backlog 27p
qdisc htb 1: r2q 10 default 13 direct_packets_stat 2773 ver 3.17
 Sent 106879496 bytes 278379 pkts (dropped 6741, overlimits 326187)
 backlog 27p

Of course no other QoS script is running, all necessary kernel mods are loaded. What am I doing wrong?

There is no 1:20 class in your example, which means that the packets that match this rule, will not go into one of the child classes, but out on the line directly. Apparently that rule matched 2773 packets (direct packets stat of HTB qdisc) so this filter actually works. Regarding the handle 0x11 filter, this filter looks like it should work. However you do not show the iptables code where you mark packets to 0x11. Also make sure you're actually marking them as 0x11 and not 11. Also make sure that the marking rule actually does mark packets, by looking at the iptables counters. If you have an -j ACCEPT or RETURN somewhere it's possible that the packets that you want to mark, never actually reach your rule.

Some issues unrelated to your question:
- Your HTB rates look weird and are most likely not what you want. From my personal experience, it's best to set the rates so that parent class rate = sum of all children class rates. In your case this is 650kbit = 1kbit + 1kbit which does not match. Also, using the prio parameter is a last resort and does not do what most people expect it to do, as it doesn't give any priority to packets.
- When using SFQ qdiscs, keep in mind that SFQ has an insanely large default queue size of 128 packets. Since it wants to re-order packets, it has to have a queue of some sort, but the more packets that have to sit in a queue, you will get high delays in transmissions. The more SFQ qdiscs you use, the larger the total amount of queued packets will be, e.g. if you're using two independent SFQ qdiscs that means that there will be 256 packets stuck in a queue. In the early days you had to patch the kernel to make SFQ use smaller queues, but today you can simply pass a limit option to the tc command. I suggest you use SFQ only where necessary (where you don't mind lag) and limit the queue size to some small value like 8 or 16.

I use HTB, SFQ and PRIO myself and they're very nice schedulers, however, you won't get good results if you don't use the correct values. Same goes for all schedulers unfortunately... TBF/RED tend to choke connections when used incorrectly, HSFC won't perform well if you don't know how it works (some claim it's better than HTB, but I didn't get any satisfying results from it, which just means I don't know how to make this damn thing work the way I want it to), etc.

Well doh. I double checked if I was writing the same script I entered to the command line. I should have done a 3rd check as well. Correct script is:

tc qdisc del dev eth0.1 root
## create qdisc + limiter class
tc qdisc add dev eth0.1 root handle 1: htb default 13
tc class add dev eth0.1 parent 1: classid 1:1 htb rate 650kbit
## traffic-specific parts:
# HTB class
# leaf qdisc
# filter
#Default (=no filter)
tc class add dev eth0.1 parent 1:1 classid 1:13 htb rate 1kbit ceil 650kbit prio 3
tc qdisc add dev eth0.1 parent 1:13 handle 13: sfq
#192.168.1.2's traffic
tc class add dev eth0.1 parent 1:1 classid 1:11 htb rate 1kbit ceil 650kbit prio 1
tc qdisc add dev eth0.1 parent 1:11 handle 11: sfq
tc filter add dev eth0.1 parent 1: protocol ip u32 match ip src 192.168.1.2/32 flowid 1:11

tc -s -d qdisc command's output is the same (apart of the package numbes / bytes of course) as above, so still no packet arrives in 11:.

Ignore rates, this was a part of a huge script I could not make work, and started deleting lines until I ended here.

I am trying to create a QoS that has hard and soft priorities as well. I wanted to create more HTB classes with the same prio providing the soft priority and using different prio parameters to have hard priority (as a prio would do). So you say that HTB's prio parameter does not enable this?

following applies in case the interface on which you do the QoS is an interface with Masquerading or SNAT active, as is often the case with the wan interface that connects to internet or an adsl modem so that the outside world sees one IP address.
Masquerading and SNAT are done before the packages hit the processing by tc, and hence they have lost their original source address by the time you test them for their source address !
If that's your problem the solution is in not doing the filter command in tc using 'u32' but to use 'fw' and use the MARK target in the MANGLE table of iptables. Below are some code snippets; as they result from quick cut/paste from my scripts check carefully.
The priorities serve to prioritize classes at the same level; the higher class (lower number) will be guaranteed its minimum before looking at any other need of other classes at that level. The mark that is set by mangling and used by fw to assign the packet to a class only exists while being processed in the router; it is not part of the data package when the package leaves the router.

TC="/usr/sbin/tc"
IPT="/usr/sbin/iptables"
WAN_IF="eth0.1"

MTU=1470
MIN_RATE=10
UPRATE=650
MINpuser=1

### module loading
for module in sch_htb cls_fw ; do
    /sbin/insmod $module 2>&- >&-
done

# Calculate r2q for htb discipline
RTOQ_U=$(($MIN_RATE*$UPRATE*10/(8*$MTU)))
[ $RTOQ_U -gt 20 ] && RTOQ_U=20
[ $RTOQ_U -eq 0 ] && RTOQ_U=1

### purge filters
( $TC filter show dev $WAN_IF | grep -q 'pref 100' ) && {
    for pref in 100; do
        $TC filter del dev $WAN_IF pref $pref &> /dev/null
    done
}

### purge classes
( $TC class show dev $WAN_IF | grep -q '1:1' ) && {
    for cnt in 3 2 1; do
        string=${TC}' class del dev '${WAN_IF}' classid 1:'${cnt}' &> /dev/null'
        eval $string
    done
}

### purge qdisc
( $TC qdisc show dev $WAN_IF | grep -q 'htb' ) && {
    $TC qdisc del dev $WAN_IF root &> /dev/null
}

### define root qdisc and its parent class that allows borrowing by lower classes; use other class as default
$TC qdisc add dev $WAN_IF root handle 1: htb default 2 r2q $RTOQ_U
$TC class add dev $WAN_IF parent 1:0 classid 1:1 htb rate ${UPRATE}kbit ceil ${UPRATE}kbit mtu $MTU

### define two leaf classes
## others
$TC class add dev $WAN_IF parent 1:1 classid 1:2 htb rate ${MINpuser}kbit ceil ${UPRATE}kbit prio 1 mtu $MTU
$TC qdisc add dev $WAN_IF parent 1:2 handle 2: pfifo limit 25
## special
$TC class add dev $WAN_IF parent 1:1 classid 1:3 htb rate ${MINpuser}kbit ceil ${UPRATE}kbit prio 2 mtu $MTU
$TC qdisc add dev $WAN_IF parent 1:3 handle 3: pfifo limit 25

### reduce long text strings
FLT="$TC filter add dev $WAN_IF parent 1:0 protocol ip pref"

### all filtering must be done on fwmark because iptables MASQUERADE will change source address !
$FLT 100 handle 0x3 fw classid 1:0x3    ## send special user to class 3

### flush and set mangle table
$IPT -t mangle -F
$IPT -t mangle -X
$IPT -t mangle -A PREROUTING -s 192.168.1.2/32 -j MARK --set-mark 0x3
$IPT -t mangle -A PREROUTING -m mark --mark 0x3 -j ACCEPT
## last line is only needed when more entries follow because '-j MARK' doesn't stop traversing the rest of matches
## and you might end up changing the mark because of yet another perhaps less specific match !

(Last edited by doddel on 11 Jan 2008, 19:03)

Not quite the way PRIO does, no. It mainly affects bandwidth borrowing / lending behaviour so it's only a hard prio if you give a class a close-to-zero rate and high ceil, so it has to borrow everything. Setting proper rates will give you a much finer level of control (e.g. class a 70% class B 30%) since the rates are also used for weighting when distributing extra bandwidth. You should try this approach first and add prio only later to see if it helps any.

HSFC is supposed to be stronger in this area, since you can work with both bandwidth and delay, but it doesn't work for me or rather I don't understand this scheduler well enough to make it work for me.