Having done lots of work on the old jdg-qos-script and then found a significant number (for me at least) of people decided to start sending comments, suggestions and bugfixes (which pushed me into a sense of guilt due to the lack of recent updates); I started the whole thing from scratch more or less. I used the same ideas as the old script however this time with a rewrite. The major difference is that the downstream part of the script can now utilise WRR with RED (hopefully GRED later on when I understand it better), this results in a very well fair shared download stream that works on a per IP basis.
The purpose of my script is to more or less translate all the QoS hardwork into a bunch of IPTables matching rules which I personally consider much easier to learn than the often cryptic ‘tc’ command sequences. Also when it comes to packet classification it is far more powerful and flexible I feel.
I feel this approach hopefully at a push opens up QoS to the masses, and gives a Bill-Gates-touchy-feely-tickbox that the M$ Whores have come to love.....only in this case it actually works and does something.
Sometime around the 12th April 2004 I decided to overhaul the whole script as previous incarnations were unable to do anything about fair sharing the downstream. Now that an entire re-write has occured and I am now throwing also WRR into the works the old stuff has been banished to the archives.
....yeah I know.....I will fill this in at some stage..... If you are really curious have a look at the script :)
The downfall of my script is the initial installation, to be frank its a complete pain in the rear, however after what feels like the countless patches and compiles (I have tried my best to detail them and make them relatively painless) the script is extremely easy to adapt to your needs and to reprioritise things simply means adjusting a handful of IPTables rules which are a lot easier to manage when compared to ‘tc’ and the likes.
If you do not like the idea of recompiling your kernel and or various userspace tools then stop now. This script might do everything you want but its going to be nigh to impossible to get running with this. You need to learn how to do this, remember how I said the downfall of this script is all the patching and compiling sonnie? Once you can do kernels you should be fine on the userspace malarkey, even if you blindly follow my instructions below.
N.B. in theory once you have compiled the userspace tools iptables and tc you should find that when you want to upgrade your kernel you will only need to recompile the kernel and you can safely forget about the userspace tools. Of course there are probably unknown exceptions to this rule so let me know if your pants set on fire when you follow this advice. If you use IPP2P you will probably need to recompile that though.
For the installation you will require in addition to the patches and the script from this site:
If everything has been download to /usr/src/ you should be able to following everything below without much problem (if not there are a few Makefiles that have been hardcoded, IPP2P for example). Below (and above) ‘x’ is the major Linux kernel version (either ‘4’ for 2.4.y or ‘6’ for 2.6.y) whilst ‘y’ is for the minor version if you had not guessed already.
Common setup instructions you need to follow and start off on:
$ cd /usr/src/
$ tar -jxf linux-2.x.y.tar.bz2
$ tar -zxf iproute2-2.6.8-ss040730.tar.gz
$ tar -jxf iptables-1.2.11.tar.bz2
$ tar -jxf qos-patches.tar.bz2
Please note although the instructions are generic, so far only kernel 2.6.x is supported as I have no time to test 2.4.x kernels plus I now no longer run anything that uses 2.4.x. Come on guys, 2.6.x is where its at :) If enough people complain and cry that there are no 2.4.x patches I might cave in, however there are lots of other features in 2.6.x which you really should be looking into using such as a lot more IPTables fun and also more schedulers to play with, plus there is also the fun that TCP Vegas can give you, or so it seems.
$ cd /usr/src/linux-2.x.y
$ patch -p1 < ../qos-patches/kernel/2.x/01_1_linux-2.6.8-imq-2.diff
$ patch -p1 < ../qos-patches/kernel/2.x/01_2_imq-nat.diff
$ patch -p1 < ../qos-patches/kernel/2.x/01_3_imq-compile-fix.diff
$ patch -p1 < ../qos-patches/kernel/2.x/02_1_esfq-2.6.patch
$ patch -p1 < ../qos-patches/kernel/2.x/02_2_esfq-fixes.diff
$ patch -p1 < ../qos-patches/kernel/2.x/03_1_linux-2.6.5-wrr.diff
$ patch -p1 < ../qos-patches/kernel/2.x/03_2_wrr-fixes.diff
$ patch -p1 < ../qos-patches/kernel/2.x/04_linux-2.6.8.1-CONNMARK.diff
$ make menuconfig
N.B. The ‘01_3_imq-compile-fix.diff’ fixes the bug with being unable to compile IMQ as a module, this patch I would have normally submit back to the LinuxIMQ folk however they use some horrible Yahoo Groups thingy mcwhatsit which I am not subscribing to. I am surprised they did not simply run a newsgroup or something. Anyway if anyone of them do see this patch and submit it back in then let me know then I can remove it from my instructions and patch set.
Once inside ‘menuconfig’ you should select the regular options you would to compile a kernel (if you do not know how to do this you should speak to someone whom can help you, not me, or one of the many HOWTO's and tutorials you can find online.
The options below marked as optional are for if you plan on compiling IPP2P too, if you are not you can forget them. I personally would recommend you go for IPP2P (if you plan on combating P2P software) as it really does a very good job.
Once compiled, installed and rebooted you may move onto IPTables and IPRoute.
$ cd /usr/src/iptables-1.2.11
$ patch -p1 < ../qos-patches/userspace/iptables/01_iptables-1.2.9-imq1.diff
$ chmod +x extensions/.IMQ-test*
$ KERNEL_DIR=/usr/src/linux-2.x.y make
$ su -c 'make install'
Before you compile iproute2 you need to make sure you have the development libraries for 'libdb' installed. Under Debian (stable) the package name is 'libdb3-dev'.
$ cd /usr/src/iproute2-2.6.8
$ patch -p1 < ../qos-patches/userspace/iproute2/01_iproute2-2.2.4-now-ss001007-esfq.diff
$ patch -p1 < ../qos-patches/userspace/iproute2/02_wrr-iproute2-2.2.4.patch
$ patch -p1 < ../qos-patches/userspace/iproute2/99_iproute_tidy.diff
$ ./configure
$ make
$ su -c 'make install'
$ cd /usr/src
$ tar -zxf ipp2p.06.tar.gz
$ cd ipp2p
$ patch -p1 < ../qos-patches/misc/ipp2p/01_cleanup.diff
$ make
$ su -c 'cp libipt_ipp2p.so /usr/local/lib/iptables/; cp ipt_ipp2p.ko /lib/modules/2.x.y/kernel/net/ipv4/netfilter/; depmod -a'
Well congratulations you have finished the difficult part, now what you need to do is tweak a few settings at the top of the script and then run it, covered later.
The first thing in the script is the paths to some of the fixed configuration files. This will enable you to use future versions of the script without having to go through the tedious part of customising for your Internet connection and also for your IPTables rulesets.
QOS_CONFIG=/etc/jdg-qos-script/config
SHAPER_OUT_RULES=/etc/jdg-qos-script/shaper-out
SHAPER_IN_RULES=/etc/jdg-qos-script/shaper-in
QOS_CONFIG points to all the connection variables that are settable and detailed below. SHAPER_OUT_RULES and SHAPER_IN_RULES contain simply a list of IPTables commands like those listed in the middle of the script which will be run instead if the files are present.
Quite clear, it simply points to the path of your utilities that make up the foundations of this QoS script. Really its only the TC and IPTABLES ones which really need changing however in 99% of the cases I would expect the defaults to be the ones you want, if you followed the installation instructions pretty much verbatim.
TC=/usr/local/sbin/tc
IPTABLES=/usr/local/sbin/iptables
IP=/sbin/ip
MODPROBE=/sbin/modprobe
Should be very straight forward, DWIF is the LAN (internal network card) interface whilst UPIF is the WAN (Internet facing one) interface.
DWIFLIMIT and UPIFLIMIT are the download and upload speeds of your link in kilobits per second however you should subtract about 20% from its true speed; once the script is running you are free to play with these as your millege may vary, 80% of the full link speed is a very good place to start if not stick with.
DWIF=eth2
DWIFLIMIT=820
UPIF=eth0
UPIFLIMIT=204
These tells the script if you have a NAT setup and what the IP address regions are of local addresses for what traffic should not be shaped by the script (this is space seperated). If you enable QOS_GATEWAY_UPSTREAM then the upstream traffic of the gateway machine is also taken into consideration. The LOCAL_TRAFFIC is just a space seperated list of IP address regions that are local so the script can tell what it should not shape and so then it only shapes for Internet bound traffic and not internally bound traffic.
NAT=1
QOS_GATEWAY_UPSTREAM=0
LOCAL_TRAFFIC="10.128.0.0/16 213.162.126.176/29"
Currently unsupported until I fix things and have a testing environment once again. If you plan to use the script in an ethernet bridge setup you should enable this.
#ETHERNET_BRIDGE=0
Now this variable is useful if you are running anything that needs a guaranteed amount of bandwidth such as VoIP and also work is being done for the cases of making sure your streaming radio station keeps running. If you set this to how much bandwidth you need (non-zero) then the script creates a special bucket that will always get this amount of bandwidth which is taken away as needed. By setting this you will need to add some IPTables rules to make use of the bucket, the rules need to be inserted (notice the ‘-I’ and not ‘-A’ that is used in the example IPTables rules in the script) at the top of the SHAPER-IN and SHAPER-OUT chains before everything else and after marking the packet you must RETURN the same rule. If this is not done you will not get your special bucket to work.
REALTIME_BANDWIDTH=0
If you have to force a couple of TCP port numbers to be in the low priority bucket you can do so here by simply putting them in the variable FORCE_LOW_PRIORITY separated by spaces.
FORCE_LOW_PRIORITY="6881 4662"
If you have compiled support for IPP2P and CONNMARK then you may turn on this variable to gain much better support for identifying P2P traffic and throwing it into the lowest priority bucket.
NETFILTER_P2P=1
A currently unsupported feature but in future it will be useful to those that want everyones traffic QoS'ed however it can all be treated as if it all came from one computer as you consider your userbase ‘fair’. When enabled its on its default of fair sharing on a per IP address basis, everyone gets their exact division, if off all downloads are considered equal (ie. ESFQ acts as SFQ and WRR is removed).
#BANDWIDTH_DIVIDE=1
Really all you should care about is changing WRR_CLIENTS to the maximum number of machines that are on your LAN and will be using the Internet link. Setting this higher than lower is perfectly safe and actually recommended.
WRR_CLIENTS=8
I do not have a clue what these mean, they came in the WRR example script and seem to work well for me. You should go to the WRR/WIPL website for insight into what these actually mean. Do let me know if you work it out, I currently do not have the time. :)
WRR_WEIGHT1_MODE=3
WRR_WEIGHT1_INCR=0.000111111111
WRR_WEIGHT1_DECR=0.000000000636
WRR_WEIGHT1_MIN=0.2
WRR_WEIGHT1_MAX=1.0
WRR_WEIGHT1_VAL=1.0
WRR_WEIGHT2_MODE=0
WRR_WEIGHT2_INCR=0.0
WRR_WEIGHT2_DECR=0.0
WRR_WEIGHT2_MIN=0.01
WRR_WEIGHT2_MAX=1.0
WRR_WEIGHT2_VAL=0.1
Now reboot your machine with your new kernel, and hopefully all you need to do is type:
# ./jdg-qos-script start
and it should work. Since version 031207 the script can now store the variable values in an external file, if you would like your configuration values to remain the same even when you place the script, copy them straight from the script into a textfile and point the script to this textfile, the default is ‘/etc/default/jdg-qos-script’ (which suits Debian perfectly).
The script has a stack of helpful parameters that you can call it with, it also has been designed to hopefully just slot in and out of your current setup regardless of what your iptables rules might currently be:
| ./jdg-qos-script <parameter> | What it does |
|---|---|
| start | Removes its-self from your machine and puts in a fresh copy |
| stop | Removes its-self from your machine, effectively disabling traffic shaping |
| restart | It actually does exactly the same as ‘start’ |
| status | A single shot printout of the iptables ‘hits’ and how ‘tc’ is getting along |
| pollbuckets | Prints the results of what ‘tc’ is up to every second, great to see if things are actually working in realtime |
| pollrules | Prints the results of what the iptables MARK'ing is up to every second. Really only useful to those wanting to tweaking the iptables rules at the end of the script |
‘Advanced’ users can tweak the iptables section at the end of the script. For upload you have ten ‘buckets’ (numbered 20 to 29, where 29 is low priority) to pick from. You work out a chain of iptables rules to decide what type of packet goes into which bucket. Treat the beginning of the chain to set a base ‘MARK’, and then later iptables rules adjust the packet so when you leave the chain the packet has the desired MARK'ing (and so priority). For downloading, you have only two buckets, number 20 where traffic cannot be dropped, such as ICMP and UDP traffic (and also high priority); whilst number 21 is for bulk traffic which can be dropped safely with knowledge that it is not going to generate more traffic, such as FTP data traffic (however not TCP ACK packets).
Released to the public and annouced on Freshmeat....
My initial motivation expired quickly but then fired up again with all the email traffic I started getting again. Also in the past two months I have moved flat and acquired a Sun Lunchbox which now runs Linux and all my QoS/Firewalling etc. So I have been busy....honest
Decided it was time to fully start off the website side of things and let the masses debug and complain about the new script
I ported WRR to kernel 2.6.x and incorporated it into my script. Pondered about updating the website for public consumption however laziness prevailed and I decided to leave the script running at home to see if it behaved.
In the meantime, Peter Frischknecht did some testing of the script to find on a network with 400 machines it crippled the gateway. Darn, however without direct access and lots of debugging time this is going to have to be looked at at some future point. At least I know its great for twenty people.
I decided it was time to re-write the website to be clearer and also redo the script but this time make the download fair on a per IP basis.
I did battle with the idea of trying to remove the IMQ dependency (with a ‘ppp-pipe’, this involved using ppp to fake two NICs with a crossover lead and some ‘raw’ netfilter black magic) however my efforts failed. The number of people whom contact me tended me to look towards the WRR which they had found worked extremely well.
Here I began work into a joint ESFQ, WRR, RED and HTB solution, along with an updated website
The first public release (and probably buggy although I have been running it fine for months) of the WRR, ESFQ and RED based script. You should find things work really quite nicely however the only current issue is if you plan on using the Realtime feature and quite often the bucket is not being used; if idle the bandwidth is not avaliable to anything else (as RED limits the download)
The WRR and ESFQ patches I had to port, however the CONNMARK is just me doing all the hard work for you guys to save you downloading the patch-o-matic from the netfilter website. I have broken out the patches as some folk, god forbid might not want to use my script.
Well to everyone whom emailed me. Most must currently think I am completely lame as I have failed to reply to pretty much all emails regarding the QoS script in the past three months. A big sorry to all those folks and a warning for those whom plan on emailing me not to worry if I do not reply quickly, I do hope to be a little quicker than three months though ;) Now this new script is out the door I promise to go through those emails and reply to them.
Below is a list of particularly helpful people whoms assistance has been greatly appreiated:
Pretty much the first person to use my script probably and has stuck with me since. He gave me a great testing ground where I sent him buggy code to test on his 128kb/s satellite link and it was with him where the VoIP support idea was originally mooted.
For his endless support and indurance with bug testing he has been a great help
Also another mighty helpful chap. He ran my script on a couple of his systems finding and fixing bugs plus most of use was him introducing me to ways that he tackled QoS issues. Although he will swear its the other way round, he gave me someone to bounce a lot of my ideas off.
Whilst staying at the BPL I grew to love the 2Mbit download/256kbit upload we had, until all eight members of the flat started using the connection. Two of whom were permanent P2P users, others gaming, others USENET downloading. Whilst all I wanted was to be able to stream Internet Radio, play XPilot and Enemy Territory plus connecting to some remote servers (and the other way round when outside the flat). “Dear God, was it too much to ask for?” Apparently so.....then did I decided it was time to get serious with Linux's implementation of QoS and throw together this script.
The BPL was the original testing grounds for my script and where I could test all my ideas
Being really the only major user of the Realtime feature of the script and so being the one whom is to find all the bugs. He has been very helpful and helped me no end.
Damn enthuasist also, already is determined to get GRED working before I can with my script, probably just to annoy me :P
My name is Alexander Clouter (pronounced ‘Clue-ter’ as opposed to ‘Clow-ter’, its a ‘bou-quet’/‘buck-et’ thing you know ;) You can get in touch with me by e-mail, remember to remove the ‘junk-this’ bit in the e-mail address. Or if you want via instant messaging:
|
|
Web Hosting very kindly provided by
|
|---|