OpenWrt Forum Archive

Topic: Limiting concurrent connections per port.

The content of this topic has been archived on 7 May 2018. There are no obvious gaps in this topic, but there may still be some posts missing at the end.

Hi,

I have an router WR1043ND v1 with only 32MB of RAM. It works mostly good but there is a bit of the problem with it. When I run qBittorrent and seed some torrents the number of active connections very often exceeds 6k even though I have limitem them to 500 in the app. I have SWAP but apparently (from what my brother tells me) he can't really play any online game because he can't connect to the servers. I'm already using SQM. It's very visible when playing Tibia it takes sometimes 30-40 tries to just get in the game and still it sometimes losses connection. Same with TeamSpeak.

So I'm looking for a way to limit a number of connections for qBittorrent. It runs on a specific port. I thought I could use iptables to put a limit on that port but I have no idea what to put in the config files.

Anyone willing to help me, please?
Thank you.

(Last edited by iletujestkont2 on 22 Mar 2016, 14:53)

SQM would solve the problem if router had enough cpu power for your connection speed.
what it is ?
Have you run "top" when system is highly loaded ?
If you see cpu exhausted then you have to reduce maximum rate in SQM configuration.
If it sounds bad to you then buy more powerful router or move gateway to a faster platform.
My experience tells that even quite powerful WNDR3800 with 650 mhz MIPS cpu can only do 60 mbit both ways with SQM enabled. It cant do 100/100. In such situation I used banana PI with its gigabit ethernet as VLAN router .

(Last edited by bolvan on 22 Mar 2016, 17:35)

iletujestkont2 wrote:

When I run qBittorrent and seed some torrents the number of active connections very often exceeds 6k even though I have limitem them to 500 in the app.

One problem -- if not the main problem -- is that you are running a torrent application on a very small CPU with very little memory. 500 active connections is already pushing it on a desktop CPU, your router's CPU may be very well be overburdened. Combined with SQM -- which is also quite CPU-heavy -- you might strain your system quite a bit too much.

And, no, I don't agree with bolvan, even a more powerful router will not help a lot. Ultimately, a router is not really well-suited to run heavy torrent applications. You might actually be better off using a separate machine for torrents and I assume, serving files. That's what NAS boxen are for and good at, after all.

Anyway, have you tried limiting the connections to something much less? Like 16 or 32 connections? Try a rather low number to see if the problems go away, and then "feel" your way to a higher number.

The P2P torrent client is not running on router. That would be suicide. No, the torrent client is running on the desktop with Windows. The app is set to 500 global connections but still in my router that is more then 6k of them. Almost all udp.

The CPU is not stressed. I'm having a script that blinks one of the LED on my router accordingly to CPU load. It's blinking slow which means that there is no much load on CPU. I'm running a 60Mb internet connection.

As soon as I run torrent client on my desktop my brother can't connect to any online game. Once the connection is established it's mostly fine, but sometimes it takes like 30 tries to connect. I'm not sure what's wrong. I think it's because of too much connections on the router.

(Last edited by iletujestkont2 on 22 Mar 2016, 22:55)

iletujestkont2 wrote:

The P2P torrent client is not running on router. That would be suicide. No, the torrent client is running on the desktop with Windows. The app is set to 500 global connections but still in my router that is more then 6k of them. Almost all udp.

The CPU is not stressed. I'm having a script that blinks one of the LED on my router accordingly to CPU load. It's blinking slow which means that there is no much load on CPU. I'm running a 60Mb internet connection.

How are you measuring load? %CPU idle, or load average # of tasks waiting for the CPU?

As soon as I run torrent client on my desktop my brother can't connect to any online game. Once the connection is established it's mostly fine, but sometimes it takes like 30 tries to connect. I'm not sure what's wrong. I think it's because of too much connections on the router.

Sounds like you might want to lower the timeouts on UDP connection tracking. Unfortunately, I don't know how to do it the OpenWRT way. It looks like you can specific a timeout for an ipset, but the details are beyond me right now.

I'm sure your device cant handle 60 mbits full duplex with SQM enabled.
It can do it one side with ~10-20 mbit other side.  More will burn CPU.
check it with top, its far more reliable then "blinking"
When cpu is overloaded user processes die or almost die. It means no DNS, no DHCP, no WIFI.
When gamer connects he requests DNS. Check if DNS can be resolved quickly and does not wait 10 seconds before resolved. Compare ping times in idle and loaded states.

(Last edited by bolvan on 23 Mar 2016, 06:46)

bolvan wrote:

I'm sure your device cant handle 60 mbits full duplex with SQM enabled.
It can do it one side with ~10-20 mbit other side.  More will burn CPU.
check it with top, its far more reliable then "blinking"
When cpu is overloaded user processes die or almost die. It means no DNS, no DHCP, no WIFI.
When gamer connects he requests DNS. Check if DNS can be resolved quickly and does not wait 10 seconds before resolved. Compare ping times in idle and loaded states.

I agree. Blinking is not very accurate. I changed SQM settings for download to around 100Mb/s but still got around 40Mb from 60Mb when only downloading ubuntu ISOs from FTP(??). Even with speedtest and SQM enabled I can't beat 40Mb. From top, CPU usage for [ksoftirqd/0] was around 60% with load around 0.72-1.0 with SQM enabled. CPU idle was 5-7%. Ping hasn't changed though. Still lower than 60ms for google.com. That happens only on WiFi connection.

On ethernet I have all 60Mb with SQM enabled and CPU usage around 10% for [ksoftirqd/0] and load around 0.20. There was a 12% CPU idle.

I'm not an expert but your saying that CPU in my router is not fast enough to handle 60Mb connection (through cable internet connection) with SQM enabled. So I should buy a new router then? Not happy about that. Still I don't understand how this is affecting Tibia and other games because even with SQM disabled I have connection issues.

(Last edited by iletujestkont2 on 23 Mar 2016, 13:45)

iletujestkont2 wrote:

Sounds like you might want to lower the timeouts on UDP connection tracking. Unfortunately, I don't know how to do it the OpenWRT way. It looks like you can specific a timeout for an ipset, but the details are beyond me right now.

I know how to do it.
How low should I get? I think that the default value is around 180.

I doubt if the large UDP table could cause any slowdowns, as most of the entries are just there waiting for timeout.

SQM and other QoS tools are CPU-heavy, like bolvan already said.

I suggest that you decrease the number of concurrent connections somewhat. SQM works per-stream, so reducing the number of stream from 500 to 400 (or 300) decreases the CPU significantly. You can surely adjust that setting in your bittorrent tool. That is the correct place for that, not iptables.

You need to make sure that you set the target speed limits low enough in the SQM settings. It is especially important that there is enough free bandwidth on the upload side, as the typical reason for connection problems is delays with the sent ack packets etc. that are needed for the tcp/ip protocols to work. SQM helps greatly to congestion, but it (and other QoS tools) do need some surplus bandwidth above the target set in the settings.

For reference:
My WNDR3800 (CPU at 680 MHz) can take ~80-85 Mb/s download+upload traffic. If you have 40+40, that would be the same. (and your CPU is apparently only 400 MHz, so 80 Mb/s might be too much for you)

I have 100/12 Mb/s fiber line and use 85000/10000 as SQM speed settings. Additionally, I have limited speed in my torrent client to 80000/9000, just to ensure that the torrent client does not hog all the bandwidth.

I have 60/3 (i guess) cable line internet connection and I have set SQM to 95% of that. Works mostly great.
I'm thinking about switching to 120/12 so I would need a better router than WNDR3800. Any suggestions?

Also from what I have already told you I assume that SQM takes more more heavy hit on CPU when using it on WiFi? On my set up I couldn't get more than 40Mb (of 60 available) when using WiFi.

Still non on you told me why online gaming is so problematic. I wouldn't want to buy a new router only to found out that the problem still exists.

(Last edited by iletujestkont2 on 23 Mar 2016, 15:44)

EDIT: I lift this bittorrent observation to top, as I see this  as a major issue for you.

iletujestkont2 wrote:

I have 60/3 (i guess) cable line internet connection and I have set SQM to 95% of that.

If your really only have 3 Mb/s upload speed, 500 simultaneous bittorrent connection is way too much. You waste too much of the upload bandwidth to protocol handling instead of actual payload data. You should reduce connection to some 150 or even lower. Each connection consumes some bandwidth just for packet acknowledgements etc. Currently you waste a lot of bandwidth.
---

Wired vs. wifi may play surprisingly in the QoS things.

You may need to try the various available alternative SQM options (scripts "simple", "hfcs_lite" etc.).

I have posted my own results to the SQM development discussion at https://github.com/tohojo/sqm-scripts/p … -185833432

I tested up-to-date Openwrt trunk with kernel 4.1.16 and the default SQM 1.0.7 with simple, layer_cake, hfsc_lite and hfsc_litest scripts with intentionally limited SQM settings 85000/10000 (that I use normally), 110000/15000 (that should exceed my connection and totally burden the router's CPU), as well as 110000/15000 using Wifi. All tests were run within an hour, so the general conditions are roughly similar.
I have maybe really good ISP connection with fiber coming to the house and a "CAT5e wired home". The latency from my router to the national TV company's site is only about 2 ms, so the results are practically only about the computing burden of the different strategies and the "default" latency from Finland to Toke's server in the neighbouring Sweden...

             wired 85/10             wired 110/15         Wifi 110/15
           Download/Up/Latency     Download/Up/Latency    Download/Up/Latency
Simple       19.5/2.1/18.5           21.2/2.7/19          11.0/3.0/21
Layer_cake   20.2/2.1/18.7           24.0/2.7/50          20.0/2.8/37
hfsc_lite    20.7/2.2/19.5           25.0/2.7/50          19.0/2.9/35
hfsc_litest  20.7/2.2/18.7           25.0/2.7/52          18.0/2.8/35

Check summary.png for an overview ;-)
https://www.dropbox.com/s/dqs01mxrrgepy … ummary.png
...
With wired 85/10 the experience was almost identical with all four qdisc strategies in SQM. Approx.  20 Mbit/s download / 2.1 Mbit/s upload and 19 ms latency shown in the flent summary graph.

With wired 110/15 there was more difference. Interestingly "simple" kept latency at 20 ms, while with the other 3 strategies latency jumped to 50 ms after ~20 seconds. (Might be a flent peculiarity, but still mentioning it.) "simple" kept low latency at 19 ms and 21 Mbit/s download, while the other 3 strategies had 50 ms latency while having 24-25 Mbit/s download per class.

But with Wifi in use from PC to router with 110/15 limits, "simple" lost its download speed. Latency was still low, but download speed was really low, just half of the normal. Interestingly, all four SQM strategies showed higher upload but lower download speed than with wired. (cake and hfs_lite(st) had also a lower latency than with wired. Strange.)

("flent" network measurement tool reports the overview as average of the 4 different traffic classes, so the total bandwidth was 4x the figures in the above table)

At least on my setup, the download speed using wifi and SQM "simple" was half of that what I could achieve with hfsc_lite+wifi, or simple+wired.
(cake is experimental, and not available in general builds. disregard that)

(Last edited by hnyman on 23 Mar 2016, 20:33)

iletujestkont2 wrote:

Also from what I have already told you I assume that SQM takes more more heavy hit on CPU when using it on WiFi?

SQM knows nothing about wifi. It works only on outbound (WAN) interface.
But yes, wifi processing takes more CPU than ethernet processing. And its more sensitive to availability of free CPU resources. If CPU is overloaded wifi can die or work slowly/unstable. Together with SQM it can easily overload CPU.
Thats why I connected banana pi to WNDR3800. WNDR3800 acts as wifi API and provides switch, banana acts as 2xVLAN gateway. I no more concerned about CPU because banana has 2 powerful ARM cores and 100/100 with HTB+fq_codel takes only 30-40% of them. And WNDR's CPU is freed for wifi. It does 110 mbits via wifi - tested with iperf. Although I do not recommend this solution to everyone because of annoying bugs in switch hardware or possibly driver, idea here is to have separate gateway device at least 2x more powerful than required.

(Last edited by bolvan on 23 Mar 2016, 16:29)

bolvan wrote:
iletujestkont2 wrote:

Also from what I have already told you I assume that SQM takes more more heavy hit on CPU when using it on WiFi?

SQM knows nothing about wifi. It works only on outbound (WAN) interface.
But yes, wifi processing takes more CPU than ethernet processing. And its more sensitive to availability of free CPU resources. If CPU is overloaded wifi can die or work slowly/unstable. Together with SQM it can easily overload CPU.
Thats why I connected banana pi to WNDR3800. WNDR3800 acts as wifi API and provides switch, banana acts as 2xVLAN gateway. I no more concerned about CPU because banana has 2 powerful ARM cores and 100/100 with HTB+fq_codel takes only 30-40% of them. And WNDR's CPU is freed for wifi. It does 110 mbits via wifi - tested with iperf.

Can you tell me more about this setup? Like how did you configured it or which version of banana are you using?
I would be interested in this solution.

iletujestkont2 wrote:

Can you tell me more about this setup? Like how did you configured it or which version of banana are you using?
I would be interested in this solution.

VLAN 1 - LAN
VLAN 2 - WAN

Inet <> Switch port 3. untagged, VLAN 2
banana <> Switch port 2. tagged VLAN 1,2
LAN <> Switch ports 0,1. utagged VLAN 1
WNDR's CPU <> Switch port 5. utagged VLAN 1

Banana has eth0.1 bound to its "LAN" and eth0.2 bound to its "WAN".
WNDR only has LAN, WAN is disabled. its configured as bridged AP without NAT, DHCP, DNS

But as I added before, I do not recomend everyone this solution because of : https://dev.openwrt.org/ticket/22024
I had to write script workaround to avoid this problem.
Better buy something with 2 network cards onboard. There are such devices.
And never rely on USB ethernet, its bad way.

(Last edited by bolvan on 23 Mar 2016, 16:36)

So it's not a working solution?
Like make it work and forget about it?

iletujestkont2 wrote:

So it's not a working solution?
Like make it work and forget about it?

Its working but requires some bug-avoiding techniques that make it look not so perfect as desired.

Then what about WR1043ND v2 or v3 with 720Mhz CPU and 64MB of RAM?
Would it suffice for 120/12Mb connection?

hnyman wrote:

At least on my setup, the download speed using wifi and SQM "simple" was half of that what I could achieve with hfsc_lite+wired, or simple+wired.
(cake is experimental, and not available in general builds. disregard that)

You're right. I was able to get full internet speed on WiFi with SQM as hfsc_lite.qos with fq_codel.
The CPU though was under heavy load. 0-3% idle time. Pings where not badly affected. I think I will stay with this configuration. Thank you for point it out.

Is there some significant difference between hfsc_lite.qos and simple.qos?

Great that it worked for you.

iletujestkont2 wrote:

Is there some significant difference between hfsc_lite.qos and simple.qos?

Quite a lot.

To summarize, SQM is an upper-level framework and several alternative low-level filters (="qdisc") can be used. Both scripts define traffic classification rules using several qdiscs on top of each other. "simple" uses htb+fq_codel, while "hfsc_lite" uses hfsc+fq_codel. They have different characteristics and apparently also quite different CPU requirements.

This goes deeply technical, but you might just quickly look at the sources to get an understanding what I mean.
Link to simple.qos source: https://github.com/tohojo/sqm-scripts/b … simple.qos
Link to hfsc_lite.qos source: https://github.com/tohojo/sqm-scripts/b … c_lite.qos

(Last edited by hnyman on 23 Mar 2016, 20:42)

hnyman wrote:

EDIT: I lift this bittorrent observation to top, as I see this  as a major issue for you.

iletujestkont2 wrote:

I have 60/3 (i guess) cable line internet connection and I have set SQM to 95% of that.

If your really only have 3 Mb/s upload speed, 500 simultaneous bittorrent connection is way too much. You waste too much of the upload bandwidth to protocol handling instead of actual payload data. You should reduce connection to some 150 or even lower. Each connection consumes some bandwidth just for packet acknowledgements etc. Currently you waste a lot of bandwidth.

I'm using qBittorrent with those settings:
Global maximum number of connections 500
Maximum number of connections per torrent: 100
Global maximum number of upload slots: 8
Maximum number of upload slots per torrent: 4

Are you suggesting I should be changing them? The are the default settings. And yes P2P is a major issue for me because my brother is bothering me about his stupid games.

hnyman wrote:
iletujestkont2 wrote:

Is there some significant difference between hfsc_lite.qos and simple.qos?

Quite a lot.

To summarize, SQM is an upper-level framework and several alternative low-level filters (="qdisc") can be used. "simple" uses htb + fq_codel, while hfsc_lite uses hfsc+fq_codel. They have different characteristics and apparently also quite different CPU requirements.

Great that it worked for you.

I think I still switch back to simple.qos when I change my router to more powerful unit. I think there is a reason why simple.qos is the default one. For now hfsc_lite.qos will do.

iletujestkont2 wrote:

I'm using qBittorrent with those settings:
Global maximum number of connections 500
Maximum number of connections per torrent: 100
Global maximum number of upload slots: 8
Maximum number of upload slots per torrent: 4

Are you suggesting I should be changing them? The are the default settings. And yes P2P is a major issue for me because my brother is bothering me about his stupid games.

Yes. I do suggest that.
With so low upload speed, you should have maybe max. 150-200 connections (global maximum of connections) at the same time. And probably no more than ~6 active torrents.

You could additionally check if your torrent client enables limiting upload speed. If yes, set the max speed a bit lower than the upload speed settings in SQM.

This link is not about your torrent client, but is a rather good summary: https://wiki.vuze.com/w/Good_settings
There are also several torrent-related calculators in the net that you could use to calculate "optimal settings". (but they may give rather varying answers ;-)

(Last edited by hnyman on 23 Mar 2016, 21:11)

iletujestkont2 wrote:

I think I still switch back to simple.qos when I change my router to more powerful unit. I think there is a reason why simple.qos is the default one. For now hfsc_lite.qos will do.

hfsc_lite was actually introduced to SQM only a few weeks ago just to improve performance on low-powered routers. You might read the reasoning in the first message here:
https://github.com/tohojo/sqm-scripts/pull/26

The discussion might have continued from here.