Hi guys,
In succession of mwan2, i have made a new multi-wan script called mwan3. With the lessons i've learned making mwan2, i realized that the approach i took was not as efficient and configurable as it should be. Therefor i have created mwan3, which should resolve these issues. The reason i named it mwan3 and not a new version of mwan2 is because the config file changed quite a bit. If you were to upgrade this would break mwan2. I'd really like it if you could give it a go and let me know what you think of it.
edit: mwan svn repository is down. Please use git: https://github.com/Adze1502/mwan
edit2: With version 1.1, you can now ping multiple hosts per wan interface to check if it is still up. With the option reliability you set how many hosts should at least reply each round of testing, before triggering the interface as down. See the config example on how to configure it.
edit3: With the great help from Arfett, mwan3 now has a luci web interface. With this addition it is really easy to configure a multi-wan OpenWrt router. Thanks Arfett !!
edit4: GitHub repo is up: https://github.com/Adze1502/mwan . svn repo is down. Please update your "feeds.conf" file with the line "src-git mwan git://github.com/Adze1502/mwan.git" and remove the old svn repo.
edit5: New 1.4 version release which which should solve all load-balancing issues in previous release, like problem with wan connections not comming back online correctly.
The mwan3 package and the mwan3 luci app:
http://213.136.13.52/mwan3_latest_all.ipk
http://213.136.13.52/luci-app-mwan3_latest_all.ipk
And the source:
git://github.com/Adze1502/mwan.git
What is mwan3:
Mwan3 is a couple of lines of code that simplifies the usage of more (up to 250) WAN interfaces in OpenWRT. It is hotplug driven and it allows for any combination of primary, secondary or more failover interfaces, load balanced or not, for any combination of traffic. Mwan3 makes policy routing with multiple wan's easy. Mwan3 can monitor the state of interfaces by sending pings to a configured tracking host and failover if necessary.
Why should i use mwan3?:
- If you have multiple internet connections, you want to control which traffic goes through which wan's.
- Mwan3 can handle multiple levels of primary and backup interfaces, load-balanced or not. Different sources can have different primary or backup wan's.
- Mwan3 uses flowmask to be compatible with other packages (such as OpenVPN, PPTP VPN, QoS-script, Tunnels, etc) as you can configure traffic to use the default routing table.
- Mwan3 can also load-balance traffic originating from the router itself.
Requirements:
Mwan3 is successfully tested on OpenWRT trunk r40512. You need the following packages (which should be installed automatically if missing): ip, iptables, iptables-mod-conntrack, iptables-mod-conntrack-extra, iptables-mod-ipopt. Mwan3 is limited to max 250 wan interfaces.
How does it work:
Mwan3 is triggered by hotplug-events. When an interface comes up it creates a new routing table and new iptables rules. A new routing table is created for each interface. It then sets up iptables rules and uses iptables MARK to mark certain traffic. Based on ip rules the kernel determines which routing table to use. When an interface goes down, mwan3 deletes all the rules and routes to that interface in all created routing tables.
Mwan3 is not a daemon that runs in the background. Ones all the routes and rules are in place, it exits. The kernel takes care of all the routing decisions. If you want to apply a change you have made to mwan3 configuration, you have to trigger a hotplug event (replace eth1 and wan with your values):
ACTION=ifup DEVICE=eth1 INTERFACE=wan /sbin/hotplug-call ifaceor you could bring up the interface using a mwan3 command:
mwan3 ifup wanHow to install and configure:
Please check the wiki http://wiki.openwrt.org/doc/howto/mwan3 for the more info.
I'll assume here you have a clean install of OpenWRT. Before installing mwan3, you need to make sure that all of your wan interfaces are correctly configured and work. Place a different metric on each WAN interface! This metric has only effect on the default routing table, not on the mwan3 routing tables. If it is configured correctly you should have a default gateway with a different metric set for each WAN interface. Something will look like this:
root@OpenWrt:~# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 213.154.232.9 0.0.0.0 UG 10 0 0 eth1
0.0.0.0 192.168.33.1 0.0.0.0 UG 20 0 0 eth0.2
192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 br-lan
192.168.33.0 0.0.0.0 255.255.255.0 U 20 0 0 eth0.2
213.154.232.8 0.0.0.0 255.255.255.248 U 10 0 0 eth1Check if above configuration works by trying to ping www.google.com form each interface:
root@OpenWrt:~# ping -c 1 -I eth1 www.google.com
PING www.google.com (74.125.136.106): 56 data bytes
64 bytes from 74.125.136.106: seq=0 ttl=50 time=23.012 ms
--- www.google.com ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 23.012/23.012/23.012 ms
root@OpenWrt:~# ping -c 1 -I eth0.2 www.google.com
PING www.google.com (74.125.136.104): 56 data bytes
64 bytes from 74.125.136.104: seq=0 ttl=47 time=17.562 ms
--- www.google.com ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 17.562/17.562/17.562 msIf above ping test are successful, you can then continue installing mwan3. If you cannot ping www.google.com from all your WAN interfaces, there is a problem with your config and installing mwan3 won't fix that!
After installing mwan3, configure it by editting /etc/config/mwan3. For each WAN interface create an interface with a name that matches the one used in /etc/config/network. The status of each interface can be monitored with the track_ip option. Create at least two members. A member is a virtual representation of a wan interface. Each member can have different metric and or weight values. Members are used to make up a routing policy. Create a routing policy of at least one member. Members within one policy with a lower metric have precedence over higher metric members. Members with the same metric will load-balance. Load balancing members (with same metric) will distribute load based on those weights values.
After that, configure the rules. With the rules you can direct types of traffic based on ip, protocol or ports to certain policies, or use the default routing table by choosing "default". The order of the rules is very important, as they are loaded in that order. If a rule is matched, but all listed WAN interfaces/members are down, traffic is blackholed.
Example configuration
config interface 'wan'
option enabled '1'
list track_ip '8.8.4.4'
list track_ip '8.8.8.8'
list track_ip '208.67.222.222'
list track_ip '208.67.220.220'
option reliability '2'
option count '1'
option timeout '2'
option interval '5'
option down '3'
option up '8'
config interface 'wan2'
option enabled '1'
list track_ip '8.8.8.8'
list track_ip '208.67.220.220'
option reliability '1'
option count '1'
option timeout '2'
option interval '5'
option down '3'
option up '8'
config member 'wan_m1_w3'
option interface 'wan'
option metric '1'
option weight '3'
config member 'wan_m2_w3'
option interface 'wan'
option metric '2'
option weight '3'
config member 'wan2_m1_w2'
option interface 'wan2'
option metric '1'
option weight '2'
config member 'wan2_m2_w2'
option interface 'wan2'
option metric '2'
option weight '2'
config policy 'wan_only'
list use_member 'wan_m1_w3'
config policy 'wan2_only'
list use_member 'wan2_m1_w2'
config policy 'balanced'
list use_member 'wan_m1_w3'
list use_member 'wan2_m1_w2'
config policy 'wan_wan2'
list use_member 'wan_m1_w3'
list use_member 'wan2_m2_w2'
config policy 'wan2_wan'
list use_member 'wan_m2_w3'
list use_member 'wan2_m1_w2'
config rule 'example_1'
option dest_ip '213.136.223.128/25'
option dest_port '80'
option proto 'tcp'
option use_policy 'wan_wan2'
config rule 'example_2'
option src_ip '1.2.3.4'
option dest_ip '5.6.7.8'
option src_port '12345:54321'
option dest_port '12345:54321'
option proto 'udp'
option use_policy 'wan2_wan'
config rule 'default_rule'
option dest_ip '0.0.0.0/0'
option use_policy 'balanced'The wiki page http://wiki.openwrt.org/doc/howto/mwan3 has more more info on this.
Now you're almost good to go. Last thing you should do is make sure the your firewall configuration allows traffic from lan to wan's and make sure the default OUTPUT policy is set to ACCEPT!
root@OpenWrt:~# cat /etc/config/firewall
config defaults
option syn_flood '1'
option input 'ACCEPT'
option output 'ACCEPT'
option forward 'REJECT'
option drop_invalid '1'
config zone
option name 'lan'
list network 'lan'
option input 'ACCEPT'
option output 'ACCEPT'
option conntrack '1'
option forward 'REJECT'
config zone
option name 'wan'
list network 'wan'
list network 'wan2'
option input 'REJECT'
option output 'ACCEPT'
option forward 'REJECT'
option masq '1'
option conntrack '1'
config forwarding
option src 'lan'
option dest 'wan'Reboot your router and try if everything works. If not please feel free to ask your questions in this thread.
Status of load-balancing:
The command "mwan3 status" shows you the current state. It lists the state of each configured wan and the current output strategy for each policy. It also shows you a list of network which are "local" and not balanced. The last table lists the mwan3 rules currently active and it's policy.
root@OpenWrt:~# mwan3 status
Interface status:
Interface wan is online (tracking active)
Interface wan2 is online (tracking active)
Policy balanced:
wan2 (40%)
wan (60%)
Policy wan1_only:
wan (100%)
Policy wan2_only:
wan2 (100%)
Policy wan2_wan:
wan2 (100%)
Policy wan_wan2:
wan (100%)
Local connected networks:
destination policy hits
------------------------------------------------
127.0.0.0/8 default 92
224.0.0.0/3 default 4
192.168.1.0/24 default 0
192.168.33.0/24 default 4
213.154.232.8/29 default 4
Active rules:
source destination proto src-port dest-port policy hits
---------------------------------------------------------------------------------------------------
0.0.0.0/0 213.136.223.128/25 tcp 0:65535 80 wan_wan2 0
1.2.3.4 5.6.7.8 udp 12345:54321 12345:54321 wan2_wan 0
0.0.0.0/0 0.0.0.0/0 all balanced 20533Troubleshooting (if necessary):
root@OpenWrt:~# logread | grep mwan3
Tue Apr 15 14:47:33 2014 user.notice mwan3: ifup interface wan (eth1)
Tue Apr 15 14:47:47 2014 user.notice mwan3: ifup interface wan2 (eth0.2)
root@OpenWrt:~# iptables -L -t mangle -v -n -w
Chain PREROUTING (policy ACCEPT 84533 packets, 50M bytes)
pkts bytes target prot opt in out source destination
101K 54M mwan3_hook all -- * * 0.0.0.0/0 0.0.0.0/0
Chain OUTPUT (policy ACCEPT 21796 packets, 1953K bytes)
pkts bytes target prot opt in out source destination
31676 3119K mwan3_hook all -- * * 0.0.0.0/0 0.0.0.0/0
Chain mwan3_connected (1 references)
pkts bytes target prot opt in out source destination
102 7198 MARK all -- * * 0.0.0.0/0 127.0.0.0/8 mark match 0x0/0xff00 MARK or 0xff00
4 292 MARK all -- * * 0.0.0.0/0 224.0.0.0/3 mark match 0x0/0xff00 MARK or 0xff00
0 0 MARK all -- * * 0.0.0.0/0 192.168.1.0/24 mark match 0x0/0xff00 MARK or 0xff00
6 725 MARK all -- * * 0.0.0.0/0 192.168.33.0/24 mark match 0x0/0xff00 MARK or 0xff00
9 604 MARK all -- * * 0.0.0.0/0 213.154.232.8/29 mark match 0x0/0xff00 MARK or 0xff00
Chain mwan3_hook (2 references)
pkts bytes target prot opt in out source destination
133K 57M CONNMARK all -- * * 0.0.0.0/0 0.0.0.0/0 CONNMARK restore mask 0xff00
31717 2695K mwan3_ifaces all -- * * 0.0.0.0/0 0.0.0.0/0 mark match 0x0/0xff00
29628 2449K mwan3_connected all -- * * 0.0.0.0/0 0.0.0.0/0 mark match 0x0/0xff00
28819 2388K mwan3_rules all -- * * 0.0.0.0/0 0.0.0.0/0 mark match 0x0/0xff00
133K 57M CONNMARK all -- * * 0.0.0.0/0 0.0.0.0/0 CONNMARK save mask 0xff00
Chain mwan3_iface_wan (1 references)
pkts bytes target prot opt in out source destination
211 7624 MARK all -- * * 213.154.232.8/29 0.0.0.0/0 mark match 0x0/0xff00 /* wan */ MARK or 0xff00
148 9892 MARK all -- * * 0.0.0.0/0 0.0.0.0/0 mark match 0x0/0xff00 /* wan */ MARK xset 0x100/0xff00
Chain mwan3_iface_wan2 (1 references)
pkts bytes target prot opt in out source destination
1066 157K MARK all -- * * 192.168.33.0/24 0.0.0.0/0 mark match 0x0/0xff00 /* wan2 */ MARK or 0xff00
147 6006 MARK all -- * * 0.0.0.0/0 0.0.0.0/0 mark match 0x0/0xff00 /* wan2 */ MARK xset 0x200/0xff00
Chain mwan3_ifaces (1 references)
pkts bytes target prot opt in out source destination
359 17516 mwan3_iface_wan all -- eth1 * 0.0.0.0/0 0.0.0.0/0 mark match 0x0/0xff00
1213 163K mwan3_iface_wan2 all -- eth0.2 * 0.0.0.0/0 0.0.0.0/0 mark match 0x0/0xff00
Chain mwan3_policy_balanced (1 references)
pkts bytes target prot opt in out source destination
8663 719K MARK all -- * * 0.0.0.0/0 0.0.0.0/0 mark match 0x0/0xff00 statistic mode random probability 0.39999999991 /* wan2 2 5 */ MARK xset 0x200/0xff00
12919 1073K MARK all -- * * 0.0.0.0/0 0.0.0.0/0 mark match 0x0/0xff00 /* wan 3 3 */ MARK xset 0x100/0xff00
Chain mwan3_policy_wan1_only (0 references)
pkts bytes target prot opt in out source destination
0 0 MARK all -- * * 0.0.0.0/0 0.0.0.0/0 mark match 0x0/0xff00 /* wan 3 3 */ MARK xset 0x100/0xff00
Chain mwan3_policy_wan2_only (0 references)
pkts bytes target prot opt in out source destination
0 0 MARK all -- * * 0.0.0.0/0 0.0.0.0/0 mark match 0x0/0xff00 /* wan2 2 2 */ MARK xset 0x200/0xff00
Chain mwan3_policy_wan2_wan (1 references)
pkts bytes target prot opt in out source destination
0 0 MARK all -- * * 0.0.0.0/0 0.0.0.0/0 mark match 0x0/0xff00 /* wan2 2 2 */ MARK xset 0x200/0xff00
Chain mwan3_policy_wan_wan2 (1 references)
pkts bytes target prot opt in out source destination
0 0 MARK all -- * * 0.0.0.0/0 0.0.0.0/0 mark match 0x0/0xff00 /* wan 3 3 */ MARK xset 0x100/0xff00
Chain mwan3_rules (1 references)
pkts bytes target prot opt in out source destination
0 0 mwan3_policy_wan_wan2 tcp -- * * 0.0.0.0/0 213.136.223.128/25 multiport sports 0:65535 multiport dports 80 mark match 0x0/0xff00 /* example_1 */
0 0 mwan3_policy_wan2_wan udp -- * * 1.2.3.4 5.6.7.8 multiport sports 12345:54321 multiport dports 12345:54321 mark match 0x0/0xff00 /* example_2 */
21582 1792K mwan3_policy_balanced all -- * * 0.0.0.0/0 0.0.0.0/0 mark match 0x0/0xff00 /* default_rule */
root@OpenWrt:~# ip rule
0: from all lookup local
1001: from all iif eth1 lookup main
1002: from all iif eth0.2 lookup main
2001: from all fwmark 0x100/0xff00 lookup 1
2002: from all fwmark 0x200/0xff00 lookup 2
2254: from all fwmark 0xfe00/0xff00 unreachable
32766: from all lookup main
32767: from all lookup default
root@OpenWrt:~# ip route
default via 213.154.232.9 dev eth1 proto static metric 10
default via 192.168.33.1 dev eth0.2 proto static metric 20
192.168.1.0/24 dev br-lan proto kernel scope link src 192.168.1.1
192.168.33.0/24 dev eth0.2 proto static scope link metric 20
213.154.232.8/29 dev eth1 proto static scope link metric 10
root@OpenWrt:~# ip route list table 1
default via 213.154.232.9 dev eth1
root@OpenWrt:~# ip route list table 2
default via 192.168.33.1 dev eth0.2(Last edited by Adze on 30 Mar 2015, 20:11)
