OpenWrt Forum Archive

Topic: Split HTTP traffic on HTTP "host" header - iptables or reverse proxy

The content of this topic has been archived on 31 Mar 2018. There are no obvious gaps in this topic, but there may still be some posts missing at the end.

Hi,

I want to split HTTP traffic coming into my Linksys OpenWrt box (on WAN), based on HTTP "host" header, to two different LAN systems running web servers.  How can it be done ?   

1. Is it possible to do via iptables ?  I googled but can't find anything that works.  The only reference I found was somewhere in HyperWRT forums.  It talks about a "webstr" match.  I can't find any iptables match extension that'd let me do that.

2. I tried lighttpd, using it as a reverse proxy.  It seems to work, but it crashes if I transfer a big file.  The problem is, lighttpd tries to cache the file or something to that effect.  It's (lighttpd's) memory consumption grows, until it becomes too much that the kernel steps in and kills the process.  I can't find how to disable caching or set a low cache limit in lighttpd.

Note on option 1 above :  I guess this is in tune with being able to block access to certain websites - like access restrictions or parental control.  This feature is there in some routers and I think it must be using a similar logic to deny access.

Are there any other reverse proxies ?   I came to know about "pound" (http://www.apsis.ch/pound/) - a reverse proxy for Linux, which can possible do what I want.  But its not available for OpenWrt sad

So, what are my options ?

Thanks in advance.

I don't know about webstr, but there is a string module for iptables.  Using the string module, plus specifying the port as 80 you should be able to do what you need.

layer7 iptables module could also be what you are looking for... search the wiki for layer7

you'd just need to write your own regex pattern and do a -m layer7 with iptables.

also, layer7 works hand in hand with conntrack so you wouldn't be wasting much resources either.

(Last edited by keitsi on 6 Feb 2006, 23:26)

miah, could you please give an example on how to use the string module ?  Thanks.

Are you running 2 webservers on internal network?

If that is the case why not keep things simple.

Setup 2nd virtual interface on WAN.

Forward port 80 on first IP to web#1
Forward port 80 on second IP to web#2.

vincentfox wrote:

Are you running 2 webservers on internal network?

If that is the case why not keep things simple.

Setup 2nd virtual interface on WAN.

Forward port 80 on first IP to web#1
Forward port 80 on second IP to web#2.

While I do my homework on layer7/conntrack, vincentfox - doesn't a 2nd virtual interface need a second IP from ISP ?  I don't have that - just have one IP from ISP.

(Last edited by venk25 on 6 Feb 2006, 23:51)

Ah I see. My ISP offered additional static IP addresses.

Seemed like a simpler solution. Good luck!

Other than the iptables solution already mentioned, here's another hack that's not as clean:

1. Setup a basic httpd with cgi on the router WAN port 80.
2. Setup a port forward on 2 other ports to your network's two web servers (eg. 8080, 8081)
3. Create a cgi script to parse the Host Header and do a Location: (302?) response to the proper port number.

From that point on, the router's httpd server isn't involved: all communications go directly to the respective internal machine.

You could get a little crazier and customize the 404 page to do the same, but intelligently redirect them to the proper full URL at the destination.

I don't think you can do this with iptables. A TCP connection already has to be established _before_ the client starts sending the HOST header, by that time it's too late to DNAT traffic. Some very simple proxy server, without any caching at all, sounds like what you need. Or the 302 redirect that jeff proposed.

The "Netfilter Extensions HOWTO", available from the Documentation section of www.netfilter.org, has some nice explanation for ipt_string.
The ipt_webstr match module has been derived by CyberTAN (for Linksys), and not been updated for quite a while, and I'm still trying
to find out about its current status. It has been dropped from an early version of OpenWRT, thus I suppose there have been license issues...

Hi Venk25 et.al.,

I have just ported pound 2.1.3 to OpenWrt (mipsel) and will put all the stuff to https://schafserv.homelinux.net/WRT/ ( not browsable)
Pound only works with thread enabled openssl which is not the case for OpenWRT's openssl. You need to recompile the package by the help of the OpenWRT SDK  with a changed configure option in the package Makefile --> change no-thread to thread !!

You can download both binary packaged form here:

  https://schafserv.homelinux.net/WRT/lib … mipsel.ipk
  https://schafserv.homelinux.net/WRT/ope … mipsel.ipk
  https://schafserv.homelinux.net/WRT/lib … mipsel.ipk
  https://schafserv.homelinux.net/pound_2 … mipsel.ipk
 

For building you own packages you need:

SDK: http://downloads.openwrt.org/whiterussi … -1.tar.bz2

openssl ipk source files:  https://schafserv.homelinux.net/WRT/openssl-pack.tar.gz  for building openssl with threads:
pound ipk source files:     https://schafserv.homelinux.net/WRT/pound-pack.tar.gz
                                      --> both to be unpacked in $YourPAth/OpenWrt-SDK-Linux-i686-1/package

pound application source file: https://schafserv.homelinux.net/WRT/pound-2.1.3.tar.bz2
                                            --> to be placed in $YourPAth/OpenWrt-SDK-Linux-i686-1/dl ( Do NOT UNPACK IT !)

I had to repack the original source from http://www.apsis.ch/pound/ because they are using capital letters for the directory name and the package name but using small letters for the application name itselfs. This was very ugly according to the OpenWrt SDK Makefile using the built-in macros - therefore I have changed all captial letters to small one. 

--> Just go to  $YourPAth/OpenWrt-SDK-Linux-i686-1/ and type 'make' and all needed packages will hopefully be built.

Good luck

Hello oltmx, been trying to download your pound port but I get a 404. Let me know if you need some hosting.

millette wrote:

Hello oltmx, been trying to download your pound port but I get a 404. Let me know if you need some hosting.

Yes, this is right. I am running a web server behind DSL and using and can only use one webserver to forward the port to.  Therefore the https port is currently not reachable (running ssl explorer on it) for testing purpose.  Yes I am looking for an external hoster or cvs/svn site.

regards

I'm also *very* interested in this.  I'm willing to both externally host via web page and provide you a svn trunk.  Mail me at rhester@multics.dynalias.com for more info.

Rodney

Hi Rodney,et.al.,

I have pushed all packages and sources to a new site:

http://www.therock-tanga.org/pound4wrt/

Be aware this software still have problems - on my WRT54GS it does not start and produces core dumps. When I will have more time I am trying to fix it. It seems to be a memory problem - probably a little/big endian issue .?

!! This site is temporary as well !! If somebody is going to host the pound port - very welcome.

regards

Olaf

(Last edited by oltmx on 13 Oct 2006, 21:53)

I too am very interested in a working port of pound to openwrt.

Sorry didn't visit this site for a while ...

After my initial posts, I did manage to port pound to OpenWRT - but built with RC4 SDK.  I had to comment out some stuff from pound source (If my memory serves me right, some #if checks on one or two header files).  There were a few warnings but it does run.  I still have one of my routers running RC4 with the pound package I built.  Shortly after that, I ran out of cycles to make it a clean "ipkg" package - meaning it doesn't automatically handle dependencies (though there were just one or two dependencies - sorry don't remember now) - so didn't post it back here.

I want to revive my work - but I lost my tree somehow.  I'll try to get it going again when I have some cycles but really can't say when.  If folks want to try, I can post the executable (built with RC4 SDK - not sure if it means anything or if it'll run on RC5).  You can try to satisfy the dependencies and see if it works.

I did however notice that when pound starts, it has high CPU usage for initial few seconds.  Then it runs fine.  I haven't stress-tested or benchmark-tested though.

Thanks !

The discussion might have continued from here.