OpenWrt Forum Archive

Topic: squid 3.4 on openwrt cosmic calmer

The content of this topic has been archived on 30 Apr 2018. There are no obvious gaps in this topic, but there may still be some posts missing at the end.

Hi,

Squid seems to have vanished from the opkg package tool, looking at the sticky I'm guessing this was because no one wanted to maintain it (I'd volunteer but I don't have the knowledge or the time ;-)

I did eventually manage to get this all working so I thought I'd leave a note on the forum in-case anyone else is interested in this.  I did try polipo but it doesn't suit my needs.

To cross compile squid:

1:  setup build environment according to openwrt instructions http://wiki.openwrt.org/doc/devel/crosscompile

2:  download latest stable squid and extract.  I tried 3.4.7 and it works great

3:  run configure enabling cross compile options and turning off parts we're not using

./configure \
    --build i386-uknown-linux-gnu \
    --host mips-openwrt-linux-uclibc \
    --disable-external-acl-helpers \
    --disable-auth-negotiate \
    --disable-auth-ntlm \
    --disable-auth-digest \
    --disable-auth-basic \
    --disable-wccp \
    --disable-wccpv2 \
    --disable-snmp \
    --disable-htcp \
    --enable-linux-netfilter \

4:  compile

make

5:  package

mkdir build

# DESTDIR must be a full absolute path so change to suit your system
make DESTDIR=/home/vagrant/squid-3.4.7/build install
tar -C build/ -zcvf squid.tar.gz . 

6:  copy the squid.tar.gz file to the router using scp

7:  install supporting libraries from opkg

opkg install libpthread
opkg install libstdcpp
opkg install librt

8:  extract tarball, assuming you uploaded to /root

cd /
tar zxvf /root/squid.tar.gz

9:  create a squid config file somewhere - I put mine at /etc/squid.conf and based it on the example file from /usr/local/squid/etc/squid.conf.default


10:  create an openwrt init script.  I wrote a basic one that handles starting, stopping, creating log and cache directories.  The values for these locations are sourced from the squid config file as specfied in the init script https://gist.github.com/GeoffWilliams/4 … e93c1a282e.

11:  install the init script and enable.  I copied my script to /etc/init.d/squid so to enable it I did:

/etc/init.d/squid enable

That was pretty much all I had to do.  Squid 3.4 compiles nicely out of the box for MIPS which is fantastic.  If you find your still getting i386 builds, make sure you properly follow all of the instructions on the openwrt cross compiling help page, if I don't set things up properly by setting the PATH and STAGING_DIR variables g++ tells me its cross compiling but produces i386 binaries regardless.

Sample config file (cache system updates only, enable interception proxy):  https://gist.github.com/GeoffWilliams/8 … d417a38bac

Sample firewall rules to enable intercept proxy (adjust addresses to suit your network):  https://gist.github.com/GeoffWilliams/a … 2f7a5e289e

Have fun openwrt squidding.

(Last edited by geoffw on 29 Sep 2014, 05:36)

Excellent. I have been looking for this for months. I will appreciate an in-depth description of the directory structure (where to extract the squid files) and any menuconfig or kernel_menuconfig  setting needed.

I have created small packages for openwrt other than that I'm not an expert but I can follow instructions.

I will try it over the weekend. I want this mainly because  squid 3.4 allows preserving the dscp/tos from the remote server and embeds to the client connection according to squid blog. The current squid on openwrt trunk is 2.7 which does not help me do what I want to achieve.

[Remote Server] -----> [squid] ------> [client]
dscp = 0x24 -----> copy over -----> dscp 0x24

Thanks in advance

geoffw wrote:

Squid seems to have vanished from the opkg package tool, looking at the sticky I'm guessing this was because no one wanted to maintain it (I'd volunteer but I don't have the knowledge or the time ;-)

2:  download latest stable squid and extract.  I tried 3.4.7 and it works great


Would you be willing to share your squid binary? I assume it is for mipsel based on your configure.

I am unable to get it to compile, I only have centos for a cross-compile dev environment and it just won't build.

I tried to manually extract the Squid 3.1.8 from the Optware ipk and manually install it but there is something funky with how it was built and it just will not run properly (segfaults shortly after start)

I really cannot believe no-one wants to update the ancient squid 2.7 from entware, it is massively behind at this point and even has several CVE security bulletins so it is dangerous to run.

Thanks to geoffw for this thread and after spending a day of trial/error and research I finally was able to build Squid 3.5 for mipsel on the openwrt variant for rt-n56u/rt-n65u/rt-n65r routers

It uses ralink chipset (rt3883) which is slightly different so I had to use their toolchain with 3.x kernel

https://code.google.com/p/rt-n56u/wiki/ … keFirmware

Just to add some tips for people trying this in the future:

I made a clean new Debian install in a VMware box, my centos wasn't cutting it

I had to build under /home/ instead of /opt/ for space reasons

This is what my environment exports looked like:

export  PATH="/home/slug/rt-n56u/toolchain-rt3883/toolchain-3.0.x/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
export STAGING_DIR="/home/slug/rt-n56u/toolchain-rt3883/toolchain-3.0.x"
export CC=mipsel-linux-uclibc-gcc 
export LD=mipsel-linux-uclibc-ld

(yes I put the cross compiler path first just to be certain the local gcc wasn't used)

I ran into a problem with cf_gen, which is built for the mipsel environment but then the makefile tries to execute locally. So to fix that you first have to compile locally without cross-compile and skipping the above environment so it makes cf_gen for x86_64 - then copy it and patch the Makefile. You can find the instructions here:

https://github.com/pjq/squid/tree/master#issues-1

My configure triplet looked like this

./configure --build=x86_64-linux-gnu --host=mipsel-linux-uclibc --target=mipsel-linux-uclibc

But I also ran into a problem where the compiler would quit on warnings, Squid 3.5 introduces a bunch of new stuff, specifically ftprelay was causing the warning->fatal and I won't be using anything but port 80 so I disabled the fatal warning with --disable-strict-error-checking



There is one last problem I ran into that I have not solved yet:

could not enable Tproxy4 http://wiki.squid-cache.org/Features/Tproxy4

this is because even though I can install libcap 2.24 on the router via opkg entware, there is no libcap or libcap-dev in the toolchain on the cross-compiler side?

I am not sure how to resolve that, I guess I could copy the libcap files from the router back to the toolchain but libcap-dev package does not exist and not sure how to create it

(Last edited by _ck_ on 8 Dec 2014, 15:14)

Hi there,

Apologies for not following up earlier, its been a really busy week.  I did my build on debian and it seemed to work fine.  I'm going to attempt to have a go at making an opkg package next time I have some free time - probably in the next week or so.

Cheers,
Geoff

geoffw wrote:

I did my build on debian and it seemed to work fine.  I'm going to attempt to have a go at making an opkg package next time I have some free time - probably in the next week or so.

That would be helpful since the version I attempted to create for myself turns out to be unstable and unusable, had to roll back to 2.7

Re-reading your original post it looks like you made a mips package, perhaps you would consider also making an attempt at mipsel once you are confident about your package making ability.

On second thought I need many more configure options than you use so I still need to figure this out for myself, it is frustrating and I already have far too many hours into attempting this. Not sure what I am missing but I guess I better re-read all the instructions from the start again.

Did you run into the same cf_gen problem that I did? You didn't mention it and that is strange.

(Last edited by _ck_ on 11 Dec 2014, 07:43)

I finally got a chance to look at this today but someones already packaged squid 3.5 on trunk:

https://github.com/openwrt/packages/tre … /net/squid

I *think* if you can get openwrt trunk flashed on your router it can be installed with opkg

I just attempted to install this version of squid and it installs with a few quirks (Missing dependencies).  I am unable to start squid I receive an error related to shm(?).

FATAL: Ipc::Mem::Segment::create failed to shm_open(/squid-ssl_session_cache.shm): (13) Permission denied

Any suggestion what exactly this is referring to?  Is this a configuration issue on my part? (Currently running  r44622)

otte_1981 I just installed on chaos calmer and it worked fine.  (13) permission denied looks like file permissions or an SELinux error maybe?

After flashing I did opkg install squid and didn't get any dependency errors - got squid 3.4 but that's fine by me.

maybe try to start squid in the foreground?

# squid -X -d 1 -N

This should hightlight any errors detected inside squid

I will try what you suggest.  But I think I might open another thread instead of piggybacking on this one. Much  Thank

@geoff

# squid -X -d 1 -N
Assisted with tracking down my issue.  It seems that when dropping group 'root' to 'nobody' /dev/shm becomes unaccessible.  Changed group to /dev/shm and it started working.  However, I'm not familiar with exactly what /dev/shm function is exactly.

I created a ln to /dev/shm to an external USB connected due to those concerns.   Once again not sure of the repercussions to this.

Is there an alternative to use as a http/s forwarder only on a local openwrt device, to send the traffic to a squid proxy server?

The discussion might have continued from here.