OpenWrt Forum Archive

Topic: adblock package, release 2.x

The content of this topic has been archived between 22 Mar 2018 and 4 May 2018. Unfortunately there are posts – most likely complete pages – missing.

WildByDesign wrote:

First, just to confirm that my setup is functioning correctly:

Thu Mar 17 14:02:04 2016 user.notice adblock[7532] info : domain adblock processing started (1.0.0, 15.05.1, 17.03.2016 14:02:04)
[...]

the adblock log looks quite fine. smile

WildByDesign wrote:

Is there a way that I can utilize your adblock for filtering and also have DNS requests forwarded through OpenDNS or other?

yes, define the custom DNS servers for your 'wan' interface:

In LuCI: Network => Interfaces => WAN => EDIT => Common Configurations => Advanced Settings
=> Use DNS servers advertised by peer    Uncheck
=> Use Custom DNS Servers                  [Add your DNS servers here]
=> Save and Apply
=> Reboot

The clients on your network don't query the DNS servers you set up in the 'wan' config directly.  When they request an IP via DHCP they are also supplied a list of name servers (this list of name servers only points to the router by default) ... the client sends a DNS request to the router and then dnsmasq on the router uses its own list of nameservers (the ones you defined) to make the request and return the answer to the client.

Hope this helps!

dibdot wrote:

Hope this helps!

Absolutely, you have been a great help.  Your recommendations and suggestions are working flawlessly now for my entire network. Thank you for your time.

Dear Dirk,

Nice work! By coincidence I found this thread, sadly a bit too late because I was in belief that nothing like an sophisticated adblock package existed for OpenWRT. That's why I recently started to develop something similar by myself.

The code is more or less based on pieces of semi-working snippets I've found throughout the web, and its focus is to support low-mem devices (>=8MB): https://github.com/katakombi/NFguardian/

My primary goals:

  • block porn sites - DONE using a compressed static host file, not very ideal solution

  • block ads (with dynamic updates & time stamp check) - DONE

  • block certain categories like porn/adult/violence/gambling etc based on web resources - TODO, seems undoable for low-mem devices

  • black&whitelists - almost DONE

  • store up-to-date host lists permanently in compressed form/restore on reboot - DONE

  • root can remove/add new lists - DONE manual cfg, no luci support

My latest 0.4 release has been developed and tested using the TP-Link TL-WR1043ND with 8MB but should just work fine on any router providing similar or a higher amount of RAM.
Just check the files folder if you want to see the code (it's not packaged).
There's just two files that contain the logic:

https://raw.githubusercontent.com/katak … ithost.cfg
https://raw.githubusercontent.com/katak … d/inithost

The persistent host files are stored in compressed form and can link to various partitions to optimize each images individual MTD layout.
I'd be very interested to learn new ideas from you about how to lower the mem consumption further, and your and other users experiences on how to exactly pre-determine dnsmasqs mem consumption when using large host lists.

cheers & happy eastern Stefan

caetydid wrote:

Nice work! By coincidence I found this thread, sadly a bit too late because I was in belief that nothing like an sophisticated adblock package existed for OpenWRT. That's why I recently started to develop something similar by myself. The code is more or less based on pieces of semi-working snippets I've found throughout the web, and its focus is to support low-mem devices (>=8MB)

Hi Stefan,

there is a close connection between number of activated adblock sites/lists and space requirements ... the default configuration should run just fine, even on low power devices. The adblock package itself has nearly "no" space requirements. I would simply suggest to use the existing, fully functional adblock package .. if you still like to invent your own solution, just a few ideas ...

  • please make sure that dnsmasq is still up & running after you've applied new block lists

  • do not use hphosts files without permission, please see/honor their terms of use on the homepage ("ALL automated use is strictly forbidden")

  • maybe it's better to use 'wget' than 'curl', the latter one has https issues in openwrt

  • check the categories of shallalist.de if you focus on porn site blocking

Happy Easter!
Dirk

Hi
What's the filename of the package corresponding to luci-app-adblock?
I cannot find it in the packages repository (at least with this name)

Menion wrote:

Hi
What's the filename of the package corresponding to luci-app-adblock?
I cannot find it in the packages repository (at least with this name)

Please look/download here (as stated in the readme)...

adblock
luci-app-adblock

Hi, thanks
Eventually I have found it, luci-app-adblock is available only in DD baseline (even if it works in CC luci also).
It was not clear to me, reading the readme

How much disk space and memory is required to run the basic filter?

trumee wrote:

How much disk space and memory is required to run the basic filter?

The script itself is lightweight ... of course the block lists take some space, see below (sorted by size).
Only the sources adaway, disconnect and yoyo are enabled by default (together approx. 200K):

root@OpenWrt:/tmp/dnsmasq.d$ ls -laSh
-rw-r--r--    1 root     root      917.1K Apr  1 13:29 adb_list.shalla
-rw-r--r--    1 root     root      542.9K Apr  1 13:29 adb_list.malware
-rw-r--r--    1 root     root      430.0K Apr  1 13:29 adb_list.winhelp
-rw-r--r--    1 root     root      422.1K Apr  1 13:29 adb_list.whocares
-rw-r--r--    1 root     root      305.1K Apr  1 13:29 adb_list.sysctl
-rw-r--r--    1 root     root      220.2K Apr  1 13:29 adb_list.spam404
-rw-r--r--    1 root     root      151.7K Apr  1 13:29 adb_list.dshield
-rw-r--r--    1 root     root       95.3K Apr  1 13:29 adb_list.disconnect
-rw-r--r--    1 root     root       82.2K Apr  1 13:29 adb_list.yoyo
-rw-r--r--    1 root     root       71.6K Apr  1 13:29 adb_list.ruadlist
-rw-r--r--    1 root     root       60.2K Apr  1 13:29 adb_list.malwarelist
-rw-r--r--    1 root     root       57.4K Apr  1 13:29 adb_list.openphish
-rw-r--r--    1 root     root       16.0K Apr  1 13:29 adb_list.zeus
-rw-r--r--    1 root     root       15.2K Apr  1 13:29 adb_list.adaway
-rw-r--r--    1 root     root         646 Apr  1 13:29 adb_list.palevo

Does this adblock script supports uclient-fetch on OpenWrt?

written_direcon wrote:

Does this adblock script supports uclient-fetch on OpenWrt?

No, it uses the wget package ... mainly for Chaos Calmer backward compability.

Ok. Do you plan to add support for it in the future, maybe optional or to switch between wget and uclient-fetch?

written_direcon wrote:

Ok. Do you plan to add support for it in the future, maybe optional or to switch between wget and uclient-fetch?

adblock 1.1.x will support uclient-fetch (only) ... no switches to wget, no further package dependencies - keep it simple. wink

Very good :-)

I can't seem to find a way to add the luci interface or where to download the IPK so i can install it

i have Tplink 4300 running 15.05.1 so transferred the ipk using wincp to tmp and installed, it is working but no clue where to get the luci app

Thanks for this great addition

goodland1988 wrote:

I can't seem to find a way to add the luci interface or where to download the IPK so i can install it

i have Tplink 4300 running 15.05.1 so transferred the ipk using wincp to tmp and installed, it is working but no clue where to get the luci app

The package luci-app-adblock is available only sources for 15.05.1, as it has been added after the 15.05.1 release. You can comple it yourself.

But the trunk Luci package should work also in 15.05.1. Just download the correct trunk package and use opkg to install the file.
For example:

cd /tmp
wget http://downloads.openwrt.org/snapshots/trunk/ar71xx/generic/packages/luci/luci-app-adblock_git-16.092.45070-c5c199b-1_all.ipk
opkg install luci-app-adblock_git-16.092.45070-c5c199b-1_all.ipk

That will also require the adblock package itself, so you should do the same for that.

dibdot wrote:
written_direcon wrote:

Ok. Do you plan to add support for it in the future, maybe optional or to switch between wget and uclient-fetch?

adblock 1.1.x will support uclient-fetch (only) ... no switches to wget, no further package dependencies - keep it simple. wink

Dirk, have you heard anything new from Felix regarding libustream-polarssl fix? uclient-fetch works fine if you install libustream-openssl (which brings libopenssl along for the party), but not with the polarssl. sad

I thought it was supposed to be fixed in late 48xxx builds in January, but I've tried the trunk libustream-polarssl package with February datestamp and it still fails on the https links.

stangri wrote:
dibdot wrote:
written_direcon wrote:

Ok. Do you plan to add support for it in the future, maybe optional or to switch between wget and uclient-fetch?

adblock 1.1.x will support uclient-fetch (only) ... no switches to wget, no further package dependencies - keep it simple. wink

Dirk, have you heard anything new from Felix regarding libustream-polarssl fix? uclient-fetch works fine if you install libustream-openssl (which brings libopenssl along for the party), but not with the polarssl. sad

I thought it was supposed to be fixed in late 48xxx builds in January, but I've tried the trunk libustream-polarssl package with February datestamp and it still fails on the https links.

I've (re-)tested yesterday adblock https sites with different uclient-fetch combinations and found ssl troubles with current DD trunk. I've filed a ticket ... at this stage uclient-fetch is no alternative for wget. Anyway, there is no hurry to make the switch ...

Weird, I'm on r49088 still and uclient-fetch works just fine with https links paired with libustream-openssl.

Looks like the polarssl library does not work, but openssl works, like stangri said:

I tested uclient-fetch with all three ustream ssl variants: openssl, polarssl and mdebtls
Both openssl and mbedtls were able to download the three links above, but the polarssl variant failed all three.

As polarssl is the default in Openwrt, making adblock to depend on openssl might increase the size of installed packages.

In any case, Openwrt devs should maybe consider switching the default from the old polarssl to its newer version mdebtls as soon as the different dependent programs have been converted to support mbedtls. (px5g and openvpn are probably the two most critical apps.)

(Last edited by hnyman on 6 Apr 2016, 11:56)

I installed the adblock package via luci gui. It gives me the following error:

Wed Apr  6 11:23:47 2016 user.notice adblock[29765] info : domain adblock processing started (1.0.2, r47075, 06.04.2016 11:23:47)
Wed Apr  6 11:23:47 2016 user.notice adblock[29765] error: no valid IPv4/IPv6 configuration for given logical LAN interface found (eth1), please set 'adb_lanif' manually, rc: 130
Wed Apr  6 11:23:47 2016 user.notice adblock[29765] info : domain adblock processing failed (1.0.2, r47075, 06.04.2016 11:23:47)

I have my router in bridged mode, acting as an access point for my main network. Any help is much appreciated!

kingflurkel wrote:

I installed the adblock package via luci gui. It gives me the following error:

Wed Apr  6 11:23:47 2016 user.notice adblock[29765] info : domain adblock processing started (1.0.2, r47075, 06.04.2016 11:23:47)
Wed Apr  6 11:23:47 2016 user.notice adblock[29765] error: no valid IPv4/IPv6 configuration for given logical LAN interface found (eth1), please set 'adb_lanif' manually, rc: 130
Wed Apr  6 11:23:47 2016 user.notice adblock[29765] info : domain adblock processing failed (1.0.2, r47075, 06.04.2016 11:23:47)

I have my router in bridged mode, acting as an access point for my main network. Any help is much appreciated!

If I understand correctly your router has no wan interface in bridged mode, correct? Than it's a not supported configuration, you can't use this package ... for further information look here

Thanks Didot! I just did a new openwrt install on the router, just to be sure I'm starting from scratch. I have a lan and wan interface now, but adblock is not showing up in the software list. I guess I have to add the repo somewhere, was trying to figure that out now.

kingflurkel wrote:

Thanks Didot! I just did a new openwrt install on the router, just to be sure I'm starting from scratch. I have a lan and wan interface now, but adblock is not showing up in the software list. I guess I have to add the repo somewhere, was trying to figure that out now.

For download links, please look in the readme (see first post of this thread).

hnyman wrote:

Looks like the polarssl library does not work, but openssl works, like stangri said:

I tested uclient-fetch with all three ustream ssl variants: openssl, polarssl and mdebtls
Both openssl and mbedtls were able to download the three links above, but the polarssl variant failed all three.

As polarssl is the default in Openwrt, making adblock to depend on openssl might increase the size of installed packages.

In any case, Openwrt devs should maybe consider switching the default from the old polarssl to its newer version mdebtls as soon as the different dependent programs have been converted to support mbedtls. (px5g and openvpn are probably the two most critical apps.)

Also works with libustream-cyassl/libcyassl. Just libustream-polarssl is a problem. And I thought it's been fixed by Felix back in January. We're getting severely OT here, I'll start a thread in Dev section.