OpenWrt Forum Archive

Topic: adblock package, release 2.x

The content of this topic has been archived between 22 Mar 2018 and 4 May 2018. Unfortunately there are posts – most likely complete pages – missing.

@dibdot, many thanks for the great idea!

I have installed the adblock on Asus RTN-66u (32ROM 256RAM) + OpenWrt Chaos Calmer 15.05 / LuCI Master (git-15.233.47308-791ca8b)

I tested adblock on the last assembly OpenWRT DD. Everything worked. Now I decided to install adblock on the stable release of OpenWRT, and here I have this error:

/etc/config$ /usr/bin/adblock-update.sh
adblock[26621] info : domain adblock processing started (0.40.2, 15.05, 17.01.2016 22:20:38)
adblock[26621] info : backup/restore will be disabled
adblock[26621] info : dns query logging will be disabled
adblock[26621] info : debug logging will be disabled
adblock[26621] info : wan update check will be disabled
adblock[26621] info : ntp time sync will be disabled
adblock[26621] info : no online timestamp received, current date will be used (adaway)
adblock[26621] error: source download failed (adaway), rc: 2
adblock[26621] info : domain adblock processing finished (0.40.2, 15.05, 17.01.2016 22:20:40)

I have installed libpolarssl.

Where to find the problem?

@Alex013: I've never tested the adblock package with CC. Anyway, 'adaway' is a https site so wget will be used for download. wget rc2 is a parse error. Please drop me a mail to my package maintainer email address and I'll try to backport this package to CC, too.

@dibdot: Yes, it's true! I'm banned in all https and allow all http:

/etc/config$ /usr/bin/adblock-update.sh
adblock[16780] info : domain adblock processing started (0.40.2, 15.05, 18.01.2016 00:28:41)
adblock[16780] info : backup/restore will be disabled
adblock[16780] info : dns query logging will be disabled
adblock[16780] info : debug logging will be disabled
adblock[16780] info : wan update check will be disabled
adblock[16780] info : ntp time sync will be disabled
adblock[16780] info : shallalist (pre-)processing started ...
adblock[16780] info : source download finished (http://www.dshield.org/feeds/suspiciousdomains_Low.txt, 4542 entries)
adblock[16780] info : domain merging finished (dshield)
adblock[16780] info : source download finished (http://mirror1.malwaredomains.com/files/justdomains, 19286 entries)
adblock[16780] info : domain merging finished (malware)
adblock[16780] info : source download finished (http://spam404bl.com/spam404scamlist.txt, 5193 entries)
adblock[16780] info : domain merging finished (spam404)
adblock[16780] info : source download finished (http://someonewhocares.org/hosts/hosts, 11989 entries)
adblock[16780] info : domain merging finished (whocares)
adblock[16780] info : source download finished (http://winhelp2002.mvps.org/hosts.txt, 13573 entries)
adblock[16780] info : domain merging finished (winhelp)
adblock[16780] info : source doesn't change, no update required (yoyo)
adblock[16780] info : source download finished (file:////tmp/tmp.7A2xgd/shallalist.txt, 32737 entries)
adblock[16780] info : domain merging finished (shalla)
adblock[16780] info : source doesn't change, no update required (blacklist)
adblock[16780] info : remove duplicates in separate adblocklists ...
adblock[16780] info : adblocklists with overall 82222 domains loaded, backups disabled
adblock[16780] info : domain adblock processing finished (0.40.2, 15.05, 18.01.2016 00:30:29)

But my level of English does not allow me to understand that I have to send for you to e-mail sad ...

(Last edited by Alex013 on 17 Jan 2016, 22:38)

@Alex013: That looks quite good. I'm just looking out for a CC beta tester ... maybe you're willing to test a pre-release ... ;-)

regards
dirk

P.S. you can write me in german language as well (if this helps?)

@dibdot: Thank you very much for your suggestions. I am willing to try to be a tester.

A German language, unfortunately, I do not know to. Maybe you know Russian? wink

(Last edited by Alex013 on 18 Jan 2016, 19:03)

@dibdot: I have tried to replace the wget 1.17.1 (DD) of the repository trunk. But the CC refused to work with this version of the wget.

The CC uses the wget 1.16.3-1. The description of the wget 1.16.3-1 contains support ssl:

"Wget is a network utility to retrieve files from the Web using http and ftp, the two most widely used Internet protocols. It works non-interactively, so it will work in the background, after having logged off. The program supports recursive retrieval of web-authoring pages as well as ftp sites -- you can use wget to make mirrors of archives and home pages or to travel the Web like a WWW robot. This package is built with SSL support."

It is not enough to have downloading via HTTPS protocol?

(Last edited by Alex013 on 18 Jan 2016, 12:47)

@dibdot: Please forgive me, but I have another question. Do you think it is right that error when downloading is interrupted work adblock? One of the websites can be temporarily unavailable. Why because of the unavailability one of "black_list_website" the remaining websites will not be processed?
Here's an example of temporary unavailability the yoyo:

/etc/adblock$ /usr/bin/adblock-update.sh
adblock[2362] info : domain adblock processing started (0.40.2, 15.05, 18.01.2016 13:59:45)
adblock[2362] info : backup/restore will be disabled
adblock[2362] info : dns query logging will be disabled
adblock[2362] info : debug logging will be disabled
adblock[2362] info : wan update check will be disabled
adblock[2362] info : ntp time sync will be disabled
adblock[2362] info : shallalist (pre-)processing started ...
adblock[2362] info : source doesn't change, no update required (dshield)
adblock[2362] info : source doesn't change, no update required (malware)
adblock[2362] info : source doesn't change, no update required (spam404)
adblock[2362] info : source doesn't change, no update required (whocares)
adblock[2362] info : no online timestamp received, current date will be used (winhelp)
adblock[2362] info : source download finished (http://winhelp2002.mvps.org/hosts.txt, 13573 entries)
adblock[2362] info : domain merging finished (winhelp)
adblock[2362] info : no online timestamp received, current date will be used (yoyo)
adblock[2362] error: source download failed (yoyo), rc: 6
adblock[2362] info : domain adblock processing finished (0.40.2, 15.05, 18.01.2016 14:01:33)

The five minutes later, everything was in order:

adblock[11885] info : domain adblock processing started (0.40.2, 15.05, 18.01.2016 14:06:01)
adblock[11885] info : backup/restore will be disabled
adblock[11885] info : dns query logging will be disabled
adblock[11885] info : debug logging will be disabled
adblock[11885] info : wan update check will be disabled
adblock[11885] info : ntp time sync will be disabled
adblock[11885] info : source doesn't change, no update required (dshield)
adblock[11885] info : source doesn't change, no update required (malware)
adblock[11885] info : source doesn't change, no update required (spam404)
adblock[11885] info : source doesn't change, no update required (whocares)
adblock[11885] info : source download finished (http://winhelp2002.mvps.org/hosts.txt, 13573 entries)
adblock[11885] info : domain merging finished (winhelp)
adblock[11885] info : source doesn't change, no update required (yoyo)
adblock[11885] info : source doesn't change, no update required (shalla)
adblock[11885] info : source doesn't change, no update required (blacklist)
adblock[11885] info : remove duplicates in separate adblocklists ...
adblock[11885] info : adblocklists with overall 82223 domains loaded, backups disabled
adblock[11885] info : domain adblock processing finished (0.40.2, 15.05, 18.01.2016 14:07:01)

But the adblock updated twice a day. The next time may not be available to another website. It may be better to skip temporarily inaccessible websites?

(Last edited by Alex013 on 18 Jan 2016, 12:48)

Hi, i'am tryiing for a few hours now, but do not get it working further than this below,

adblock[4954] info : domain adblock processing started (0.40.2, r48259, 24.01.2016 16:24:26)
adblock[4954] info : overall sort/unique processing will be disabled,
adblock[4954] info : please consider adding an external swap device to supersize your /tmp directory (total: 60808, free: 19768, swap: )
adblock[4954] info : backup/restore will be disabled
adblock[4954] info : dns query logging will be disabled
adblock[4954] info : debug logging will be disabled
adblock[4954] info : wan update check will be disabled
adblock[4954] info : ntp time sync will be disabled
adblock[4954] info : created new dynamic/volatile network interface (adblock, 10.0.2.1)
adblock[4954] error: failed to initialize new dynamic/volatile uhttpd instance (adblock, 10.0.

--
who can help me configure this? Thanks

Hi @dibdot, Huge thanks for finally making an openwrt port of this adblock script!

I have installed the script and everything runs successfully on the first install, however it seems like there are still many ads not being blocked. I'm guessing this is because I have not properly configured the pixel server.
For instance, the ads that are blocked just don't resolve the page like ---> this - https://i.imgur.com/JUYdqIv.png instead of showing the blank pixel.

But even then there are still basic adsense ads that I would expect to return the same resolve error, https://i.imgur.com/4hoZmwn.png yet it still feels as though adsense is unblocked.

So yeah, it kinda seems like I missed one of the initial settings or something tongue

Thanks again!

(Last edited by cal3b on 3 Feb 2016, 17:21)

@cal3b: https connections can't be redirected to adblock pixel server, cause a https connection requires a valid certificate (i.e. from doubleclick.net). We can't fake certificates, therefore adblock simply rejects all ad related things <> port 80.

The adsense listing itself is OK, cause the adsense entry comes from goggle.com ... but the underlying adsense link (to googleadservices.com) is dead.

For additional domain blocking you can manual update adblock black- and white-lists. They are located under /etc/adblock/...

Hope this helps!
dirk

dibdot wrote:

https connections can't be redirected to adblock pixel server, cause a https connection requires a valid certificate (i.e. from doubleclick.net). We can't fake certificates, therefore adblock simply rejects all ad related things <> port 80.

Ah, that explains it! I just tried it with non https youtube and it worked great!

And yeah as you guessed the adsense ad was blocked when i tried to click sooo thanks! 

Also, I'm curious if this kind of system wide adblocking would be able block ads on streaming devices such as a Chromecast or Apple TV, with youtube perhaps?

Thanks again!

@cal3b: I didn't have such devices like apple g(ad)gets ... if the ad related things are coming from a different domain than the "real" content you can block them, if they're using the same domain you can't. For domain tracing please set in /etc/config/dhcp

config dnsmasq
[...]
   option logqueries '1'

restart the dnsmasq service and start domain tracing on the router with something like this ...

logread -f -e "dnsmasq" -e "query\[A\]"

After that enable your apple device and you will see which domains will be requested during startup ... happy hunting! wink

dibdot wrote:

therefore adblock simply rejects all ad related things <> port 80.
dirk

How is this done ?
I had a look at the code, but did not undertand enough :-)

augustus_meyer wrote:
dibdot wrote:

therefore adblock simply rejects all ad related things <> port 80.
dirk

How is this done ?
I had a look at the code, but did not undertand enough :-)

It's a firewall rule in the forward chain which rejects all packets destinated to the pre-defined blackhole ip, ad related things on port 80 will be redirected to pixel server in pre-routing chain.

dibdot wrote:

@cal3b: I didn't have such devices like apple g(ad)gets ... if the ad related things are coming from a different domain than the "real" content you can block them, if they're using the same domain you can't. For domain tracing please set in /etc/config/dhcp

config dnsmasq
[...]
   option logqueries '1'

restart the dnsmasq service and start domain tracing on the router with something like this ...

logread -f -e "dnsmasq" -e "query\[A\]"

After that enable your apple device and you will see which domains will be requested during startup ... happy hunting! wink

Awesome going to do this right now!

hi, I state that I have in my wdr3600 I like OpenWRT firmware version Chaos Calmer 15.05 / LuCI (git-15.248.30277-3836b45),
I have the following error when I try to install the package:

:~# opkg install adblock
Unknown package 'adblock'.
Collected errors:
* opkg_install_cmd: Cannot install package adblock.

The package currently exists only in trunk (Designated Driver).

I wrote a simple LuCI user interface for enabling/disabling the adblock package as well as selecting the applied blocklists.

That has been added to the DD trunk as 'luci-app-adblock'.

I am using hnyman's OpenWRT build which includes the adblock package and LuCI user interface on a WNDR3700.
I have a question: I am using my WNDR3700 as a DHCP server and DNS caching server on my LAN. This means that I have a different router that is connected to Internet. The WNDR3700 only has a LAN side. When I enable the adblocking feature I see the following Log entries:

Wed Feb 24 09:36:29 2016 user.notice adblock[6544] info : domain adblock processing started (0.80.0, 15.05-r48749, 24.02.2016 09:36:29)
Wed Feb 24 09:36:29 2016 user.notice adblock[6544] error: LAN only (lan) network, no valid IPv4/IPv6 wan update interface found, rc: 125
Wed Feb 24 09:36:29 2016 user.notice adblock[6544] info : domain adblock processing failed (0.80.0, 15.05-r48749, 24.02.2016 09:36:29)

It seems that the ad-blocking process requires a WAN interface to update. Can this be made working in any way with just a LAN interface?

Thx

(Last edited by Romulus on 24 Feb 2016, 19:34)

No, it's not supported, see this readme chapter for details.

Romulus wrote:

I am using hnyman's OpenWRT build which includes the adblock package and LuCI user interface on a WNDR3700.
I have a question: I am using my WNDR3700 as a DHCP server and DNS caching server on my LAN. This means that I have a different router that is connected to Internet. The WNDR3700 only has a LAN side. When I enable the adblocking feature I see the following Log entries:

Wed Feb 24 09:36:29 2016 user.notice adblock[6544] info : domain adblock processing started (0.80.0, 15.05-r48749, 24.02.2016 09:36:29)
Wed Feb 24 09:36:29 2016 user.notice adblock[6544] error: LAN only (lan) network, no valid IPv4/IPv6 wan update interface found, rc: 125
Wed Feb 24 09:36:29 2016 user.notice adblock[6544] info : domain adblock processing failed (0.80.0, 15.05-r48749, 24.02.2016 09:36:29)

It seems that the ad-blocking process requires a WAN interface to update. Can this be made working in any way with just a LAN interface?

Thx

Currently I'm in preparation for the first stable release, as soon as 1.0 is in DD trunk I plan to backport the package to CC branch, too.

Unfortunately the openwrt download servers are still somewhat broken, therefore I've pulled the latest 'ready to run' adblock package (0.91.0) to my private git repo. You can download the opkg file here.

br
dirk

Edit: removed link to outdated adblock package, release 1.0 is now on github!

Algoalgo wrote:

Is this package going to be added to the official repository soon?

Btw, the link is currently unavailable https://downloads.openwrt.org/snapshots … s/packages

(Last edited by dibdot on 10 Mar 2016, 20:40)

Thanks for your reply.

Just installed the package and started it using putty.
Now, where can i get the luci-app-adblock?
It seems like its not available on CC.

One last question: will this block youtube-preroll ads?

Thanks!

(Last edited by Algoalgo on 7 Mar 2016, 00:03)

if one of your block lists contain domains like

s0.2mdn.net
googleads.g.doubleclick.net
ad.doubleclick.net
files.adform.net
secure-ds.serving-sys.com

then ad related pre-roll youtube videos will probably be blocked.

Algoalgo wrote:

Thanks for your reply.

Just installed the package and started it using putty.
Now, where can i get the luci-app-adblock?
It seems like its not available on CC.

One last question: will this block youtube-preroll ads?

Thanks!