Topic: adblock package, release 2.x

The content of this topic has been archived between 22 Mar 2018 and 4 May 2018. Unfortunately there are posts – most likely complete pages – missing.

Page 14 of 17

Post #326

dibdot

29 Nov 2016, 20:32

huangjqiu wrote:

Thanks dibdot for your response. below is the log where I saw "created volatile uhttpd instances", however I couldn't find any processes with 'ps'. I did notice the process /usr/sbin/uhttpd -f -h /www -r lede -x /cgi-bin -u /ubus was running.

Please send a short email to my maintainer email address (see online documentation on github) and I'll send you a debug script back for further analysis ... thanks.

Post #327

ExaltedVanguard

11 Dec 2016, 03:54

Adblock seems to be starting very unreliably. Some reboots it will start, others it will fail to start at all. Here's the information provided in the system log after a failed-to-start boot.

Sat Dec 10 21:36:27 2016 user.notice adblock[1619] info : adblock service started due to 'ifup' of 'wan' interface
Sat Dec 10 21:36:28 2016 user.notice adblock[1712] warn : adblock installation finished successfully, 'opkg' currently locked by package installer
Sat Dec 10 21:36:30 2016 user.notice adblock[1970] error: adblock service already running (1712)

When it does work, I get repeated errors about the adblock service already running. My best guess (knowing next to nothing) on these errors is that OpenWrt is repeatedly trying to start the service due to how long the script takes to run and some timeout.

It's not a huge deal, since it's not like my router is being rebooted very often, and I have a cronjob to update adblock every morning so it will "fix itself" in less than 24 hours anyway. Less than ideal though.

Post #328

dibdot

20 Dec 2016, 14:30

A new major adblock release 2.0.0 has been pushed on github and will be available soon.

* changed complete dnsmasq handling
  - no longer redirect ad requests to internal web server via firewall 
    rules etc., in fact send back a simple NXDOMAIN for all
    ad related (sub-)domains
  - smaller memory footprint for block lists
  - removed needless uhttpd instances
  - removed needless firewall rules/redirects
* init/hotplug system migrated to procd
  - removed hotplug.d script, now using procd interface trigger
* reduced code size/complexity
  - removed needless internal pre-checks & function blocks,
    no longer rely on a separate helper library
  - removed flash writes to adblock config
* support different download tools like wget (default), aria2c, uclient-fetch, curl (see online doc)
* adblock status/statistics via ubus call (see online doc)
* various bug fixes
* documentation update
* changed makefile copyright notice

Post #329

cybrnook2002

21 Dec 2016, 17:48

I swear there was another area where a conversation on this was happening, I can't seem to find it now that I was to add to it.

But in 2.0 I am seeing this (same as others):

Wed Dec 21 11:46:14 2016 user.notice adblock-[2.0.0] info : status ::: adblock is currently disabled, please set adb_enabled to '1' to use this service
Wed Dec 21 11:46:48 2016 daemon.info dnsmasq[19820]: exiting on receipt of SIGTERM
Wed Dec 21 11:46:48 2016 daemon.err adblock.sh[20644]: udhcpc: started, v1.25.1
Wed Dec 21 11:46:48 2016 daemon.err adblock.sh[20644]: udhcpc: sending discover
Wed Dec 21 11:46:51 2016 daemon.err adblock.sh[20644]: udhcpc: no lease, failing
Wed Dec 21 11:46:51 2016 daemon.info dnsmasq[20873]: started, version 2.76 cachesize 150
Wed Dec 21 11:46:51 2016 daemon.info dnsmasq[20873]: compile time options: IPv6 GNU-getopt no-DBus no-i18n no-IDN DHCP no-DHCPv6 no-Lua TFTP no-conntrack no-ipset no-auth no-DNSSEC no-ID loop-detect inotify
Wed Dec 21 11:46:51 2016 daemon.info dnsmasq[20873]: DNS service limited to local subnets
..................
..................
..................
..................
Wed Dec 21 11:46:52 2016 user.notice adblock-[2.0.0] info : status ::: block lists with overall 38727 domains loaded (LEDE GEMINI 3.0 r2608-05abcf5)

Easy to reproduce. Just disable adblock2.0 via luci. Wait a few, then re-enable.

(Last edited by cybrnook2002 on 21 Dec 2016, 17:50)

Post #330

dibdot

21 Dec 2016, 18:49

cybrnook2002 wrote:

I swear there was another area where a conversation on this was happening, I can't seem to find it now that I was to add to it.

That has been already discussed here ... and it seems that it makes no harm.

Post #331

ExaltedVanguard

21 Dec 2016, 19:34

I'm using 2.0 and it appears to be working great! Thanks!

One caveat is that it causes dnsmasq to spew a bunch of garbage into the system log. I fixed this by editing /etc/init.d/dnsmasq and appending the option "--log-facility=/dev/null"

Not the most ideal of solutions, but it works.

Post #332

dibdot

21 Dec 2016, 21:02

ExaltedVanguard wrote:

I'm using 2.0 and it appears to be working great! Thanks!
One caveat is that it causes dnsmasq to spew a bunch of garbage into the system log. I fixed this by editing /etc/init.d/dnsmasq and appending the option "--log-facility=/dev/null"
Not the most ideal of solutions, but it works.

I've sent a PR to LEDE source to make it configurable via UCI, see here.

Post #333

jglooije

22 Dec 2016, 20:38

Thanks for the great adblocker. Using it for a while now. In the new version 2 there is a problem, when running multiple instances of dnsmasq. The result of the dnsmasq running is not being recognized as "true". The result is "true true" because of the two instances I have. So I changed the code a little bit to resolve(not perfect):

   dns_running="false"

    sync
    "/etc/init.d/${adb_dns}" restart
    while [ ${cnt} -le 10 ]
    do
        dns_result="$(ubus -S call service list '{"name":"dnsmasq"}' | jsonfilter -e '@.dnsmasq.instances.*.running')"
        if [ "${dns_result:0:4}" = "true" ]
        then
            dns_running="true"

Keep up the good work.

Post #334

dibdot

22 Dec 2016, 21:00

jglooije wrote:

Thanks for the great adblocker. Using it for a while now. In the new version 2 there is a problem, when running multiple instances of dnsmasq. The result of the dnsmasq running is not being recognized as "true". The result is "true true" because of the two instances I have. So I changed the code a little bit to resolve(not perfect):

Thanks for the report! Will be fixed in 2.0.2 ... I've released 2.0.1 today (sadly without this one) with the following fixes:

    fixed a dnsmasq restart issue (udhcpc error)
    fixed a long standing corner case bug in "disabled" state (does not remove active block lists!)
    simplified overall sort, removed needless 'for loop'
    cosmetics

Post #335

cybrnook2002

22 Dec 2016, 21:39

thanks for the 2.01 release @dibdot, looking for forward to future releases :-)

If I am reading your change log correctly, I no longer see the pre-req of kmod-ipt-nat6? Is this intentional, or do you really no longer need it to support IPV6?

(Last edited by cybrnook2002 on 22 Dec 2016, 21:39)

Post #336

dibdot

23 Dec 2016, 07:34

@all: adblock update 2.0.2 is on the road ... last github PR before christmas ...

* fixed dnsmasq check if multiple instances are present
* bring back query function on highly demand
* documentation update

Finally, I wish all of you a merry Christmas!

Post #337

dibdot

23 Dec 2016, 07:38

cybrnook2002 wrote:

thanks for the 2.01 release @dibdot, looking for forward to future releases :-)
If I am reading your change log correctly, I no longer see the pre-req of kmod-ipt-nat6? Is this intentional, or do you really no longer need it to support IPV6?

adblock fully supports IPv4/IPv6 but does no longer use error prone external iptables rulesets, http pixel server instances and things like that (see online doc).

Post #338

cybrnook2002

24 Dec 2016, 17:25

dibdot wrote:

cybrnook2002 wrote:
thanks for the 2.01 release @dibdot, looking for forward to future releases :-)
If I am reading your change log correctly, I no longer see the pre-req of kmod-ipt-nat6? Is this intentional, or do you really no longer need it to support IPV6?
adblock fully supports IPv4/IPv6 but does no longer use error prone external iptables rulesets, http pixel server instances and things like that (see online doc).

Thank dibdot! Merry Christmas.

Post #339

imbaSD

28 Dec 2016, 15:29

dibdot ty for ths great package, i update to 2.0.2 and seems work

but, i an very very newbie at this field ( ) and with previous version (1.5) when i restart the package via PuTTy (/etc/init.d/adblock restart) it was displayed every step of package status, now it's not
is there another way to see that?

also at the page Services/Adblock at the right of every source it's not described when it was updated last time and how many urls it download (at 1.5 versions it was displayed)

did i make something wrong?

(Last edited by imbaSD on 28 Dec 2016, 15:30)

Post #340

Comitizer

29 Dec 2016, 18:07

So far this package is great. The only thing I would like is some more stats (similar to Pi Hole https://pi-hole.net/). It would be really cool to see what is being blocked and so forth.

Regardless, I'm loving the system so I wondering whether or not it is a good idea to enable ALL of the ad block lists. Is there any reason not to? Performance? What would the impact be?

Post #341

dibdot

29 Dec 2016, 18:56

imbaSD wrote:

dibdot ty for ths great package, i update to 2.0.2 and seems work
but, i an very very newbie at this field ( ) and with previous version (1.5) when i restart the package via PuTTy (/etc/init.d/adblock restart) it was displayed every step of package status, now it's not
is there another way to see that?
also at the page Services/Adblock at the right of every source it's not described when it was updated last time and how many urls it download (at 1.5 versions it was displayed)
did i make something wrong?

Please check the current online documentation (see the link in the first post). adblock 2.x writes the output only to syslog (logread -e "adblock") and to see more output please set 'adb_debug' to '1', too.

Post #342

imbaSD

29 Dec 2016, 18:58

When i run ubus call service list i receive the following only

        "adblock": {
                "instances": {
                        "adblock": {
                                "running": false,
                                "command": [
                                        "\/usr\/bin\/adblock.sh",
                                        "reload"
                                ],
                                "env": {
                                        "adb_procd": "true"
                                }
                        }
                }
        },
        "adblock_stats": {
                "instances": {

                }
        },

so with ubus call service list '{"name":"adblock_stats"}' | jsonfilter -e '@.*.instances.stats.env' i get nothing

(Last edited by imbaSD on 29 Dec 2016, 19:16)

Post #343

imbaSD

29 Dec 2016, 19:18

dibdot wrote:

imbaSD wrote:
dibdot ty for ths great package, i update to 2.0.2 and seems work
but, i an very very newbie at this field ( ) and with previous version (1.5) when i restart the package via PuTTy (/etc/init.d/adblock restart) it was displayed every step of package status, now it's not
is there another way to see that?
also at the page Services/Adblock at the right of every source it's not described when it was updated last time and how many urls it download (at 1.5 versions it was displayed)
did i make something wrong?
Please check the current online documentation (see the link in the first post). adblock 2.x writes the output only to syslog (logread -e "adblock") and to see more output please set 'adb_debug' to '1', too.

just did it and manage to get some infos what i wanted, ty

Post #344

dibdot

29 Dec 2016, 21:16

imbaSD wrote:

so with ubus call service list '{"name":"adblock_stats"}' | jsonfilter -e '@.*.instances.stats.env' i get nothing

Hmm, work for me ... maybe something goes wrong during reload action!?

root@blackhole:~# ubus call service list '{"name":"adblock"}'
{
    "adblock": {
        "instances": {
            "adblock": {
                "running": false,
                "command": [
                    "\/usr\/bin\/adblock.sh",
                    "reload"
                ],
                "env": {
                    "adb_procd": "true"
                }
            }
        }
    }
}

root@blackhole:~# ubus call service list '{"name":"adblock_stats"}'
{
    "adblock_stats": {
        "instances": {
            "stats": {
                "running": false,
                "env": {
                    "blocked_domains": "136159",
                    "last_rundate": "29.12.2016 21:09:46"
                }
            }
        }
    }
}

Post #345

dibdot

29 Dec 2016, 21:42

Comitizer wrote:

So far this package is great. The only thing I would like is some more stats (similar to Pi Hole https://pi-hole.net/). It would be really cool to see what is being blocked and so forth.

Currently I focus on fast & neat dns adblocking - nothing more. Basic stats are only available via ubus call (see online doc).

Comitizer wrote:

Regardless, I'm loving the system so I wondering whether or not it is a good idea to enable ALL of the ad block lists. Is there any reason not to? Performance? What would the impact be?

No (known) problems with that.

Post #346

imbaSD

29 Dec 2016, 22:03

dibdot wrote:

imbaSD wrote:

so with ubus call service list '{"name":"adblock_stats"}' | jsonfilter -e '@.*.instances.stats.env' i get nothing

Hmm, work for me ... maybe something goes wrong during reload action!?

root@blackhole:~# ubus call service list '{"name":"adblock"}'
{
    "adblock": {
        "instances": {
            "adblock": {
                "running": false,
                "command": [
                    "\/usr\/bin\/adblock.sh",
                    "reload"
                ],
                "env": {
                    "adb_procd": "true"
                }
            }
        }
    }
}

root@blackhole:~# ubus call service list '{"name":"adblock_stats"}'
{
    "adblock_stats": {
        "instances": {
            "stats": {
                "running": false,
                "env": {
                    "blocked_domains": "136159",
                    "last_rundate": "29.12.2016 21:09:46"
                }
            }
        }
    }
}

tried /etc/init.d/adblock reload and restart but still the same

root@ArcherC7:~# ubus call service list '{"name":"adblock_stats"}'
{
        "adblock_stats": {
                "instances": {

                }
        }
}

is there any way to make a clean reinstallation? delete everything related with adblock that remain after uninstallation (opkg remove adblock)

(Last edited by imbaSD on 29 Dec 2016, 22:06)

Post #347

dibdot

29 Dec 2016, 22:07

imbaSD wrote:

tried /etc/init.d/adblock reload and restart but still the same

no errors in the log?

Post #348

imbaSD

29 Dec 2016, 23:09

dibdot wrote:

imbaSD wrote:
tried /etc/init.d/adblock reload and restart but still the same
no errors in the log?

[....]
Fri Dec 30 00:07:58 2016 daemon.info dnsmasq[8473]: using local addresses only for domain 7bpeople.com
Fri Dec 30 00:07:58 2016 daemon.info dnsmasq[8473]: using local addresses only for domain 77tracking.com
Fri Dec 30 00:07:58 2016 daemon.info dnsmasq[8473]: using local addresses only for domain 777partner.com
Fri Dec 30 00:07:58 2016 daemon.info dnsmasq[8473]: using local addresses only for domain 600z.com
Fri Dec 30 00:07:58 2016 daemon.info dnsmasq[8473]: using local addresses only for domain 51yes.com
Fri Dec 30 00:07:58 2016 daemon.info dnsmasq[8473]: using local addresses only for domain 518ad.com
Fri Dec 30 00:07:58 2016 daemon.info dnsmasq[8473]: using local addresses only for domain 50websads.com
Fri Dec 30 00:07:58 2016 daemon.info dnsmasq[8473]: using local addresses only for domain 4d5.net
Fri Dec 30 00:07:58 2016 daemon.info dnsmasq[8473]: using local addresses only for domain 4affiliate.net
Fri Dec 30 00:07:58 2016 daemon.info dnsmasq[8473]: using local addresses only for domain 360yield.com
Fri Dec 30 00:07:58 2016 daemon.info dnsmasq[8473]: using local addresses only for domain 2o7.net
Fri Dec 30 00:07:58 2016 daemon.info dnsmasq[8473]: using local addresses only for domain 2mdn.net
Fri Dec 30 00:07:58 2016 daemon.info dnsmasq[8473]: using local addresses only for domain 24pm-affiliation.com
Fri Dec 30 00:07:58 2016 daemon.info dnsmasq[8473]: using local addresses only for domain 24log.de
Fri Dec 30 00:07:58 2016 daemon.info dnsmasq[8473]: using local addresses only for domain 24log.com
Fri Dec 30 00:07:58 2016 daemon.info dnsmasq[8473]: using local addresses only for domain 247media.com
Fri Dec 30 00:07:58 2016 daemon.info dnsmasq[8473]: using local addresses only for domain 207.net
Fri Dec 30 00:07:58 2016 daemon.info dnsmasq[8473]: using local addresses only for domain 1x1rank.com
Fri Dec 30 00:07:58 2016 daemon.info dnsmasq[8473]: using local addresses only for domain 180searchassistant.com
Fri Dec 30 00:07:58 2016 daemon.info dnsmasq[8473]: using local addresses only for domain 180hits.de
Fri Dec 30 00:07:58 2016 daemon.info dnsmasq[8473]: using local addresses only for domain 123found.com
Fri Dec 30 00:07:58 2016 daemon.info dnsmasq[8473]: using local addresses only for domain 101order.com
Fri Dec 30 00:07:58 2016 daemon.info dnsmasq[8473]: using local addresses only for domain 101com.com
Fri Dec 30 00:07:58 2016 daemon.info dnsmasq[8473]: using local addresses only for domain lan
Fri Dec 30 00:07:58 2016 daemon.info dnsmasq[8473]: using nameserver 192.168.1.1#53
Fri Dec 30 00:07:58 2016 daemon.info dnsmasq[8473]: using nameserver 192.168.1.254#53
Fri Dec 30 00:07:58 2016 daemon.info dnsmasq[8473]: read /etc/hosts - 1 addresses
Fri Dec 30 00:07:58 2016 daemon.info dnsmasq[8473]: read /tmp/hosts/dhcp - 1 addresses
Fri Dec 30 00:07:58 2016 daemon.info dnsmasq[8473]: read /tmp/hosts/odhcpd - 1 addresses
Fri Dec 30 00:07:58 2016 daemon.info dnsmasq-dhcp[8473]: read /etc/ethers - 0 addresses

(Last edited by imbaSD on 29 Dec 2016, 23:11)

Post #349

Comitizer

30 Dec 2016, 04:05

dibdot wrote:

Comitizer wrote:
So far this package is great. The only thing I would like is some more stats (similar to Pi Hole https://pi-hole.net/). It would be really cool to see what is being blocked and so forth.
Currently I focus on fast & neat dns adblocking - nothing more. Basic stats are only available via ubus call (see online doc).

Comitizer wrote:
Regardless, I'm loving the system so I wondering whether or not it is a good idea to enable ALL of the ad block lists. Is there any reason not to? Performance? What would the impact be?
No (known) problems with that.

I found the stats, thank you. That works ok.

As to the block lists, let's put it this way, what do you have enabled?

But seriously, thank you for the package, it's fantastic.

(Last edited by Comitizer on 30 Dec 2016, 04:06)

Post #350

dibdot

30 Dec 2016, 06:33

imbaSD wrote:

[....]
Fri Dec 30 00:07:58 2016 daemon.info dnsmasq[8473]: using local addresses only for domain 7bpeople.com

That's only a dnsmasq info, not an error. Please enable adblock debug in config, make a '/etc/init.d/adblock restart' and send the output of logread -e "adblock" - thanks.

Sorry, posts 351 to 350 are missing from our archive.

Page 14 of 17