Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Wed 20 Sep 2017, 10:46
All times are UTC - 4

 Forum index » Advanced Topics » Additional Software (PETs, n' stuff) » Browsers and Internet
Advert-Blocker 0.5
Post new topic   Reply to topic View previous topic :: View next topic
Page 1 of 4 [59 Posts]   Goto page: 1, 2, 3, 4 Next
Author Message
sc0ttman


Joined: 16 Sep 2009
Posts: 2489
Location: UK
PostPosted: Sat 28 Aug 2010, 12:41    Post subject:  Advert-Blocker 0.5
Subject description: easy GUI to remove online ads		  
Pup-Advert-Blocker
block many online ads in any browser


This tool will download a list of known advertising servers and add them to your "/etc/hosts" file - blocking them from you.

Then you can browse the web without seeing or downloading a large number of popular and very annoying online ads Very Happy

This tool is very simple, but does the trick nicely, I think.

Using all services currently blocks around 30,300 ad servers (Aug 2010)

Suggestions and improvements still welcome.

Changelog:

version 0.5
- added a fix I accidentally un-added! Rolling Eyes
- changed detection of empty list
- cleaned the code a tiny little bit
- added a timeout to wget (avoids endless pausing)

version 0.4
- added (only some of) Barry Ks bugfixes (whoops! fixed in v0.5)

version 0.3
- fixed duplicate removal (using dos2unix to convert carriage returns)
- updated help GUI

version 0.2
- added more servers
- servers can be combined or chosen separately
- ads can be disabled by not selecting any servers
- (duplicates are still found in the final built list!) Sad

(Thanks to this thread) (thx 2 lguleder etc)
pup-ad-blocker.png
 Description   
 Filesize   40.7 KB
 Viewed   6480 Time(s)

pup-ad-blocker.png
pup-advert-blocker.pet
Description  version 0.5
 pet

 Download 
Filename  pup-advert-blocker.pet 
Filesize  3.13 KB 
Downloaded  3062 Time(s) 
pup-advert-blocker.pet
Description  version 0.4
 pet

 Download 
Filename  pup-advert-blocker.pet 
Filesize  3.15 KB 
Downloaded  1372 Time(s) 
pup-advert-blocker.pet
Description  version 0.3
 pet

 Download 
Filename  pup-advert-blocker.pet 
Filesize  2.99 KB 
Downloaded  1359 Time(s) 
pup-advert-blocker.pet
Description  version 0.2
 pet

 Download 
Filename  pup-advert-blocker.pet 
Filesize  3.03 KB 
Downloaded  1277 Time(s) 
pup-advert-blocker.pet
Description  version 0.1
 pet

 Download 
Filename  pup-advert-blocker.pet 
Filesize  2.74 KB 
Downloaded  1297 Time(s) 

_________________
Akita Linux, VLC-GTK, Pup Search, Pup File Search

Last edited by sc0ttman on Thu 02 Sep 2010, 01:34; edited 16 times in total
Back to top
View user's profile Send private message 
Iguleder


Joined: 11 Aug 2009
Posts: 2031
Location: Israel, somewhere in the beautiful desert
PostPosted: Sat 28 Aug 2010, 12:44    Post subject:  
Once of those moments when I wonder why the forums don't have a "like" button Laughing

Goo'job!
_________________
My homepage
My GitHub profile
Back to top
View user's profile Send private message Visit poster's website 
chrismt


Joined: 21 Apr 2010
Posts: 255
PostPosted: Sat 28 Aug 2010, 13:10    Post subject:  
@ Scottman

Thanks!

This is what I was asking someone to make

You can learn a lot of things from Hostsman, a windows Hosts Manager that blocks ads and malware

Here

http://www.abelhadigital.com/hostsman



This PET. is my favorite!

You can also add more adblock hosts providers like MVPS etc

You not only block ads but also malware sites in Windows with Hostsman because it has got many hosts providers as default.

You can even update all the hosts providers automatically and delete duplicates


Check it out!

Regards
Back to top
View user's profile Send private message 
technosaurus


Joined: 18 May 2008
Posts: 4735
PostPosted: Sat 28 Aug 2010, 17:08    Post subject:  
I made a simple command line script that will integrate both lists and sort the duplicates out (the commented out line also removes some know adult content)

Code:
#!/bin/sh
wget -c -O hosts1 http://sysctl.org/cameleon/hosts
wget -c -O hosts2 http://pgl.yoyo.org/as/serverlist.php?hostformat=hosts&showintro=0&mimetype=plaintext
#wget -c -O hosts3 http://www.mvps.org/winhelp2002/hosts.txt #removes adult content
cat /etc/hosts hosts1 hosts2 hosts3 |grep ^[1-9] |sed "s/\t//g"|sort |uniq >/etc/hosts
#rm -f hosts1 hosts2


other possible host lists:
http://my.opera.com/Tamil/blog/ad-block
http://code.google.com/p/fanboy-adblock-list/source/browse/
http://www.technobeta.com/download/urlfilter.ini
http://my.opera.com/Ghostshaw/blog/
http://www.technobeta.com/download/urlfilter.ini

here is the link for adblock plus, but it will require some manipulation to make it work
https://easylist-downloads.adblockplus.org/easylist.txt
https://easylist-downloads.adblockplus.org/easyprivacy.txt
more here: http://adblockplus.org/en/subscriptions and here:
http://easylist.adblockplus.org/
_________________
Web Programming - Pet Packaging 100 & 101
Back to top
View user's profile Send private message 
sc0ttman


Joined: 16 Sep 2009
Posts: 2489
Location: UK
PostPosted: Sun 29 Aug 2010, 13:32    Post subject:  
version 0.2 released - see main post.
_________________
Akita Linux, VLC-GTK, Pup Search, Pup File Search
Back to top
View user's profile Send private message 
technosaurus


Joined: 18 May 2008
Posts: 4735
PostPosted: Sun 29 Aug 2010, 14:17    Post subject:  
Nice work!
Bugfix#1
Code:
cat /tmp/adlist1 /tmp/adlist2 /tmp/adlist3 /tmp/adlist4 |grep ^[1-9] |sed "s/\t//g"|sort |uniq -u
leave off the -u from uniq otherwise you will miss anything that is on more than one list

Bugreport
in order for lines to be seen as true duplicates, the formatting has to be exactly the same (uniq compares the entire line as a string), so you may need to pass it through sed a couple of times to replace combinations of tabs/spaces/etc... with a single space in all entries and no trailing tabs/spaces possibly leading tabs/spaces as well

sed 's/[ \t]*/ /' #replace combinations of tabs/spaces with a space
sed 's/[ \t]*$//' #remove trailing spaces/tabs
sed 's/[^ \t]*//' #remove leading spaces/tabs
_________________
Web Programming - Pet Packaging 100 & 101
Back to top
View user's profile Send private message 
sc0ttman


Joined: 16 Sep 2009
Posts: 2489
Location: UK
PostPosted: Sun 29 Aug 2010, 19:02    Post subject:  
updated to 0.3

all duplicates now removed from hosts file automatically Very Happy

I personally think this (or similar) should be included in Puppy, not because I made it of course, but because it makes browsing soooooo much faster and less annoying!

thanks for the help guys
_________________
Akita Linux, VLC-GTK, Pup Search, Pup File Search
Back to top
View user's profile Send private message 
BarryK
Puppy Master


Joined: 09 May 2005
Posts: 8223
Location: Perth, Western Australia
PostPosted: Sun 29 Aug 2010, 20:59    Post subject:  
Hi, great app, I posted a bug fix for 0.3:

http://bkhome.org/blog/?viewDetailed=01777
_________________
http://barryk.org/news/
Back to top
View user's profile Send private message Visit poster's website 
jemimah


Joined: 26 Aug 2009
Posts: 4309
Location: Tampa, FL
PostPosted: Sun 29 Aug 2010, 23:28    Post subject:  
Nice. I've added it to Puppeee and Fluppy.
Back to top
View user's profile Send private message Visit poster's website 
sc0ttman


Joined: 16 Sep 2009
Posts: 2489
Location: UK
PostPosted: Mon 30 Aug 2010, 01:42    Post subject:  
version 0.4 is out..

Added Barry Ks bugfixes.. but removed hard-coding of app title.

Thanks for the help! Smile
_________________
Akita Linux, VLC-GTK, Pup Search, Pup File Search
Back to top
View user's profile Send private message 
technosaurus


Joined: 18 May 2008
Posts: 4735
PostPosted: Mon 30 Aug 2010, 04:14    Post subject:  
I adapted it for standard dialog for the minimalist puppies using PupnGo. This will work with dialog on the command line or Xdialog (gtk1 or gtk2) in X, but not as pretty either way (I removed any non-essentials to work with dialog)
Code:
#!/bin/sh
[ -z $DISPLAY ] && DIALOG=dialog || DIALOG=Xdialog
for x in `$DIALOG --stdout --checklist "Choose your ad blocking service(s)" 0 0 4 1 "mvps.org" ON 2 "systcl.org" ON 3 "technobeta.com" ON 4 "yoyo.org" ON |tr "/" " " |tr '\"' ' '`; do
   case $x in
   1)wget -c -4 -t 0 -O /tmp/adlist1 'http://www.mvps.org/winhelp2002/hosts.txt';;
   2)wget -c -4 -t 0 -O /tmp/adlist2 'http://sysctl.org/cameleon/hosts';;
   3)wget -c -4 -t 0 -O /tmp/adlist3 'http://www.technobeta.com/download/urlfilter.ini';;
   4)wget -c -4 -t 0 -O /tmp/adlist4 'http://pgl.yoyo.org/as/serverlist.php?hostformat=hosts&showintro=0&mimetype=plaintext';;
   *)echo $x;;
   esac
done
touch /tmp/adlist{1,2,3,4}
cat /etc/hosts /tmp/adlist{1,2,3,4} |grep ^[1-9] |sed '/^$/d' |sed "s/\t/ /g"|sed '/^#/d' |sed 's/  / /g' |dos2unix -u |sort |uniq > /etc/hosts
rm -f /tmp/adlist{1,2,3,4}


Edit: this one should work better
Code:
#!/bin/sh
[ -z $DISPLAY ] && DIALOG=dialog || DIALOG=Xdialog
[ ! -f /etc/hosts.usr ] && touch /etc/hosts && cp -f /etc/hosts /etc/hosts.usr #echo 127.0.0.1 localhost puppypc >/etc/hosts #r1 maybe user added others
for x in `$DIALOG --stdout --checklist "Choose your ad blocking service(s)" 0 0 5 1 "mvps.org" ON 2 "systcl.org" ON 3 "technobeta.com" ON 4 "yoyo.org" ON 5 "turn off adblocking" off |tr "/" " " |tr '\"' ' '`; do
   case $x in
   1)wget -c -4 -t 0 -O /tmp/adlist1 'http://www.mvps.org/winhelp2002/hosts.txt';;
   2)wget -c -4 -t 0 -O /tmp/adlist2 'http://sysctl.org/cameleon/hosts';;
   3)wget -c -4 -t 0 -O /tmp/adlist3 'http://www.technobeta.com/download/urlfilter.ini';;
   4)wget -c -4 -t 0 -O /tmp/adlist4 'http://pgl.yoyo.org/as/serverlist.php?hostformat=hosts&showintro=0&mimetype=plaintext';;
   5)ln -sf /etc/hosts.usr /etc/hosts && exit;;
   *)echo $x;;
   esac
done
touch /tmp/adlist{1,2,3,4}
cat /etc/hosts.usr /tmp/adlist{1,2,3,4} |sed 's/^[ \t]*//' |sed 's/\t/ /g' |sed 's/  / /g' |grep ^[1-9] |dos2unix -u |sort |uniq > /etc/hosts.adblock
ln -sf /etc/hosts.adblock /etc/hosts

_________________
Web Programming - Pet Packaging 100 & 101

Last edited by technosaurus on Mon 30 Aug 2010, 22:03; edited 4 times in total
Back to top
View user's profile Send private message 
abushcrafter


Joined: 30 Oct 2009
Posts: 1438
Location: England
PostPosted: Mon 30 Aug 2010, 08:16    Post subject:  
Could you add the option to disable ad blocking please. So when one is on sites like lastfm there services like there player works.
_________________
adobe flash is rubbish!
My Quote:"Humans are stupid, though some are clever but stupid." http://www.dependent.de/media/audio/mp3/System_Syn_Heres_to_You.zip http://www.systemsyn.com/
Back to top
View user's profile Send private message Visit poster's website 
sc0ttman


Joined: 16 Sep 2009
Posts: 2489
Location: UK
PostPosted: Mon 30 Aug 2010, 08:38    Post subject:  
abushcrafter wrote:
Could you add the option to disable ad blocking please. So when one is on sites like lastfm there services like there player works.

My GUI already has that feature. Just don't select any lists and click Start'.
You just need to restart your browser, I think..

The code above, by techonosaurus, probably just needs the '/etc/hosts' part removed from the 'cat' command, in order to enable the removal of all ad lists (I think - correct me if I am wrong!)
_________________
Akita Linux, VLC-GTK, Pup Search, Pup File Search
Back to top
View user's profile Send private message 
abushcrafter


Joined: 30 Oct 2009
Posts: 1438
Location: England
PostPosted: Mon 30 Aug 2010, 08:51    Post subject:  
sc0ttman wrote:
abushcrafter wrote:
Could you add the option to disable ad blocking please. So when one is on sites like lastfm there services like there player works.

My GUI already has that feature. Just don't select any lists and click Start'.
You just need to restart your browser, I think..

The code above, by techonosaurus, probably just needs the '/etc/hosts' part removed from the 'cat' command, in order to enable the removal of all ad lists (I think - correct me if I am wrong!)
Great!
_________________
adobe flash is rubbish!
My Quote:"Humans are stupid, though some are clever but stupid." http://www.dependent.de/media/audio/mp3/System_Syn_Heres_to_You.zip http://www.systemsyn.com/
Back to top
View user's profile Send private message Visit poster's website 
sc0ttman


Joined: 16 Sep 2009
Posts: 2489
Location: UK
PostPosted: Mon 30 Aug 2010, 09:22    Post subject:  
Not so great.. I'm dumb.. Removing the /etc/hosts from technosaurus' code will just break it and do weird things..
But you can do it in mine, that was correct at least..
_________________
Akita Linux, VLC-GTK, Pup Search, Pup File Search
Back to top
View user's profile Send private message 
Display posts from previous:   Sort by:   
Page 1 of 4 [59 Posts]   Goto page: 1, 2, 3, 4 Next
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » Advanced Topics » Additional Software (PETs, n' stuff) » Browsers and Internet
Jump to:  


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group

[ Time: 0.0783s ][ Queries: 13 (0.0056s) ][ GZIP on ]