ThreadGet

Browsers, email, chat, etc.
Message
Author
seaside
Posts: 934
Joined: Thu 12 Apr 2007, 00:19

ThreadGet

#1 Post by seaside »

- ThreadGet -

A program to download phpBB forum threads (like Puppys'). It's based somewhat on Mu's original program Fetchforum, written in basic.

Here's how it works -

After you've made selections, ThreadGet downloads each page of the forum thread, saves it as a single sequentially numbered html file in a subdirectory of a created /root/ForumThreads directory.

When all pages have been downloaded, the separate html files are combined into a single one. The individual pages are then deleted.

Usuage:

Find the forum thread you like and go to the first page. Then copy and paste the URL line (be sure that the line ends with only a number - no "&start" or other symbols)

Enter the last page number of the thread you've chosen or enter the last page you'd like to download. (In attempting to ensure that all pages are downloaded, you may get a couple of extra unfilled pages at the end, due to rounding and partial pages)

Enter the name of the file you'd like to save the thread under (e.g configure-wifi). No need to enter any extension, as .htm will be added to the name (e.g. configure-wifi.htm)

You'll now have a single searchable html file with all those informative posts. :D

UPDATE:3-30-2011 Added "Start Page" and append to existing files

Cheers,
s

Below is a Pic & Pet....
Attachments
ThreadGet.pet
ThreadGet
(1.51 KiB) Downloaded 1549 times
ThreadGet.png
(19.64 KiB) Downloaded 3401 times
Last edited by seaside on Wed 30 Mar 2011, 14:38, edited 1 time in total.

aragon
Posts: 1698
Joined: Mon 15 Oct 2007, 12:18
Location: Germany

#2 Post by aragon »

awesome :!: :!: :!:

my loved "The Text-User Interface" in one html-file...

will test a little more tomorrow.

thanks
aragon

User avatar
Sit Heel Speak
Posts: 2595
Joined: Fri 31 Mar 2006, 03:22
Location: downwind

#3 Post by Sit Heel Speak »

:D Yes indeed-y !!! Two dew-claws up !!!

User avatar
plankenstein
Posts: 120
Joined: Sun 16 Nov 2008, 00:49
Location: Arkansas, USA

#4 Post by plankenstein »

This sounds awsome! I'll definitely be trying this one out. :D
I carefully plan ALL my random acts! :lol:

User avatar
TheAsterisk!
Posts: 406
Joined: Tue 10 Feb 2009, 08:52

#5 Post by TheAsterisk! »

Thanks! This could be really helpful!

User avatar
Flash
Official Dog Handler
Posts: 13071
Joined: Wed 04 May 2005, 16:04
Location: Arizona USA

#6 Post by Flash »

Um, some threads in this forum are several hundred pages long. If a lot of people download them, it's going to really tie up the forum server and bandwidth. All I'm saying is, please be discriminating in your choices. :)

aarf

#7 Post by aarf »

successfully just got six pages from a non-phpbb but near clone.
have often been in need of this pet in the past so no doubt will in future. thanks muchly.

User avatar
chrome307
Posts: 708
Joined: Thu 15 Jan 2009, 11:00

#8 Post by chrome307 »

I have tried this out and got a strange result.

When I download a link, a new directory named Thread is created in:

usr/share/applications

however there is no content.

There is also another directory created 'Forum threads' which does have the content as mentioned in your thread.

Is there a reason why the additional directory is created?

I am using BrowserLinux, which is a cut down version of Puppy Linux 4.31.

The application works great and is a life saver for reading long threads. :)

scsijon
Posts: 1596
Joined: Thu 24 May 2007, 03:59
Location: the australian mallee
Contact:

#9 Post by scsijon »

Flash wrote:Um, some threads in this forum are several hundred pages long. If a lot of people download them, it's going to really tie up the forum server and bandwidth. All I'm saying is, please be discriminating in your choices. :)
Like ttuuxxx's Classic 2.14X you mean, 281 pages at last view, must be close to the record by now.

tlchost
Posts: 2057
Joined: Sun 05 Aug 2007, 23:26
Location: Baltimore, Maryland USA
Contact:

Thanks!

#10 Post by tlchost »

Great script.....no more saving each page, then adding them all together.

Perhaps allow the user to specify the destination for the file? I set up a symlink to have then stored in /mnt/home.

Thanks!

Thom

tlchost
Posts: 2057
Joined: Sun 05 Aug 2007, 23:26
Location: Baltimore, Maryland USA
Contact:

Feature Request

#11 Post by tlchost »

Would it be possible to add a feature that would allow us to specify a starting page and the option to save with append to or overwrite the local file?

Lets assume we wanted to save the Fluppy 013 thread...it has 117 pages. When new posts are added to the forum, having the ability to start at page 118 and add those new posts to the local file would not only be very convenient, but also reduce bandwidth used by the forum.

Thom

User avatar
Flash
Official Dog Handler
Posts: 13071
Joined: Wed 04 May 2005, 16:04
Location: Arizona USA

#12 Post by Flash »

The little box just to the left of the word "Posted" at the top left of a post is the URL to that post. If you copy that URL and enter it in the GUI box, does it start downloading from that post?

seaside
Posts: 934
Joined: Thu 12 Apr 2007, 00:19

#13 Post by seaside »

I think I could add a starting page for downloading and perhaps a way to append to existing downloaded threads.

If there is interest, l'll mull over how to implement this.

Cheers,
s

User avatar
russoodle
Posts: 707
Joined: Fri 12 Sep 2008, 17:36
Location: Down-Under in South Oz

#14 Post by russoodle »

Thank you seaside, what a brilliant idea!
[i][color=Green][size=92]The mud-elephant, wading thru the sea, leaves no tracks..[/size][/color][/i]

tlchost
Posts: 2057
Joined: Sun 05 Aug 2007, 23:26
Location: Baltimore, Maryland USA
Contact:

#15 Post by tlchost »

russoodle wrote:Thank you seaside, what a brilliant idea!
Glad I suggested it...and looking forward to seeing it.

Thom

Jasper

#16 Post by Jasper »

Hi seaside,

Fantastic; I just tried a two pager.

I would like to ask if it is possible to bookmark posts of particular interest and/or to delete (or partially delete - see PS) those those of little or no interest?

Also, I assume it works in other cases and that it is not restricted to this forum?

My regards and thanks

PS An observation:

Sometimes quotes are unnecessarily entire, rather than selective.

seaside
Posts: 934
Joined: Thu 12 Apr 2007, 00:19

New Version of ThreadGet

#17 Post by seaside »

Ok, here it is.

An update to ThreadGet.

Added is an entry box "Start Page" where you can start downloading forum pages beginning with the page number selected. If you don't place a page number here, it defaults to page 1.

Some other changes in the structure have been made as well-

The html files are now stored in /root/ForumThreads.

If the name selected is the same as an existing html file in /root/ForumThreads, the downloaded pages are appended.

A new pet file is available on the first post.

Cheers,
s

JASPER: You can use ThreadGet on many phpBB forums- one way to check is go to page 2 of the forum thread and see if the url ends with "&start=15" (if it does, ThreadGet should work -- as far as editing the html file, you might use an editor like Bluefish.

tlchost
Posts: 2057
Joined: Sun 05 Aug 2007, 23:26
Location: Baltimore, Maryland USA
Contact:

#18 Post by tlchost »

Thanks.....works like a charm.

Thom

User avatar
Aitch
Posts: 6518
Joined: Wed 04 Apr 2007, 15:57
Location: Chatham, Kent, UK

#19 Post by Aitch »

Thanks seaside

added to my Search thread

http://www.murga-linux.com/puppy/viewto ... 614#537614

Aitch :)

seaside
Posts: 934
Joined: Thu 12 Apr 2007, 00:19

#20 Post by seaside »

And thank you Aitch for the search thread. It's very useful and necessary 8) s.

Post Reply