Page 1 of 3

ThreadGet

Posted: Tue 23 Nov 2010, 20:46
by seaside
- ThreadGet -

A program to download phpBB forum threads (like Puppys'). It's based somewhat on Mu's original program Fetchforum, written in basic.

Here's how it works -

After you've made selections, ThreadGet downloads each page of the forum thread, saves it as a single sequentially numbered html file in a subdirectory of a created /root/ForumThreads directory.

When all pages have been downloaded, the separate html files are combined into a single one. The individual pages are then deleted.

Usuage:

Find the forum thread you like and go to the first page. Then copy and paste the URL line (be sure that the line ends with only a number - no "&start" or other symbols)

Enter the last page number of the thread you've chosen or enter the last page you'd like to download. (In attempting to ensure that all pages are downloaded, you may get a couple of extra unfilled pages at the end, due to rounding and partial pages)

Enter the name of the file you'd like to save the thread under (e.g configure-wifi). No need to enter any extension, as .htm will be added to the name (e.g. configure-wifi.htm)

You'll now have a single searchable html file with all those informative posts. :D

UPDATE:3-30-2011 Added "Start Page" and append to existing files

Cheers,
s

Below is a Pic & Pet....

Posted: Tue 23 Nov 2010, 21:39
by aragon
awesome :!: :!: :!:

my loved "The Text-User Interface" in one html-file...

will test a little more tomorrow.

thanks
aragon

Posted: Wed 24 Nov 2010, 00:03
by Sit Heel Speak
:D Yes indeed-y !!! Two dew-claws up !!!

Posted: Wed 24 Nov 2010, 00:17
by plankenstein
This sounds awsome! I'll definitely be trying this one out. :D

Posted: Wed 24 Nov 2010, 01:41
by TheAsterisk!
Thanks! This could be really helpful!

Posted: Wed 24 Nov 2010, 02:01
by Flash
Um, some threads in this forum are several hundred pages long. If a lot of people download them, it's going to really tie up the forum server and bandwidth. All I'm saying is, please be discriminating in your choices. :)

Posted: Thu 25 Nov 2010, 15:45
by aarf
successfully just got six pages from a non-phpbb but near clone.
have often been in need of this pet in the past so no doubt will in future. thanks muchly.

Posted: Thu 02 Dec 2010, 11:51
by chrome307
I have tried this out and got a strange result.

When I download a link, a new directory named Thread is created in:

usr/share/applications

however there is no content.

There is also another directory created 'Forum threads' which does have the content as mentioned in your thread.

Is there a reason why the additional directory is created?

I am using BrowserLinux, which is a cut down version of Puppy Linux 4.31.

The application works great and is a life saver for reading long threads. :)

Posted: Sat 19 Mar 2011, 08:50
by scsijon
Flash wrote:Um, some threads in this forum are several hundred pages long. If a lot of people download them, it's going to really tie up the forum server and bandwidth. All I'm saying is, please be discriminating in your choices. :)
Like ttuuxxx's Classic 2.14X you mean, 281 pages at last view, must be close to the record by now.

Thanks!

Posted: Sat 19 Mar 2011, 11:04
by tlchost
Great script.....no more saving each page, then adding them all together.

Perhaps allow the user to specify the destination for the file? I set up a symlink to have then stored in /mnt/home.

Thanks!

Thom

Feature Request

Posted: Fri 25 Mar 2011, 10:03
by tlchost
Would it be possible to add a feature that would allow us to specify a starting page and the option to save with append to or overwrite the local file?

Lets assume we wanted to save the Fluppy 013 thread...it has 117 pages. When new posts are added to the forum, having the ability to start at page 118 and add those new posts to the local file would not only be very convenient, but also reduce bandwidth used by the forum.

Thom

Posted: Fri 25 Mar 2011, 12:49
by Flash
The little box just to the left of the word "Posted" at the top left of a post is the URL to that post. If you copy that URL and enter it in the GUI box, does it start downloading from that post?

Posted: Mon 28 Mar 2011, 21:08
by seaside
I think I could add a starting page for downloading and perhaps a way to append to existing downloaded threads.

If there is interest, l'll mull over how to implement this.

Cheers,
s

Posted: Tue 29 Mar 2011, 12:39
by russoodle
Thank you seaside, what a brilliant idea!

Posted: Tue 29 Mar 2011, 18:39
by tlchost
russoodle wrote:Thank you seaside, what a brilliant idea!
Glad I suggested it...and looking forward to seeing it.

Thom

Posted: Tue 29 Mar 2011, 19:34
by Jasper
Hi seaside,

Fantastic; I just tried a two pager.

I would like to ask if it is possible to bookmark posts of particular interest and/or to delete (or partially delete - see PS) those those of little or no interest?

Also, I assume it works in other cases and that it is not restricted to this forum?

My regards and thanks

PS An observation:

Sometimes quotes are unnecessarily entire, rather than selective.

New Version of ThreadGet

Posted: Wed 30 Mar 2011, 14:57
by seaside
Ok, here it is.

An update to ThreadGet.

Added is an entry box "Start Page" where you can start downloading forum pages beginning with the page number selected. If you don't place a page number here, it defaults to page 1.

Some other changes in the structure have been made as well-

The html files are now stored in /root/ForumThreads.

If the name selected is the same as an existing html file in /root/ForumThreads, the downloaded pages are appended.

A new pet file is available on the first post.

Cheers,
s

JASPER: You can use ThreadGet on many phpBB forums- one way to check is go to page 2 of the forum thread and see if the url ends with "&start=15" (if it does, ThreadGet should work -- as far as editing the html file, you might use an editor like Bluefish.

Posted: Thu 31 Mar 2011, 14:17
by tlchost
Thanks.....works like a charm.

Thom

Posted: Fri 24 Jun 2011, 20:05
by Aitch
Thanks seaside

added to my Search thread

http://www.murga-linux.com/puppy/viewto ... 614#537614

Aitch :)

Posted: Fri 24 Jun 2011, 23:03
by seaside
And thank you Aitch for the search thread. It's very useful and necessary 8) s.