How to display all forum pages on only one page

How to do things, solutions, recipes, tutorials
Post Reply
Message
Author
User avatar
MochiMoppel
Posts: 2084
Joined: Wed 26 Jan 2011, 09:06
Location: Japan

How to display all forum pages on only one page

#1 Post by MochiMoppel »

Sometimes I wish I could display multipage threads on only one long page. Why? Because it makes searching for keywords so much easier.
- The forum search is slow. For each new or refined search I have to go to the Search page and then I have to wait for the search results.
- The forum search is also incomplete. Search for "Puppy" in the Beginners forum and the oldest posts date from March 2007. Anything older is missing.

Compared to the forum search searching an already displayed webpage is blazingly fast. However switching from page to page is no fun. Combining all pages into only one is the answer and can be achieved by using the URL parameter ppage.

Such all-inclusive pages are also ideal for saving them locally. Future searches can then be made offline.

All topics on one page
Here is an example: This HOWTO forum is accessed from the main index page with the URL
     http://www.murga-linux.com/puppy/index.php?f=4
At the time of this writing the top left part reads "Page 1 of 57 [1692 Topics]" and on the right it reads "Goto page: 1, 2, 3, ..., 55, 56, 57 Next".
By default a forum page displays 30 topics (aka threads) per page, so 1692/30=56.4=57 pages needed. The math matches.

Now let's tell the server that we don't want 30 topics per page, we want 2000 (actually 1672 would suffice, but the server will stop serving after 1672 anyway, so let's be generous here and add a buffer, just in case new topics are posted during our experiments). In the address field of our browser we add &ppage=2000:
     http://www.murga-linux.com/puppy/index. ... ppage=2000
When refreshing the page, we will end up with 1 page and the top left part will read "Page 1 of 1 [1692 Topics]".
The 10th post from back is titled "GSview & antialias". Searching for keyword "GSview" in the forum search would never find this topic, searching with the browser now finds it in a fraction of a second.

All posts on one page
Searching topics is useful, but given the sometimes idiotic topic titles ("I have a question"), searching the message bodies can be more helpful.

Another example. In the Bugs forum find BK's topic "Quirky Xerus 8.3 for x86_64, released July 29, 2017":
     http://www.murga-linux.com/puppy/viewtopic.php?t=106460
"Page 1 of 47 [700 Posts]" it reads, and as a page holds only 15 posts per page (in contrast to 30 topics per page), so 700/15=46.6=47pages. Especially for bug reports it is a common courtesy to carefully search for previous bug reports before submitting a new report, but searching 47 pages one by one....

Let's try ppage=1000:
     http://www.murga-linux.com/puppy/viewto ... ppage=1000
Works! :lol:

Limitations:
How many posts will fit on a page? This depends not on the browser, which should be able to display thousands of posts. The bottleneck is the server. It seems that the server will not serve a page with huge content. I have no problems fetching 3000 posts per page (and compared to topics single posts can be quite large), but 4000 posts seem impossible. I make it a habit to hammer in &ppage=3333. Easy to type and in the worst case I will end up with more than one page, but not dozens. The only disadvantage of this method is that in case of posts the server might not display attachments. This may happen when requesting a high ppage number, but not necessarily is lower better, e.g. in same cases ppage=600 will display posts complete with attachments while ppage=500 will not. I haven't found a pattern. You will have to experiment when attachments are missing.

User avatar
tallboy
Posts: 1760
Joined: Tue 21 Sep 2010, 21:56
Location: Drøbak, Norway

#2 Post by tallboy »

MochiMoppel, you have just shown me my new favorite tool &ppage=
It takes a little time to produce the page, but the result is very impressive. Thank you very much!

tallboy
True freedom is a live Puppy on a multisession CD/DVD.

User avatar
smokey01
Posts: 2813
Joined: Sat 30 Dec 2006, 23:15
Location: South Australia :-(
Contact:

#3 Post by smokey01 »

@mm thanks for this, it's fantastic. It's going straight to the pool room.

For non Australians it's a term from a classic Australian movie called the castle. In this case the pool room is code for newsletter.

User avatar
fredx181
Posts: 4448
Joined: Wed 11 Dec 2013, 12:37
Location: holland

#4 Post by fredx181 »

That's really great!
So much easier to find something in one thread (not having to search through all pages)

Thanks!

Fred

B.K. Johnson
Posts: 807
Joined: Mon 12 Oct 2009, 17:11

#5 Post by B.K. Johnson »

@MochiMoppel
Nice. Thanks!
[color=blue]B.K. Johnson
tahrpup-6.0.5 PAE (upgraded from 6.0 =>6.0.2=>6.0.3=>6.0.5 via quickpet/PPM=Not installed); slacko-5.7 occasionally. Frugal install, pupsave file, multi OS flashdrive, FAT32 , SYSLINUX boot, CPU-Dual E2140, 4GB RAM[/color]

User avatar
Flash
Official Dog Handler
Posts: 13071
Joined: Wed 04 May 2005, 16:04
Location: Arizona USA

#6 Post by Flash »

Anyone have an idea about how much of an extra load, if any, this puts on the forum's server?

User avatar
bigpup
Posts: 13886
Joined: Sun 11 Oct 2009, 18:15
Location: S.C. USA

#7 Post by bigpup »

I would say it has a big effect on server load.
It asks the server to transmit all pages of a topic, at one time, instead of only one page at a time when requested.
Just the big difference in download speed indicates the load is way up.
The things they do not tell you, are usually the clue to solving the problem.
When I was a kid I wanted to be older.... This is not what I expected :shock:
YaPI(any iso installer)

User avatar
Flash
Official Dog Handler
Posts: 13071
Joined: Wed 04 May 2005, 16:04
Location: Arizona USA

#8 Post by Flash »

Then I would ask people to please use this sparingly and judiciously.

User avatar
MochiMoppel
Posts: 2084
Joined: Wed 26 Jan 2011, 09:06
Location: Japan

#9 Post by MochiMoppel »

Flash wrote:Anyone have an idea about how much of an extra load, if any, this puts on the forum's server?
Good question.
The server load is always lower than fetching pages one by one.

Let's take my first example, the 57 HOWTO pages:
If I would have to search the pages one by one (F3..push Next..F3..push Next..F3..push Next...until my finger falls off) my browser would have to make 56 additional connections to the server and the server has to respond to 56 additional individual request. How much stress this puts on the server I don't know, probably peanuts, but it's definitely less efficient than serving only 1 request.

Now let's look at the volume:
1693 topics on 57 pages, each 77K = 4389K, incl. 500K junk
1693 topics on 1 page= 2448K, incl.8.8K junk

Each individual page contains of a header portion, the body (that's what you see on screen) and a footer. Multiply header/footer by 56 and you have the excess size the server has to deliver.

What I really see as an issue here is the ridiculously large size of "junk" in the header, code that does nothing but eats bandwidth, download time and server costs. This affects every page the forum server delivers and you might be able to see the server statistics and calculate how much unnecessary traffic it generates

One month ago I tried to draw your attention to this problem in a PM titled "Making Forum pages faster and leaner" - I never received an answer. In case you have missed it I add the "extra fat" each page has to carry, mainly the <style> section, which is not needed at all if you uncomment the link to the external stylesheet (as the instructions recommend and as all other phpBB forums sites I checked do).

Removing this <style> section and the comments in the footer would have a real impact on server performance:

Code: Select all

<style type="text/css">
<!--
/*
The original subSilver Theme for phpBB version 2+
Created by subBlue design
http://www.subBlue.com
NOTE: These CSS definitions are stored within the main page body so that you can use the phpBB2
theme administration centre. When you have finalised your style you could cut the final CSS code
and place it in an external file, deleting this section to save bandwidth.
*/
/* General page style. The scroll bar colours only visible in IE5.5+ */
body { 
background-color: #E5E5E5;
scrollbar-face-color: #DEE3E7;
scrollbar-highlight-color: #FFFFFF;
scrollbar-shadow-color: #DEE3E7;
scrollbar-3dlight-color: #D1D7DC;
scrollbar-arrow-color:  #006699;
scrollbar-track-color: #EFEFEF;
scrollbar-darkshadow-color: #98AAB1;
}
/* General font families for common tags */
font,th,td,p { font-family: Verdana, Arial, Helvetica, sans-serif }
a:link,a:active,a:visited { color : #006699; }
a:hover		{ text-decoration: underline; color : #DD6900; }
hr	{ height: 0px; border: solid #D1D7DC 0px; border-top-width: 1px;}
/*--- mod : categories hierarchy */
form {
margin: 0px;
padding: 0px;
}
td.attachrow {
color: #000000;
font-size: 11px;
font-weight: normal;
border: 1px #006699 solid;
}
td.attachheader {
color: #314687;
font-size: 11px;
font-weight: bold;
background: #D7D7E7 url(http://www.murga-linux.com/puppy/images/cellpic1.gif) repeat-x;
height: 21px;
white-space: nowrap;
}
table.attachtable {
color: #000000;
font-size: 12px;
font-weight: normal;
border: 1px #006699 solid;
border-collapse: collapse;
}
.absbottom {
vertical-align: text-bottom;
}
.dom_overview_abshidden { position: absolute; visibility: hidden; width: 300px; }
/*--- fin mod : categories hierarchy */
/* This is the border line & background colour round the entire page */
.bodyline	{ background-color: #FFFFFF; border: 1px #98AAB1 solid; }
/* This is the outline round the main forum tables */
.forumline	{ background-color: #FFFFFF; border: 2px #006699 solid; }
/* Main table cell colours and backgrounds */
td.row1	{ background-color: #EFEFEF; }
td.row2	{ background-color: #DEE3E7; }
td.row3	{ background-color: #D1D7DC; }
/*
This is for the table cell above the Topics, Post & Last posts on the index.php page
By default this is the fading out gradiated silver background.
However, you could replace this with a bitmap specific for each forum
*/
td.rowpic {
background-color: #FFFFFF;
background-image: url(1Puppy\ Linux\ Discussion\ Forum\ ::\ View\ topic\ -\ SEARCHING;\ Ways\ to\ search\ the\ forum\ for\ problems_solutions_files/cellpic2.jpg);
background-repeat: repeat-y;
}
/* Header cells - the blue and silver gradient backgrounds */
th	{
color: #FFA34F; font-size: 11px; font-weight : bold; 
background-color: #006699; height: 25px;
background-image: url(1Puppy\ Linux\ Discussion\ Forum\ ::\ View\ topic\ -\ SEARCHING;\ Ways\ to\ search\ the\ forum\ for\ problems_solutions_files/cellpic3.gif);
}
td.cat,td.catHead,td.catSides,td.catLeft,td.catRight,td.catBottom {
background-image: url(1Puppy\ Linux\ Discussion\ Forum\ ::\ View\ topic\ -\ SEARCHING;\ Ways\ to\ search\ the\ forum\ for\ problems_solutions_files/cellpic1.gif);
background-color:#D1D7DC; border: #FFFFFF; border-style: solid; height: 28px;
}
/*
Setting additional nice inner borders for the main table cells.
The names indicate which sides the border will be on.
Don't worry if you don't understand this, just ignore it :-)
*/
td.cat,td.catHead,td.catBottom {
height: 29px;
border-width: 0px 0px 0px 0px;
}
th.thHead,th.thSides,th.thTop,th.thLeft,th.thRight,th.thBottom,th.thCornerL,th.thCornerR {
font-weight: bold; border: #FFFFFF; border-style: solid; height: 28px;
}
td.row3Right,td.spaceRow {
background-color: #D1D7DC; border: #FFFFFF; border-style: solid;
}
th.thHead,td.catHead { font-size: 12px; border-width: 1px 1px 0px 1px; }
th.thSides,td.catSides,td.spaceRow	 { border-width: 0px 1px 0px 1px; }
th.thRight,td.catRight,td.row3Right	 { border-width: 0px 1px 0px 0px; }
th.thLeft,td.catLeft	  { border-width: 0px 0px 0px 1px; }
th.thBottom,td.catBottom  { border-width: 0px 1px 1px 1px; }
th.thTop	 { border-width: 1px 0px 0px 0px; }
th.thCornerL { border-width: 1px 0px 0px 1px; }
th.thCornerR { border-width: 1px 1px 0px 0px; }
/* The largest text used in the index page title and toptic title etc. */
.maintitle	{
font-weight: bold; font-size: 22px; font-family: "Trebuchet MS",Verdana, Arial, Helvetica, sans-serif;
text-decoration: none; line-height : 120%; color : #000000;
}
/* General text */
.gen { font-size : 12px; }
.genmed { font-size : 11px; }
.gensmall { font-size : 10px; }
.gen,.genmed,.gensmall { color : #000000; }
a.gen,a.genmed,a.gensmall { color: #006699; text-decoration: none; }
a.gen:hover,a.genmed:hover,a.gensmall:hover	{ color: #DD6900; text-decoration: underline; }
/* The register, login, search etc links at the top of the page */
.mainmenu		{ font-size : 11px; color : #000000 }
a.mainmenu		{ text-decoration: none; color : #006699;  }
a.mainmenu:hover{ text-decoration: underline; color : #DD6900; }
/* Forum category titles */
.cattitle		{ font-weight: bold; font-size: 12px ; letter-spacing: 1px; color : #006699}
a.cattitle		{ text-decoration: none; color : #006699; }
a.cattitle:hover{ text-decoration: underline; }
/* Forum title: Text and link to the forums used in: index.php */
.forumlink		{ font-weight: bold; font-size: 12px; color : #006699; }
a.forumlink 	{ text-decoration: none; color : #006699; }
a.forumlink:hover{ text-decoration: underline; color : #DD6900; }
/* Used for the navigation text, (Page 1,2,3 etc) and the navigation bar when in a forum */
.nav			{ font-weight: bold; font-size: 11px; color : #000000;}
a.nav			{ text-decoration: none; color : #006699; }
a.nav:hover		{ text-decoration: underline; }
/* titles for the topics: could specify viewed link colour too */
.topictitle,h1,h2	{ font-weight: bold; font-size: 11px; color : #000000; }
a.topictitle:link   { text-decoration: none; color : #006699; }
a.topictitle:visited { text-decoration: none; color : #5493B4; }
a.topictitle:hover	{ text-decoration: underline; color : #DD6900; }
/* Name of poster in viewmsg.php and viewtopic.php and other places */
.name			{ font-size : 11px; color : #000000;}
/* Location, number of posts, post date etc */
.postdetails		{ font-size : 10px; color : #000000; }
/* The content of the posts (body of text) */
.postbody { font-size : 12px; line-height: 18px}
a.postlink:link	{ text-decoration: none; color : #006699 }
a.postlink:visited { text-decoration: none; color : #5493B4; }
a.postlink:hover { text-decoration: underline; color : #DD6900}
/* Quote & Code blocks */
.code { 
font-family: Courier, 'Courier New', sans-serif; font-size: 11px; color: #006600;
background-color: #FAFAFA; border: #D1D7DC; border-style: solid;
border-left-width: 1px; border-top-width: 1px; border-right-width: 1px; border-bottom-width: 1px
}
.quote {
font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 11px; color: #444444; line-height: 125%;
background-color: #FAFAFA; border: #D1D7DC; border-style: solid;
border-left-width: 1px; border-top-width: 1px; border-right-width: 1px; border-bottom-width: 1px
}
/* Copyright and bottom info */
.copyright		{ font-size: 10px; font-family: Verdana, Arial, Helvetica, sans-serif; color: #444444; letter-spacing: -1px;}
a.copyright		{ color: #444444; text-decoration: none;}
a.copyright:hover { color: #000000; text-decoration: underline;}
/* Form elements */
input,textarea, select {
color : #000000;
font: normal 11px Verdana, Arial, Helvetica, sans-serif;
border-color : #000000;
}
/* The text input fields background colour */
input.post, textarea.post, select {
background-color : #FFFFFF;
}
input { text-indent : 2px; }
/* The buttons used for bbCode styling in message post */
input.button {
background-color : #EFEFEF;
color : #000000;
font-size: 11px; font-family: Verdana, Arial, Helvetica, sans-serif;
}
/* The main submit button option */
input.mainoption {
background-color : #FAFAFA;
font-weight : bold;
}
/* None-bold submit button */
input.liteoption {
background-color : #FAFAFA;
font-weight : normal;
}
/* This is the line in the posting page which shows the rollover
help line. This is actually a text box, but if set to be the same
colour as the background no one will know ;)
*/
.helpline { background-color: #DEE3E7; border-style: none; }
/* Import the fancy styles for IE only (NS4.x doesn't use the @import function) */
@import url("http://www.murga-linux.com/puppy/templates/subSilver/formIE.css"); 
-->
</style>

<!--
We request you retain the full copyright notice below including the link to www.phpbb.com.
This not only gives respect to the large amount of time given freely by the developers
but also helps build interest, traffic and use of phpBB 2.0. If you cannot (for good
reason) retain the full copyright we request you at least leave in place the
Powered by phpBB line, with phpBB linked to www.phpbb.com. If you refuse
to include even this then support on our forums may be affected.
The phpBB Group : 2002
// -->

<!--
AD PLACEHOLDER
<div id="wb_Image1" style="position:absolute; width:auto; height:auto; top:18; right:18; z-index:1">
</div>
-->

User avatar
Flash
Official Dog Handler
Posts: 13071
Joined: Wed 04 May 2005, 16:04
Location: Arizona USA

#10 Post by Flash »

I forwarded your PM to John Murga. He's the only one who can do anything with it.

belham2
Posts: 1715
Joined: Mon 15 Aug 2016, 22:47

#11 Post by belham2 »

Hi Flash/Mochi/all,

I've got a question: why, since this thread first appeared, has the murga site been slowing down like crazy at random times throughout the day? I'm no expert, but sometimes I go to login, or post a msg, or just use the site, and it becomes slowa## hell in responding. All other websites (and I test them to make sure it isn't my internet provider) run & load incredibly quick. Yet, there Murga is all of a sudden, grinding down, not responding, only to suddenly a minute or two or three later, like a damn bursts, and it magically responds properly again. Have never, ever seen this before with Murga. There's been a few times it was down over the years, but never this irritable, noticeable slowdown going on.

Unless there is no correlation (or causation), I say the server needs to be hardened against this crap (sorry, Mochi). Today I just talked to two of our server guys, had them look at what you're doing & what you wrote, and they just shook their heads and said collectively a big "definite yes" that the Murga server can be overloaded with this type of behavior from queuing site customers. In essence, it can cause queuing, file-serving and response time problems for those in the queue coming into the site.

AS great as Mochi's insight is, if this behavior from arriving server customers turns out to be the cause of the Murga site slowness I am now seeing, my opinion is stop it completely. Just my two cents.

Again, to recap, at various random times since Mochi's post first appeared about this, the murga site at times has ground down to a halt with response times---and this I have seen on home computers and on 'at work' computers (and those computers there are on a massive fatpipe to the world compared to my 2GB down home setup).

User avatar
Flash
Official Dog Handler
Posts: 13071
Joined: Wed 04 May 2005, 16:04
Location: Arizona USA

#12 Post by Flash »

Mochi may be right that the total server load is greater when pages are loaded one at a time, but that's only if the entire thread is read. Even when someone does read the whole thread, there's a considerable pause from one page request to the next, so the load on the server is spread out over a much greater time.

I understand that someone may occasionally want to download an entire thread, but please consider each time whether it's really necessary and don't just do it out of habit or (worst reason of all) for the novelty of it and because you can. :)

Having said that, I haven't noticed any slowdown so far. I think I'm pretty close to the server, so perhaps any slowdown you experience is due to some other part of the Internet between the server and you.

User avatar
tallboy
Posts: 1760
Joined: Tue 21 Sep 2010, 21:56
Location: Drøbak, Norway

#13 Post by tallboy »

Flash, belham2, I would ask you to read the first two lines of this thread again. I think you compare apples with oranges, the issue should be how much load there is on the server during a search. If you try to do two searches in sequence, you get a dialog window telling you to wait, because you cannot do a second search immediatly after the first. Is there a reason for that? Heavy server load, maybe?

BTW, I have no knowledge of the server location, but I sit on the other side of 'the pond', and I don't see any slowdown like what belham2 describe.

tallboy
True freedom is a live Puppy on a multisession CD/DVD.

User avatar
Flash
Official Dog Handler
Posts: 13071
Joined: Wed 04 May 2005, 16:04
Location: Arizona USA

#14 Post by Flash »

I have no idea what search algorithm the server uses, but I'd bet it's pretty efficient. Consider how fast Google finds stuff you're searching for. Google has indexed words and topics so that it doesn't have to search the entire Internet every time. So I doubt that the forum's search puts much strain on the server. Besides, what we're really talking about is not the server itself but the Internet bandwidth John Murga is paying for.

belham2
Posts: 1715
Joined: Mon 15 Aug 2016, 22:47

#15 Post by belham2 »

Flash,

I'm sure you saw the thread started Tuesday, about people reporting problems with the site. http://murga-linux.com/puppy/viewtopic.php?t=111370. This was coming from murga people around the world, North America, Europe, Australia/Asia all piped in given the members who posted .

So if we had a worldwide net event for two days (Tues and Weds), then that explains it. But that theory is sort off the table as no one else reported (anywhere on the net I saw) problems with any sites. Therefore, this was a problem specifically at the murga-site and the server. The randomness, and how it acted, all pointed to something going on with the server. Especially since some would say it is/was ok, others say it wasn't, then it would suddenly be ok. Anyhow, haven't seen it late yesterday and/or today.....hopefully the slowness and server response times (especially since other computers and Netflix, are on all the time here in this house and nothing was seen by any of the crew like I was seeing with murga)...hopefully it never reappears and stays away.

Besides, Mochi is one of our resident "mad geniuses" (say this with affection and respect) on murga. If he says he didn't cause the solar eclipse of the murga server, then I believe him :wink:

Post Reply