New Puppy Linux with PAE uses all RAM on your PC(s)

What features/apps/bugfixes needed in a future Puppy
Post Reply
Message
Author
gcmartin

New Puppy Linux with PAE uses all RAM on your PC(s)

#1 Post by gcmartin »

On Sun Mar 13, 2011, @P310Don, alerted the Puppy community with an appeal. This appeal was met with an investigation of memory use by Puppy Linux particularly on 32bit PCs.That investigation,not only, zeroed in on the problem, but one Puppy member, then another, then another, then others began providing, first experimental, then stable desktop distros for our use. Today, there are several stable implementations that many have tested. For me, one of them, running for 4 months, has all of the applications that I use active and stable (Thanks to that distro owner - you know who your are.)

This technological approach is newer than the one used over the past 9 years of Puppy life. It exploits a 15 year old hardware addition. (That I understood its OS architectural assistance is one thing, but, I did not ever envision use of it.) Then comes Puppy.. Of all the Linuxes on the planet, this one is different. Without addressing all the things that make it different, I will ONLY address RAM.

What Puppy does, is NOT unique to Puppy per se, but is unique, in that, it publicly announces Puppy to be a "Live RAM based OS". This is the number one reason why I, personally, use it. Over my career, I have worked in systems that require high transaction rates. The fastest way to deliver those is to keep as much as necessary in memory to address the transactional needs. I have worked with Main-frame planning all the way to OSes no one remembers anymore where we have addressed data use, and application execution needs to achieve the predictions required.

Puppy is a practical working example of this. And,,this is the primary reason I use it in the fashion that I do. I have been a user of RAM based system from IBM ATT, Microsoft, and most recently in Linux Knoppix and Puppy. The various incarnations have been mostly PXE implementations, and Live media RAM use.

What does this have to do with PAE, some may ask. well, on a 32bit system, it means that after Puppy loads, i have a large fast data filesystem supporting the OS as it brings applications into my desktop as I use them. Further, because of this, I don't have a need to close any of them as I use them. And, lastly, I can now open many-many more browser pages that I was ever able to do in OSes which had a ceiling limiting your use. PAE allows Puppy to use ALL of my PCs memory! It merely uses a different hardware for RAM references which is shown by testing ni this community to work without failures or impediments.

This is not something to sneeze at. The last corporate order I saw ordered a large number of PCs which came from the vendor with 4GB of RAM. The 2 servers came with 64 (Yes,you got that right 64) GB of RAM. Considering this company's finance model, these systems will be replaced at the end of the 3 years Lease . As such, these PCs will be dumped into the public market as they are being replaced (The older PCs were multicore 2GB desktops/laptops).

We are witnessing a mainstream change in Puppy. All of the reasons for taking this step, that we now know as "PAE", has been discussed, at length, in @P310Don's thread on PAE.
That thread was the thread which brought to light a need, and began the quest for addressing it in the Puppy world. @P310Don should be credited with his contribution to bring to light this technology.

This particular Thread
In this thread, I am asking the community for its help is presenting anything negative they can find that PAE presents. This is being done so that the distro developers will have any an all information pertinent to Puppy Linux. This way, Puppy developers can understand these as the Pups the produce for us can navigate, in such a way as they can overcome or steer around obstacles.

Request
Have you witnessed or know of any negative behavior n PAE use on ANY PAE PUP distro? In this request, I am SPECIFICALLY asking for Puppy negative behavior seen. Not any other distro, or any other approach, merely want to cover Puppy behavior in this thread.

The community prize
The prize of the community is a PUP that runs on all PCs in 32bit mode that can exploit, take advantage and use ALL of the PC's memory for user operations.

This thread is opened, again requesting something from the community as it nestle in on a technology that could help the multitude of those who have,or expect to have a large RAM PC which will be used run Puppy.

Help if you can by sharing that..

gcmartin

#2 Post by gcmartin »

Puppy Linux, traditionally, has had problms with Video, WiFI, audio, I have seen so many of these over the years, that for me (even though I don't agree it should continue) have become common-place. If I had a means to educate any Linux user coming from Midrosoft, I would instruct them to look for an deal with those items, for they will probably see them during there use of Linux at some point in time.

But, I am wondering if PAE has introduced some other problem(s) that is outside of the scope of these, that the community need be alerted to. [list][*]Has anyone seen anything?
[*]Also, is there any advice that can be given in use of the new PAE distro?[/list]
Help if you can

User avatar
rjbrewer
Posts: 4405
Joined: Tue 22 Jan 2008, 21:41
Location: merriam, kansas

#3 Post by rjbrewer »

"Some desktop Linux distributions such as Fedora Core 6, Ubuntu and openSUSE do not enable the HIGHMEM64 option by default, which is required to gain access to the NX bit in 32-bit mode, in their default kernel; this is because the PAE mode that is required to use the NX bit causes pre-Pentium Pro (including Pentium MMX) and Celeron M and Pentium M processors without NX support to fail to boot. Other processors that do not support PAE are AMD K6 and earlier, Transmeta Crusoe, VIA C3 and earlier, and Geode GX and LX. VMware Workstation versions older than 4.0, Parallels Workstation versions older than 4.0, and Microsoft Virtual PC and Virtual Server do not support PAE on the guest."

Will not work on my laptop.

Inspiron 700m, Pent.M 1.6Ghz, 1Gb ram.
Msi Wind U100, N270 1.6>2.0Ghz, 1.5Gb ram.
Eeepc 8g 701, 900Mhz, 1Gb ram.
Full installs

gcmartin

#4 Post by gcmartin »

@RJBrewer, you are correct. I believe this was discussed in P310Don's thread as well.There are some CPUs that do NOT have the built-in hardware support. Thus, on those systems, there is NOT a 32bit solution should they add more than 3GB of RAM. The CPU does not provide the OS any means of addressing that menory feature.

Thus, as @RJBrewer points out, on any processor that does not have this support, you cannot run a PAE Puppy solution to take advantage of the "RAM usage" it affords.

For those who have those CPUs, This is a CAUTION! ... to everyone.

Note, though, that of all the PCs manufactured since 2000, the overwhelming majority of PCs have this hardware feature built-in. And, PAE Puppy should will work well for those systems.

Additionally, I might add, when runnig virtual guest OS, be aware that unless the Virtual host "specifically" allow PAE, you cannot run a PAE OS as a guest. This does NOT apply to the host OS, rather, though, it DOES apply to the the Virtual guest(s). These guest must be non-PAE guest unless the host has a definition to alllow PAE enabled for use in the guest's VM..

Hope this helps. And thanks @RJBrewer.

User avatar
sickgut
Posts: 1156
Joined: Tue 23 Mar 2010, 19:11
Location: Tasmania, Australia in the mountains.
Contact:

#5 Post by sickgut »

gcmartin wrote:@RJBrewer, you are correct. I believe this was discussed in P310Don's thread as well.There are some CPUs that do NOT have the built-in hardware support. Thus, on those systems, there is NOT a 32bit solution should they add more than 3GB of RAM. The CPU does not provide the OS any means of addressing that menory feature.

Thus, as @RJBrewer points out, on any processor that does not have this support, you cannot run a PAE Puppy solution to take advantage of the "RAM usage" it affords.

For those who have those CPUs, This is a CAUTION! ... to everyone.

Note, though, that of all the PCs manufactured since 2000, the overwhelming majority of PCs have this hardware feature built-in. And, PAE Puppy should will work well for those systems.

Additionally, I might add, when runnig virtual guest OS, be aware that unless the Virtual host "specifically" allow PAE, you cannot run a PAE OS as a guest. This does NOT apply to the host OS, rather, though, it DOES apply to the the Virtual guest(s). These guest must be non-PAE guest unless the host has a definition to alllow PAE enabled for use in the guest's VM..

Hope this helps. And thanks @RJBrewer.
most distros have a -bigmem or similar 32 bit kernel, lets you use more than 4gb etc with 32 bit computer

for instance debian squeeze can be used with a 2.6.32-5-686-bigmem kernel

but these same distros usually have 2 kernels on the live-cdrom so the user can choose between a older -486 or the -686-bigmem so it still offers compatibility for cpus with out the PAE. I spose Puppys golden unbreakable rule of we must have the smallest iso possible kinda conflicts with doing things like that

User avatar
James C
Posts: 6618
Joined: Thu 26 Mar 2009, 05:12
Location: Kentucky

#6 Post by James C »

sickgut wrote:
gcmartin wrote:@RJBrewer, you are correct. I believe this was discussed in P310Don's thread as well.There are some CPUs that do NOT have the built-in hardware support. Thus, on those systems, there is NOT a 32bit solution should they add more than 3GB of RAM. The CPU does not provide the OS any means of addressing that menory feature.

Thus, as @RJBrewer points out, on any processor that does not have this support, you cannot run a PAE Puppy solution to take advantage of the "RAM usage" it affords.

For those who have those CPUs, This is a CAUTION! ... to everyone.

Note, though, that of all the PCs manufactured since 2000, the overwhelming majority of PCs have this hardware feature built-in. And, PAE Puppy should will work well for those systems.

Additionally, I might add, when runnig virtual guest OS, be aware that unless the Virtual host "specifically" allow PAE, you cannot run a PAE OS as a guest. This does NOT apply to the host OS, rather, though, it DOES apply to the the Virtual guest(s). These guest must be non-PAE guest unless the host has a definition to alllow PAE enabled for use in the guest's VM..

Hope this helps. And thanks @RJBrewer.
most distros have a -bigmem or similar 32 bit kernel, lets you use more than 4gb etc with 32 bit computer

for instance debian squeeze can be used with a 2.6.32-5-686-bigmem kernel

but these same distros usually have 2 kernels on the live-cdrom so the user can choose between a older -486 or the -686-bigmem so it still offers compatibility for cpus with out the PAE. I spose Puppys golden unbreakable rule of we must have the smallest iso possible kinda conflicts with doing things like that
Debian Squeeze 6.0.1 live on my quad-core with 8 gb of ram.Where is the PAE kernel?
AFAIK, Debian doesn't include a PAE kernel by default.
When installing Debian,the installer can download the PAE kernel if needed but it's not included on this 1.1 gb dvd. actually about the only distro to include PAE by default is Red Hat and it's spinoffs.
Attachments
Screenshot.png
(64.76 KiB) Downloaded 1602 times

gcmartin

#7 Post by gcmartin »

Yes, I can see the point in those distro doing this.

In addition, I have known of couple distros who have included 32&64bit versions on the same DVD as well.

I have a feeling that PAE will become the winner in Puppy 32bit because the older ISA bus PCs are rapidly disappearing from the Planet. Which then leaves us with later models of 32bit architectures which readily allow adding memory above 3GB RAM.

New Puppy applications and technology changes will make use of that RAM is some very good ways.

Thanks for shining the light on that fact for us.

User avatar
sickgut
Posts: 1156
Joined: Tue 23 Mar 2010, 19:11
Location: Tasmania, Australia in the mountains.
Contact:

#8 Post by sickgut »

you could have 2 kernels to choose from at the boot menu when puppy boots, and it would add maybe 35mb or so to the size of the ISO. Yes things like nvidia drivers would have to have 2 versions available for the 2 kernels but, realistically are you really gonna Be running one kernel on your computer one day then another the next day?

I think the live cdrom etc should have a choice between 2 kernels, but at install time when installing to HDD you can either install both (god knows why...) or select just one kernel to install.

User avatar
Moose On The Loose
Posts: 965
Joined: Thu 24 Feb 2011, 14:54

#9 Post by Moose On The Loose »

sickgut wrote:you could have 2 kernels to choose from at the boot menu when puppy boots, and it would add maybe 35mb or so to the size of the ISO. Yes things like nvidia drivers would have to have 2 versions available for the 2 kernels but, realistically are you really gonna Be running one kernel on your computer one day then another the next day?

I think the live cdrom etc should have a choice between 2 kernels, but at install time when installing to HDD you can either install both (god knows why...) or select just one kernel to install.
Those who develop things like drivers may need to switch back and forth to compile and test the two versions needed.

It really is a shame that the interface to modules didn't get abstracted in some way that made them not depend on versions. It does point up the advantage of avoiding making things modules when they could be daemons.

gcmartin

Shinobar's Live media Boot Selection ISOs

#10 Post by gcmartin »

sickgut wrote:you could have 2 kernels to choose from at the boot menu when puppy boots, and it would add maybe 35mb or so to the size of the ISO. Yes things like nvidia drivers would have to have 2 versions available for the 2 kernels but, realistically are you really gonna Be running one kernel on your computer one day then another the next day?

I think the live cdrom etc should have a choice between 2 kernels, but at install time when installing to HDD you can either install both (god knows why...) or select just one kernel to install.
The comments here may have a solution already available within the forum community.

@Shinobar, over a year ago, produced a boot approach that a couple others have used.

His method allows a selection at boot time of what you would want to boot.

Question
Wouldn't this allow the distro developers to provide a easy and meaningful boot selection that would address the items raised?

Hope this helps distro builders who would like to employ a method for LIve media options and use.

But, beware, that for Live media users who run their systems using Multi-session DVD/CD, there could be some not too pleasant impacts if the wrong selections are made.

User avatar
sickgut
Posts: 1156
Joined: Tue 23 Mar 2010, 19:11
Location: Tasmania, Australia in the mountains.
Contact:

#11 Post by sickgut »

James C wrote:
sickgut wrote:
gcmartin wrote:@RJBrewer, you are correct. I believe this was discussed in P310Don's thread as well.There are some CPUs that do NOT have the built-in hardware support. Thus, on those systems, there is NOT a 32bit solution should they add more than 3GB of RAM. The CPU does not provide the OS any means of addressing that menory feature.

Thus, as @RJBrewer points out, on any processor that does not have this support, you cannot run a PAE Puppy solution to take advantage of the "RAM usage" it affords.

For those who have those CPUs, This is a CAUTION! ... to everyone.

Note, though, that of all the PCs manufactured since 2000, the overwhelming majority of PCs have this hardware feature built-in. And, PAE Puppy should will work well for those systems.

Additionally, I might add, when runnig virtual guest OS, be aware that unless the Virtual host "specifically" allow PAE, you cannot run a PAE OS as a guest. This does NOT apply to the host OS, rather, though, it DOES apply to the the Virtual guest(s). These guest must be non-PAE guest unless the host has a definition to alllow PAE enabled for use in the guest's VM..

Hope this helps. And thanks @RJBrewer.
most distros have a -bigmem or similar 32 bit kernel, lets you use more than 4gb etc with 32 bit computer

for instance debian squeeze can be used with a 2.6.32-5-686-bigmem kernel

but these same distros usually have 2 kernels on the live-cdrom so the user can choose between a older -486 or the -686-bigmem so it still offers compatibility for cpus with out the PAE. I spose Puppys golden unbreakable rule of we must have the smallest iso possible kinda conflicts with doing things like that
Debian Squeeze 6.0.1 live on my quad-core with 8 gb of ram.Where is the PAE kernel?
AFAIK, Debian doesn't include a PAE kernel by default.
When installing Debian,the installer can download the PAE kernel if needed but it's not included on this 1.1 gb dvd. actually about the only distro to include PAE by default is Red Hat and it's spinoffs.
There are a zillion debian live images you can download, you can even use the cutom image creator site to supply you with an image of including any kernel you want. Just because the 4 or 5 images you have seen dont have it doesnt mean its not offered. None of the -486 kernels have PAE because none of the motherboards used with the CPUs the -486 kernel is meant to be used on can have 4GB RAM to start with.

Just a pointer not realy related to this thread subject, but the -486 kernel that the pic you supplied says you are running, doesnt support multiple cores on your CPU, whereas a -686 kernel does. I made this mistake myself when i made an OS for someone with a quad core cpu using the 486 kernel, only one core gets used. If you used the -686 kernel then it will use all your cores.

User avatar
James C
Posts: 6618
Joined: Thu 26 Mar 2009, 05:12
Location: Kentucky

#12 Post by James C »

The statement was about the default download......not creating something custom.The screenshot shows the default folders.
Besides,if I choose to run Squeeze on this box I'd most likely run 64 bit anyway.

EDIT:
Thanks for giving me the idea.....just built a Squeeze 686-bigmem iso.Pretty easy for a custom build.
Attachments
Capture.JPG
(44.27 KiB) Downloaded 1779 times

gcmartin

RAM Memory Pricing Action

#13 Post by gcmartin »

I just saw 4GB DDR3 RAM for $20 (US) and 8GB for $40. This is for DIMMs (desktops) and SO-DIMMs (laptops) That price decrease is significant The DDR2 was only $5 more.

I'm sure if you look around you can find this too, I found MicroCenter, Nextag, and Pricewatch a shopping help. Hope it works same for your RAM needs.

Post Reply