Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Wed 31 Aug 2016, 22:07
All times are UTC - 4
 Forum index » Advanced Topics » Puppy Derivatives
Not your average Puppy box
Moderators: Flash, JohnMurga
Post new topic   Reply to topic View previous topic :: View next topic
Page 1 of 2 [27 Posts]   Goto page: 1, 2 Next
Author Message
prehistoric


Joined: 23 Oct 2007
Posts: 1472

PostPosted: Wed 17 Jun 2015, 12:47    Post subject:  Not your average Puppy box
Subject description: Quirky 7.0.4.1 with 8 cores, 32 GB and 10 TB
 

I'm posting this from an unusually large Puppy box. The hard_info report is attached, (and yes it is really gzipped.) The motherboard is by Gigabyte, a used GA-970A-DS3. The processor, also used, is an AMD 8320, with 8 cores running at up to 3.5 GHz without overclocking. It has 32 GB of 1600 MHz DDR3 RAM which could be tweaked to run at 1866 MHz. I've opted for stability instead. There are 4x3 terabyte Seagate hard drives currently installed. Formatted capacity is about 2.73 TB each, or a total 0f 10.93 TB, most of which should be available for scratch memory during computations.

There were some adventures in getting to this point, and I'm not through with problems, but quirky doesn't seem to be much more problematic than many standard distros when confronted with nothing but GPT disks. I've also tried Ubuntu 15.04 and Linux Mint 17.1. Even when told they had complete freedom to wipe disks and start over these typically failed installation.

In the past I've used distros like Linux Mint which had been designed to make installation automatic on machines with a UEFI BIOS. Once I had any kind of Linux booting, it was possible to edit GRUB2 configurations to boot other systems. This was particularly helpful when dealing with laptops, which were never really intended to boot anything except Windoze.

At the moment I'm not able to recommend any distro which really does well on problem machines with UEFI BIOS. I'm booting this one from a flash drive because of problems installing to those GPT-only drives. For me this counts as a failure. On other machines I have actually installed SSDs for the OS, to get around problems in booting, leaving GPT hard drives for data.

Other problems have included either not allowing cpu frequency scaling, or setting this to such conservative values as to make the resulting system useless for the heavy computation intended. It is possible to override such settings, but I don't feel like investing that effort in a distro which would rather I did not use it that way. Quirky is now straightforward in this regard.

Still another problem turned up when several distributions failed to establish a working Internet connection using the Realtech 8169 ethernet controller. This seemed to be close to working, but it was easier to use Quirky than to debug the problems on other systems.

There were even problems recognizing mouse and keyboard when these were connected via a hub in a KVM switch. I ended up plugging in directly, though Quirky did a better job than others. It was just too much trouble to rearrange cables when I switched distros.

I've decided that implementation of GPT partition tables is uneven. The only characteristic I've really found consistent in different BIOSes is the ability to boot Windoze. Simply using Gparted on different distributions is likely to wreck partition tables and filesystems. For safety, stay with one version you can trust, there are definite incompatibilities. I was fortunate that I was building this box from scratch, and did not have any files to risk on those drives.

My overall impression of UEFI is that a great deal has been done to complicate a simple idea of being able to initialize a variety of devices, address huge mass storage devices and bootstrap systems from minimal code in a BIOS. (We tend to forget that the first BIOS was in 8 KB of mask-programmed ROM. We now have a complete OS in some BIOSes.) Whatever abstractions have been introduced to make code independent of particular hardware have been severely compromised by vendors attempting to gain temporary competitive advantage.

So far, this machine has demonstrated stability by calculating Pi to 5,000,000,000 decimal places in a little over half an hour. I've still got more to do in convincing existing computational software it is not a good idea to put huge temporary files in the same place as the OS.

I'm open to suggestions for what you do with this much computational power, besides running servers or playing games. (My only reason for that rebuilt Radeon card is to get video memory bandwidth off the processor-memory bus.) This is a machine one could scarcely have imagined when I started in computing. The millennium has arrived. What do we do next?

p.s. there was a mysterious problem with drive icons. They got messed up during experiments with Gparted changing disk drive partition tables and file systems, and never recovered. At one point the software identified an internal hard drive as a USB drive for reasons I can't explain. At another point it showed icons for USB drives not plugged in. I did a clean install from DVD to USB flash drive to get this running nicely for the screenshot.
screenshot.jpg.gz
Description 
gz

 Download 
Filename  screenshot.jpg.gz 
Filesize  126.21 KB 
Downloaded  190 Time(s) 
dmesg.txt.gz
Description 
gz

 Download 
Filename  dmesg.txt.gz 
Filesize  12.97 KB 
Downloaded  71 Time(s) 
hardinfo_report.html.gz
Description  hardinfo report on machine actually running Quirky 7.0.4.1
gz

 Download 
Filename  hardinfo_report.html.gz 
Filesize  7.01 KB 
Downloaded  98 Time(s) 
Back to top
View user's profile Send private message 
rokytnji


Joined: 20 Jan 2009
Posts: 2128
Location: Pecos/ Texas

PostPosted: Wed 17 Jun 2015, 13:29    Post subject:  

Sounds pretty prehistoric. Smile

Poor mans version I guess would be a HP Z800 Workstation with 12 ram slots.

Linux-Lite installed to my Apple 120 gig GPT SSD drive on my Dell E5500 laptop ok.
Not sure why you are having trouble with GPT partitions.

Quote:
I'm open to suggestions for what you do with this much computational powe


Kernel building will only take minutes than hours. Video editing should fly like the wind also.
But I am no expert on these sort of things.
Boinc maybe?

_________________
Linux Training More More Accepting Death is the only way to stay alive.
Back to top
View user's profile Send private message Visit poster's website 
Jasper

Joined: 25 Apr 2010
Posts: 1350
Location: England

PostPosted: Wed 17 Jun 2015, 15:06    Post subject:  

Hi prehistoric,

Re your: "I'm open to suggestions for what you [I] do with this much computational power,......"

The largest prime number found thus far has 17,425,170 decimal digits so it would likely take more than a year to make a manuscript copy (your pi calc was impressive, too).

So you might join GIMPS (Great Internet Mersenne Prime Search)
http://www.mersenne.org/ letting us know if you do and especially if you become a "winner".

That's a lot of power you have (perhaps with even more oomph than Forum member Q5sys and his jet-propelled flying m/c).

One of our sons has just given me his discarded 8to10 year old desktop computer (lightning fast to me). I run Precise 5.6 Multi-session-CD/DVD (so it runs entirely in ram).

A 240 M sized remaster using Gzip-1 and the code below takes 10 seconds here and it would be interesting to know your result for an identical, or similar, experiment.

My regards
ram2sfs.jpg
Description  amend the highlighted text as, and if, appropriate
jpg

 Download 
Filename  ram2sfs.jpg 
Filesize  50.29 KB 
Downloaded  155 Time(s) 
ram2sfs.gz
Description  rename after download by removing the fake (dot).gz
gz

 Download 
Filename  ram2sfs.gz 
Filesize  313 Bytes 
Downloaded  81 Time(s) 
Back to top
View user's profile Send private message 
Ted Dog


Joined: 13 Sep 2005
Posts: 4010
Location: Heart of Texas

PostPosted: Wed 17 Jun 2015, 15:15    Post subject:  

lol, why just why are you a puppylinux fan? You seem so out of the norm here, of course you should be using expand2 ram option in FatDog with all that memory. Wink

Hay Mods can we force his name from prehistoric to future-storic, I do not know if I could even fill up that amount of hard drive space in my lifetime.
Back to top
View user's profile Send private message 
bigpup


Joined: 11 Oct 2009
Posts: 7758
Location: Charleston S.C. USA

PostPosted: Wed 17 Jun 2015, 16:19    Post subject:  

When using Gparted. Use the latest version, not what comes with an operating system.

Gparted live CD or USB.

For partitioning I would suggest you use the Gparted live CD or USB that you can get from here.
You can download a free version to make your own Gparted live CD or USB.
It is up to date and specifically made to run Gparted.
Info:
http://gparted.sourceforge.net/livecd.php
Download:
http://sourceforge.net/projects/gparted/files/gparted-live-stable/

From the latest version of Gparted manual.
Quote:
Note

The default partition table type is msdos for disks smaller than 2 Tebibytes in size (assuming a 512 byte sector size) and gpt for disks 2 Tebibytes and larger.

See the section called “Specifying Partition Type” for msdos partition table limitations.
Note

To use a disk without a partition table, choose loop to create a virtual partition that spans the disk. Then format to the desired file system.

See the section called “Formatting a Partition” to format a virtual partition with a file system.
Caution

Many operating systems recognize gpt and msdos partition tables, but do not recognize all types of file systems. This lack of file system recognition means that using a disk without a partition table involves more risk. For example, some operating systems might prompt to format an unpartitioned disk if the file system is not recognized.

_________________
I have found, in trying to help people, that the things they do not tell you, are usually the clue to solving the problem.
When I was a kid I wanted to be older.... This is not what I expected Shocked
Back to top
View user's profile Send private message 
prehistoric


Joined: 23 Oct 2007
Posts: 1472

PostPosted: Wed 17 Jun 2015, 18:22    Post subject:  

Jasper wrote:
...One of our sons has just given me his discarded 8to10 year old desktop computer (lightning fast to me). I run Precise 5.6 Multi-session-CD/DVD (so it runs entirely in ram).

A 240 M sized remaster using Gzip-1 and the code below takes 10 seconds here and it would be interesting to know your result for an identical, or similar, experiment.
I've downloaded your script, and will report results when I get them, but right now I'm embroiled in some pesky activities required to take care of two houses and a menagerie. Computers get deferred because I've never had one crap on the floor.
Back to top
View user's profile Send private message 
prehistoric


Joined: 23 Oct 2007
Posts: 1472

PostPosted: Wed 17 Jun 2015, 18:42    Post subject:  

Ted Dog wrote:
lol, why just why are you a puppylinux fan? You seem so out of the norm here, of course you should be using expand2 ram option in FatDog with all that memory. Wink

Hay Mods can we force his name from prehistoric to future-storic, I do not know if I could even fill up that amount of hard drive space in my lifetime.
You will understand that prehistoric remains a suitable moniker when you learn that I'm running the heavy computation from a pure console interface without a desktop. I only use the desktop in setting things up for a run. If I can separate the trivial-sized temporary files from the humongous ones, quirky may be the fastest OS available. Running from RAM cuts I/O activity, which becomes a bottleneck when you are working with large scratch files or RAID arrays. A few megabytes out of 32 GB is a small price to pay for this speed up.

If I had to, I could install a 5th 3 TB drive. This would give me over 12 TB for scratch space. This would be enough to calculate Pi to 1.5 trillion decimal digits, though it would take a while. I just got a buy on 3 TB drives, and had this chassis sitting here, so I thought I'd see what happens when I try to max it out. The drives may still go in other computers, with the statement they have been carefully burned in and tested for reliability.

I'm actually curious about problems in combinatorial optimization which might be cracked this way, maybe something which might give Puppy systems an unfair advantage. More generally, I wonder about problems which might benefit people in general which could use a dedicated machine, perhaps something in computational biology.

For integer arithmetic I think this AMD cpu is quite competitive. For compute-bound problems with heavy floating point, Intel processors probably have an advantage, but I'm not planning to run out and plop down a kilobuck for a chip with a lower clock rate.
Back to top
View user's profile Send private message 
prehistoric


Joined: 23 Oct 2007
Posts: 1472

PostPosted: Wed 17 Jun 2015, 18:55    Post subject:  

bigpup wrote:
When using Gparted. Use the latest version, not what comes with an operating system.

Gparted live CD or USB.

For partitioning I would suggest you use the Gparted live CD or USB that you can get from here.
You can download a free version to make your own Gparted live CD or USB.
It is up to date and specifically made to run Gparted.
Info:
http://gparted.sourceforge.net/livecd.php
Download:
http://sourceforge.net/projects/gparted/files/gparted-live-stable/...
Thanks bigpup! I'll try the latest.

I have an older gparted CD, but didn't want to mix versions while I was trying automatic installation. I think some of these installation systems in distros depend on defects in the version they call to match defects in their set-up scripts. Ugh!

I'm hoping BarryK will figure this mess out in 7.0.5 or so. My short term memory is not what it was when I was a young whippersnapper. I can recall a time when I read an 8 foot shelf of manuals before I understood how to install a new computer. Today, I forget what I'm doing while I doing it. I've become extremely cautious about things like leaving stoves on or water running.

I also have a problem with not completely forgetting things like how to punch PL/1 scientific characters on an O26 keypunch.
Back to top
View user's profile Send private message 
8Geee


Joined: 12 May 2008
Posts: 866
Location: N.E. USA

PostPosted: Thu 18 Jun 2015, 06:37    Post subject:  

I'm a bit behind the times here also... I thought UEFI could be disabled by using the legacy setting in bios. Or maybe the MoBo will not allow? AMD going UEFI... perish the thought.
_________________
Linux user #498913
Back to top
View user's profile Send private message 
prehistoric


Joined: 23 Oct 2007
Posts: 1472

PostPosted: Thu 18 Jun 2015, 09:14    Post subject:  

8Geee wrote:
I'm a bit behind the times here also... I thought UEFI could be disabled by using the legacy setting in bios. Or maybe the MoBo will not allow? AMD going UEFI... perish the thought.
There certainly are workarounds, one of which I'm using at present. I've turned off secure boot. I can boot legacy devices.

The problem is that accessing a full 3 TB drive without workarounds currently requires gpt disks w/o traditional partition tables. I can see the handwriting on the wall for a time when proprietary systems will close off those other options, if not absolutely, then by making the complexity even worse for consumers wishing to do anything that is not proprietary.

Open Source systems are currently at a disadvantage they did not create themselves. Perversely, this complexity is then attributed to those systems. These changes are always being pushed as matters of "security". This despite abundant evidence that many commercial systems are riddled with terrible vulnerabilities. The term "security" must mean "provides us with a secure cash-flow."

I support some systems for a few friends who can't wean themselves from dependence on Microsoft, etc. Keeping them from doing their banking via Moldova is a constant struggle.

I not only run Linux myself, I have also provided Linux systems to people willing to make the effort. Some use Puppy, some use Ubuntu or Linux Mint. I'll admit that updates for Puppy are not yet as easy as I'd like. Updates for Ubuntu and Linux Mint are now easier to deal with than those for Windoze.

(If you don't believe me, try to take a W7 netbook with 2 GB RAM back to system restore, then do all the updates to bring it up to date. I spent several days on the telephone with one friend who had to do this after his system was infested. It would have taken 30 minutes to do a complete reinstall of a popular Linux system from scratch, and I'm not even considering Puppy. Oh yes, anyone else get hit by the Windows update that declared some legitimate W7 systems were pirated?)

One friend who had IT experience, but not much Linux experience, took my suggestion about putting a popular Linux system on the machine in his workshop. (He chose Ubuntu.) I get about 5 questions from him about problems with Windoze for every question concerning Linux. We've almost found substitutes for the last applications tying him to Windoze.

At the other extreme I want people to understand that the U.S. Office of Personnel Management was hacked, losing records on some 14 million employees with security clearances. These included a 127-page background questionnaire which one friend spent a month filling out. (How does he feel about Chinese agents getting this? Unprintable.) Do government leaders have better security at their command? Nein. What about computer security companies? Oops.

All this is without considering "coming attractions". I still haven't seen the USB flashdrive firmware hack in the wild, (probably because normal security is so poor it isn't necessary to go to such lengths.) And what about firmware modifications in storage systems?

I lack the resources to deal with all these potential threats, but there is one thing I can do right now: stop running common software which makes it easy for such compromises to "phone home" to their creators. If such hacks need to try many different approaches to compromising the OS the probability of detecting them goes way up.

Does this help explain why I am doing things "the hard way"?
Back to top
View user's profile Send private message 
prehistoric


Joined: 23 Oct 2007
Posts: 1472

PostPosted: Thu 18 Jun 2015, 10:22    Post subject:  

I'm still not on that monster machine, so various experiments will have to wait.

There is probably some misunderstanding of what I'm asking for in terms of suggested computational tasks. I use programs for calculating ridiculous numbers of digits for Pi or gamma to test that machines are reliable. I have no particular need of billions of digits which might be mistaken for random numbers.

I'm aware of such things as searches for Mersenne primes or non-trivial zeros of the Riemann zeta function, and feel like this has gone quite far already. I don't feel like further computation there is of much use to anyone, except for bragging rights.

I'm probably not a good sysadmin for a compile farm, though I might be willing to do something particularly important if it doesn't overload my declining mental powers. I no longer consider myself a programmer, and my mental stamina is limited.

What I'm wondering about are applications of modern techniques in combinatorial optimization which nobody else has tried. Here's something pretty strange in antenna design as an example. This used a genetic algorithm, but there are other techniques like simulated annealing. These are just two approaches out of many. Here's a book which gives an overview. (Caution: if you buy the ebook watch out for "preapproved future charges" to credit cards or Paypal. This could be a legitimate way to provide updates, or it could be a scam.) You can read the book on-line for free.

There is no shortage of algorithms, and a range of programs are available. What I'm looking for are applications which have not already been done. It would be especially cute to apply this to the problem of packing a maximum of functionality into Puppy, if only we could automate the evaluation of solutions.
Back to top
View user's profile Send private message 
Puppus Dogfellow


Joined: 07 Jan 2013
Posts: 1354
Location: nyc

PostPosted: Sun 21 Jun 2015, 22:32    Post subject:  

prehistoric wrote:

[...]
What I'm looking for are applications which have not already been done. It would be especially cute to apply this to the problem of packing a maximum of functionality into Puppy, if only we could automate the evaluation of solutions.


i'm not sure if this is what you have in mind, but BarryK said it took him 12 hours to compile libreoffice from scratch in (iirc) april and that he hasn't yet attempted to do so in t2. from the post that follows the link:


BarryK wrote:
Bindee wrote:
Quote:
takes about 12 hours to compile.


Shocked

What are you using , An atom based notebook.?

Mr. Green


I have an Intel i3-370M CPU, with effectively 4 cores, however I compiled Libreoffice on 1 core only, which made it extra slow.

I have a problem with big compiles if use all cores, it seems my CPU overheats and the laptop abruptly turns itself off.

I get this compiling gcc in T2, and Seamonkey inside or outside T2. For T2, the default was one-less than the max cores, so it was using 3. No good, still crashed the laptop.

maybe 2 cores would work, but I have got T2 set to one core, very painfully slow. Besides, we have discovered some apps fail to compile in T2 if use more than one core.
Though, T2 can be setup to use one core on a per-app basis.



link has Barry's notes as an attachment if you want to look into it further. maybe you could tax the machine by running a 32 bit virtual machine simultaneously and compile updated for 32 and 64 at one go. see if you could beat barry's time. if the doggy bag benchmark takes off, useful apps may start really rolling in. or not. time to beat: 12 hours.

Very Happy
(i think he's a minor revision or two behind and i don't think he plans to make many more stripped down libreoffice packages).
Back to top
View user's profile Send private message 
NeroVance


Joined: 10 Oct 2012
Posts: 186
Location: Canada

PostPosted: Sun 21 Jun 2015, 22:54    Post subject:  

For a prehistoric fellow, you sure got some highend tech there mate Wink.
I imagine that there are probably people out there who would kill for a machine like that, though as for what to do with such a beast, I can't say atm.
Back to top
View user's profile Send private message Visit poster's website 
prehistoric


Joined: 23 Oct 2007
Posts: 1472

PostPosted: Mon 22 Jun 2015, 10:38    Post subject:  

NeroVance wrote:
For a prehistoric fellow, you sure got some highend tech there mate Wink.
I imagine that there are probably people out there who would kill for a machine like that, though as for what to do with such a beast, I can't say atm.
Please note that I did not buy this stuff new, and I couldn't very well get it used until the bloom was off the rose as far as gamers were concerned.

@Puppus Dogfellow,

I've checked that this system does not overheat when all cores are fully loaded. That Pi computation is an example. I doubt there would be a problem with compiling.

I'm running this with a stock AMD cooler, though even that has heat pipes. I can check the progress of a heavy computation by listening to the CPU fan rev up.

It would help if someone else were to do the business of setting various flags, and writing scripts. I've stopped running kernel compiles myself because there are simply too many things to remember for my impaired short-term memory.

I'm still in the business of house and dog sitting. I'll have a chance to experiment with that monster in a few days.
Back to top
View user's profile Send private message 
gcmartin


Joined: 14 Oct 2005
Posts: 6620
Location: Earth

PostPosted: Mon 22 Jun 2015, 19:50    Post subject:  

Hello @Prehistoric

2 PUPPY distros come to mind which scale to your system. Namely @StemSee's distro(s) and @Dry Falls distro(s).

Both of these provide OOTB operations with almost NO NEED to install additional software to become immediately useful. @StemSee's distro has all the necessary stuff builtin to allow your machine to run as many VM Guests that you would ever want, again, with no need to install software to immediately test and use.

Post any behavior you experience on their threads for any findings, assistance, or guidance you desire, as they are responsively helpful.

Hope this is useful info as you proceed. And congratulation on your newest BIG DOG! Cool

_________________
Get ACTIVE Create Circles; Do those good things which benefit people's needs!
We are all related ... Its time to show that we know this!
3 Different Puppy Search Engines or use DogPile
Back to top
View user's profile Send private message 
Display posts from previous:   Sort by:   
Page 1 of 2 [27 Posts]   Goto page: 1, 2 Next
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » Advanced Topics » Puppy Derivatives
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.1642s ][ Queries: 12 (0.0047s) ][ GZIP on ]