(OLD) (ARCHIVED) Puppy Linux Discussion Forum Forum Index (OLD) (ARCHIVED) Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info

This forum can also be accessed as http://oldforum.puppylinux.com
It is now read-only and serves only as archives.

Please register over the NEW forum
https://forum.puppylinux.com
and continue your work there. Thank you.

 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups    
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Thu 24 Sep 2020, 08:51
All times are UTC - 4
 Forum index » Advanced Topics » Puppy Projects
DebianDog64 - 64 bit DebianDog-Jessie
Moderators: Flash, JohnMurga
This forum is locked: you cannot post, reply to, or edit topics.   This topic is locked: you cannot edit posts or make replies. View previous topic :: View next topic
Page 17 of 38 [560 Posts]   Goto page: Previous 1, 2, 3, ..., 15, 16, 17, 18, 19, ..., 36, 37, 38 Next
Author Message
fredx181


Joined: 11 Dec 2013
Posts: 4481
Location: holland

PostPosted: Fri 20 May 2016, 14:28    Post subject:  

Hi William,

Quote:
I haven't really been bothering about copy2ram capability of Porteus boot method, but inspired by amazing fast performance of Slitaz cooking I'm inspired to work play with DDjessie64 running in RAM (I have 2GB system, so I'm presuming, like Slitaz, that will work).


From what I experienced copy2ram doesn't make (much?) difference, I *think* specially when using the changes=EXIT:/... cheatcode (or no changes).
I don't have much knowledge about this copy2ram matter so just some thoughts and tests from me here, I think rufwoof wrote a good explanation, specially this:

rufwoof wrote:
A benefit of copy2ram is that assuming it does all fit into memory, then once copied you can remove the medium from which it was copied. On the fly copying however requires that the medium remains available throughout the session.


I've found that when copy2ram is used, when booting from USB without save (or with save to HDD) I could indeed unplug the USBflash (or whatever) or unmount it, and the system still runs normally, which proves to me that it's really running in memory then.

Using the changes=EXIT:/... cheatcode:
This way the changes made are done in tmpfs first before you'd run save2flash (at shutdown or in the middle of a session)
That alone makes it a whole lot faster.
(see also below df -h output for /mnt/live/memory/changes)
I found when not using EXIT:/... and using a cheap USB flash drive (and changes saving to it directly), it's much slower, specially when installing programs but also for running e.g. firefox, as it constantly creates new files.

Here's output for me of free, df -h and losetup -a with and without copy2ram used (both without changes=.... parameter and just booted):

Code:
######################################
   With copy2ram
######################################

root@jessie64:~# free
                     total       used       free         shared       buffers     cached
Mem:       3084368     460572    2623796     186004      48856     291280
-/+ buffers/cache:       120436    2963932
Swap:       418812          0     418812

root@jessie64:~# df -h
Filesystem      Size  Used Avail Use% Mounted on
aufs            1.8G  956K  1.8G   1% /
tmpfs           1.5G  159M  1.4G  11% /mnt/live
devtmpfs         10M     0   10M   0% /dev
tmpfs           1.8G  956K  1.8G   1% /mnt/live/memory/changes
/dev/loop0      158M  158M     0 100% /mnt/live/memory/images/01-filesystem.squashfs
....

root@jessie64:~# losetup -a
/dev/loop0: [000e]:9409 (/memory/copy2ram/01-filesystem.squashfs)

######################################
   Without copy2ram
######################################

root@jessie64:~# free
                    total       used       free           shared    buffers     cached
Mem:       3084368     371004    2713364      24944      49732     200860
-/+ buffers/cache:       120412    2963956
Swap:       418812          0     418812

root@jessie64:~# df -h
Filesystem      Size  Used Avail Use% Mounted on
aufs            1.8G  952K  1.8G   1% /
tmpfs           1.5G  1.3M  1.5G   1% /mnt/live
devtmpfs         10M     0   10M   0% /dev
/dev/sdb1       1.1G  787M  252M  76% /mnt/sdb1
tmpfs           1.8G  952K  1.8G   1% /mnt/live/memory/changes
/dev/loop0      158M  158M     0 100% /mnt/live/memory/images/01-filesystem.squashfs
....

root@jessie64:~# losetup -a
/dev/loop0: [0811]:36826 (/mnt/sdb1/live/01-filesystem.squashfs)


Note that:
- free output for 'used' on first line is almost 90MB higher when copy2ram is used, don't know where that comes from.
- df -h output of /mnt/live (tmpfs) is with copy2ram a lot higher, by the same amount as the size of 01-filesystem,squashfs is (157MB)
- losetup -a output, with copy2ram it's the (compressed) file: (/mnt/live)/memory/copy2ram/01-filesystem.squashfs
(don't understand that df -h says in both cases: /mnt/live/memory/images/01-filesystem.squashfs, btw)

Again, not really conclusions, just some thoughts and tests, hope it helps.

Edit: BTW, of course copy2ram can be a great option when booting from cd or dvd, because then read speed is a lot higher running the system (read speed of the cdrom drive or the cd/dvd is always much slower, but depends on the quality though)
But booting might be a lot slower though (delay, waiting for copying to ram)

Fred
Back to top
View user's profile Send private message 
mcewanw

Joined: 16 Aug 2007
Posts: 3200

PostPosted: Fri 20 May 2016, 19:45    Post subject:  

Thanks Fred,

I see what you mean - it obvously does something since tmpfs indeed showing as being used by 01-filesystem.squashfs size.

The reason, as I said, I became interested was because I tried Slitaz cooking, which also loads completely into RAM, but it is so blazingly fast loading/running everything. It is so fast I can't believe it is just because the apps used in Slitaz are specially compiled/chosen. But very noticeable in Slitaz is that it uses some hundreds of MB RAM according to 'free' report.

What I'm wondering is that maybe its squashfs's (assuming it uses such) are being uncompressed when loaded into RAM, whereas DebianDog, from the figures you display is loading them in still compressed - resulting in not a great performance difference. Perhaps if I unsquash and resquash 01-filesystem.squashfs with lowest compression will have a major effect or I wonder how to load it in completely uncompressed as an experiment. I guess it is fast enough as it is, but wow, with systems that have enough RAM I'd love it to be as fast as Slitaz.

I'll check Puppy Tahr also, because from memory it didn't run any faster than DebianDog so maybe Puppy load to RAM leaves things uncompressed also, which surprises me in view of Puppy's desire for fastest performance amongst the tiny distributions?

I've noted jamesbond commenting somewhere once that he had experimented with uncompressing into RAM with his FatDog, but I presume he didn't adopt that so that his system worked better with low RAM machines - but with systems having much more RAM nowadays, it would be nice to use fastest setup possible I feel.

William

_________________
github mcewanw
Back to top
View user's profile Send private message Visit poster's website 
mcewanw

Joined: 16 Aug 2007
Posts: 3200

PostPosted: Fri 20 May 2016, 23:25    Post subject:  

Hi Fred,

After checking performance stats of the different compressors here:

http://catchchallenger.first-world.info/wiki/Quick_Benchmark:_Gzip_vs_Bzip2_vs_LZMA_vs_XZ_vs_LZ4_vs_LZO

I simply used remasterdog program to remaster my 01-filesystem.squashfs with LZ4 compression rather than xz.

On my system, which is 2GHz Core Duo II with 2 GB RAM, Palemoon now starts up in about 1 second (immediately after first booting) compared to the 3 or 4 seconds it did before. Very impressed with the new system responsiveness more generally too. Fast enough for me since most apps load pretty much instantaneous now (and that's without copy2ram - see note below).

Thus I'll be sticking with LZ4 since disc storage space isn't an issue and the default 01-filesystem.squashfs I have is still only 245 MiB in size (with xz compression it was 157 MiB).

Note: that copy2ram does copy that compressed file into RAM (tmpfs) but Palemoon still took 1 sec to load so no advantage for me - best I keep my RAM and just use the LZ4 squashfs loop mounted, as it is, from my hard disk. Of course, with slower storage medium copy2ram might make a difference, but machines with lots of RAM usually have reasonable fast storage medium too.

EDIT: I also checked TahrPup64 in case I could find an improvement (yes, a comparison, but making a Puppylike system and improving systems more generally relies on that). In fact, Palemoon (on first boot - faster after of course) in TahrPup64 also took about 3 or 4 seconds to load on my machine and I checked its puppy....sfs and zdr....sfs compression type with unsquashfs -s', which revealed xz compression (like default DDjessie64). I guess therefore that TahrPup64 could also be made 3 to 4 times more responsive (on my machine specs) by re-squashing these with LZ4 compression instead. From the compression-type comparison link above, I don't expect gz would give much decompression speed advantage compared to xz - not sufficient to bother about anyway - but change to LZ4 squashfs compression algorithm definitely worth it on my class of machine.

EDIT2: Another big advantage of using LZ4 rather than xz or gz is that it makes Remastering similarly much faster too. I now prefer to remaster regularly so as to only keep a very small persistence changes save folder (which I use with save on EXIT). For small iso download size, however, xz remains best IMO, however, and easy to remaster with LZ4 once downloaded.

William

_________________
github mcewanw
Back to top
View user's profile Send private message Visit poster's website 
fredx181


Joined: 11 Dec 2013
Posts: 4481
Location: holland

PostPosted: Sat 21 May 2016, 04:34    Post subject:  

Hi William, yes LZ4 is a great invention.
Maybe you didn't follow back then as it was summer for you, but we had discussion about LZ4 initiated by rufwoof in the Jessie thread starting around here;
http://www.murga-linux.com/puppy/viewtopic.php?p=883550#883550
(and continued in the next few pages, testing newer kernels, but finally turned out that it was just a matter of compiling only squashfs.ko, and replace the original one with it in initrd*). http://www.murga-linux.com/puppy/viewtopic.php?p=884874#884874

Fred
Back to top
View user's profile Send private message 
mcewanw

Joined: 16 Aug 2007
Posts: 3200

PostPosted: Sat 21 May 2016, 04:59    Post subject: LZ4  

Oops, yes, sorry... I had family visitors from Philippines staying here as well as my university-going daughter at that time and was hardly online at all. Oh well, better late than never! It's a terrific DebianDog addition. Thanks also to rufwoof for bringing that idea up.

William

_________________
github mcewanw
Back to top
View user's profile Send private message Visit poster's website 
rufwoof


Joined: 24 Feb 2014
Posts: 3725

PostPosted: Sat 21 May 2016, 08:19    Post subject:  

I boot DD Jessie (32 bit) with changes on EXIT. Generally I don't save ... only when any updates occur i.e. apt-get update; apt-get upgrade after initial boot periodically and if updates occur I run save2flash.

If between those times I compress my changes folder into a squashfs and clear out the changes folder ... it all runs much quicker. The inconvienience however is that to apply updates I extract the squashfs back into changes folder, delete the squashfs ... before rebooting and running apt-get .... etc.

lz4 is great (can decompress at close to ram speeds), but in DD-Jessie that's not available to me so I use low lzop compression (which is also reasonably fast to decompress) i.e. something like

mksquashfs changes 09-changes.squashfs -comp lzo -Xcompression-level 1

One thing that clearly stands out is starting up conky. When changes folder is loaded it takes a few seconds, when changes folder content is in a squashfs its near instant.

A few months back I mostly did squashfs the changes folder for that extra speed benefit, but more recently I don't bother, its easier just to leave changes folder as-is. I guess I could create a 2nd main filesystem of the changes to date to sit alongside 01-filesystem.squashfs, but I'd rather keep the system relatively clean (avoid layering upon layers).

I'm still running on a 32 bit Celeron despite having a 64 bit PC sitting behind me waiting to be setup. DD-Jessie run the way Toni suggests is just great and I've had no need/little inclination to swap over the PC just yet. When I do I'll likely swap over to DD-Jessie 64 (and start using LZ4). Or I may go down the Debian full install path. I do like booting the exact same pristine system (that can have updates installed/saved) each/every time and as such dislike full installs that dynamically change. But if I can figure out a way to relatively quickly copy in a backup image as the main full install copy then that would have appeal to me. i.e. when updates are apparent copy in the last pristine backup, reboot, apply updates, make backup of that as the latest pristine version. If however the backup copy/restore process takes more than a few minutes then the appeal of that full install choice is lost to me.
Back to top
View user's profile Send private message 
fredx181


Joined: 11 Dec 2013
Posts: 4481
Location: holland

PostPosted: Sat 21 May 2016, 09:24    Post subject:  

rufwoof wrote:
lz4 is great (can decompress at close to ram speeds), but in DD-Jessie that's not available to me so I use low lzop compression (which is also reasonably fast to decompress) i.e. something like

You mean that the lz4-support package doesn't work for you?
https://github.com/DebianDog/Jessie/wiki/LZ4-compression-boot-support

Fred
Back to top
View user's profile Send private message 
rufwoof


Joined: 24 Feb 2014
Posts: 3725

PostPosted: Sat 21 May 2016, 14:39    Post subject:  

It did (LZ4) - when I was first learning DD-Jessie. But then Toni steered me towards using the formal setup/updates to which I migrated to and have stuck with since.

Still don't really understand updates etc. just a end user who presses buttons, and like the stability and relative speed of security patches Smile
Back to top
View user's profile Send private message 
mcewanw

Joined: 16 Aug 2007
Posts: 3200

PostPosted: Sat 21 May 2016, 18:44    Post subject:  

rufwoof wrote:
It did (LZ4) - when I was first learning DD-Jessie. But then Toni steered me towards using the formal setup/updates to which I migrated to and have stuck with since.


I'm not sure I understand. Does that mean I'll lose LZ4 on my DDjessie64 if I apt-get update?

William

_________________
github mcewanw
Back to top
View user's profile Send private message Visit poster's website 
rufwoof


Joined: 24 Feb 2014
Posts: 3725

PostPosted: Sat 21 May 2016, 19:45    Post subject:  

Sorry I don't know for sure. Toni's guidance fundamentally was be careful what you install outside of Synaptic as that could screw up the database ... or something to that effect. So I just followed that guidance. I don't load other SFS's or anything other than what can be installed via Synaptic (apt-get) - so as to maintain a more conformist/stable system.

I believe (??) that DD Jessie 64 is different - isn't based totally on the stable Debian version ??? (Has elements of the next 'test' (Stretch ??) version). So more up to date, but potentially less stable (although by the time Debian stuff reaches 'test' phase its apparently quite stable). i.e. Fred's not as rigid about stability/core 'Debian Stable' as Toni).
Back to top
View user's profile Send private message 
fredx181


Joined: 11 Dec 2013
Posts: 4481
Location: holland

PostPosted: Sun 22 May 2016, 04:07    Post subject:  

rufwoof wrote:
I believe (??) that DD Jessie 64 is different - isn't based totally on the stable Debian version ??? (Has elements of the next 'test' (Stretch ??) version). So more up to date, but potentially less stable (although by the time Debian stuff reaches 'test' phase its apparently quite stable).


No, DD64 is pure Jessie with just one exception: the squashfs-tools package is newer, borrowed from Stretch (added it to the custom repository). (for LZ4 support)
But that's not doing any harm, no risk of breaking dpkg.
It would be another story if it would depend on other libraries from Stretch, but it doesn't.

Quote:
Fred's not as rigid about stability/core 'Debian Stable' as Toni

I am also when it's about risk of breaking dpkg, I would never recommend a mixed Stable/Testing system.
(e.g.libc6 from Stretch in Jessie is going to break dpkg sooner or later, unless the (advanced) user knows exactly what to do or not).

Fred
Back to top
View user's profile Send private message 
rufwoof


Joined: 24 Feb 2014
Posts: 3725

PostPosted: Sun 22 May 2016, 05:47    Post subject:  

fredx181 wrote:
Quote:
Fred's not as rigid about stability/core 'Debian Stable' as Toni

I am also when it's about risk of breaking dpkg, I would never recommend a mixed Stable/Testing system.
(e.g.libc6 from Stretch in Jessie is going to break dpkg sooner or later, unless the (advanced) user knows exactly what to do or not).

Thanks Fred.

Where/what has libc6 from Stretch in Jessie? Is DD64 (and DD32) Jessie not at risk from that potential dpkg corruption?
Back to top
View user's profile Send private message 
fredx181


Joined: 11 Dec 2013
Posts: 4481
Location: holland

PostPosted: Sun 22 May 2016, 06:12    Post subject:  

Hi rufwoof,

Quote:
Where/what has libc6 from Stretch in Jessie? Is DD64 (and DD32) Jessie not at risk from that potential dpkg corruption?


Not sure if I understand the question.
I just meant if libc6 from Stretch would be installed in Jessie.
You'd get that situation only when mixing Jessie and Stretch, e.g. by adding Stretch repository in /etc/apt/sources.list (which is not the case in DD64 and DD32) or installing .deb packages from Stretch (depending on libc6 from Stretch).
Then there's risk of potential dpkg corruption, but nothing to worry about for DD64 and DD32 as they are both not a mixed system (based on Jessie only).

Edit: Btw, about the LZ4 support in DD32:
Knowing Toni, if the lz4-support package would be a risk in some way, he wouldn't have added it to the wiki on github.
(or at least added some big fat warning, which he didn't)

Fred
Back to top
View user's profile Send private message 
rufwoof


Joined: 24 Feb 2014
Posts: 3725

PostPosted: Sun 22 May 2016, 14:47    Post subject:  

Thanks again Fred. A lot clearer now Smile
Back to top
View user's profile Send private message 
fredx181


Joined: 11 Dec 2013
Posts: 4481
Location: holland

PostPosted: Thu 07 Jul 2016, 15:25    Post subject:  

Hi All,

DebianDog website launched (includes similar projects, such as this DebianDog64), see more info here:
http://www.murga-linux.com/puppy/viewtopic.php?p=911585#911585

Fred
Back to top
View user's profile Send private message 
Display posts from previous:   Sort by:   
Page 17 of 38 [560 Posts]   Goto page: Previous 1, 2, 3, ..., 15, 16, 17, 18, 19, ..., 36, 37, 38 Next
This forum is locked: you cannot post, reply to, or edit topics.   This topic is locked: you cannot edit posts or make replies. View previous topic :: View next topic
 Forum index » Advanced Topics » Puppy Projects
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.1299s ][ Queries: 12 (0.0580s) ][ GZIP on ]