Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Tue 20 Nov 2018, 16:02
All times are UTC - 4
 Forum index » House Training » Beginners Help ( Start Here)
Defragment Linux file systems
Moderators: Flash, Ian, JohnMurga
Post new topic   Reply to topic View previous topic :: View next topic
Page 1 of 1 [11 Posts]  
Author Message
artsown

Joined: 12 Sep 2012
Posts: 288

PostPosted: Mon 08 Oct 2018, 10:41    Post subject:  Defragment Linux file systems
Subject description: You might improve performance
 

On a hard drive, I had occasion to move three frugally installed puppies
from one ext2 partition to another (empty) partition. I was surprised to
experience a significant performance improvement when using the
moved pups. I thought "hmm, just like defragging Windoze".

It turns out that I had indeed defragged, and stumbled on the
recommended method of doing so (in a way). Googling "linux
defragment" turned up some good articles and explanations. Simply
moving the files out of a partition and back again will do the trick.

To check that everything is cool on sda2 (for example), in a terminal type:
fsck /dev/sda2 -f
(sda2 cannot be mounted for this check)

This will provide you with info on "non-contiguous inodes" which is linux-
speak for "fragmentation" Smile I don't know how much fragmentation there
was before I moved pups to a different partition. I don't know how much
will likely lead to significant performance increase after defrag.

Art
Back to top
View user's profile Send private message 
musher0

Joined: 04 Jan 2009
Posts: 13003
Location: Gatineau (Qc), Canada

PostPosted: Mon 08 Oct 2018, 13:07    Post subject:  

I'm glad you brought the subject back, artsown!

Fragmentation is something that occurs in magnetized hard drives, whatever
the OS. It is a physical thing, as in "physics".

On Linux, 10 % hard disk fragmentation is acceptable, although ever seen,
because the Linux file systems ("ext" or others) have incorporated ways NOT
to let fragmentation happen.

You will find my script "automverif" for hard drives, here. ("Automverif"
stands for automatic verification.) Please use the latest version if you do.

If I may... When you say:
Quote:
Simply moving the files out of a partition and back again will do the trick.
I beg to differ! If you only do that, the files will go back where they
were, and the fragmentation rate will not change. You have to copy a rather
BIG FILE to the original disk before you recopy your big files for some
defragmentation to happen. This BIG FILE can be full of zeroes and you may
even erase it later, but the disk table used by the file system has to change.

The absolute defragger remains, as you guessed in your 1st paragraph:
-- copy all files from partition A to partition B.
-- reformat partition A
-- recopy all files from partition B to partition A.

BFN.

_________________
musher0
~~~~~~~~~~
Je suis né pour aimer et non pas pour haïr. (Sophocle) /
I was born to love and not to hate. (Sophocles)
Back to top
View user's profile Send private message 
artsown

Joined: 12 Sep 2012
Posts: 288

PostPosted: Mon 08 Oct 2018, 15:23    Post subject:  

Hi musher0,

As I understand from the articles I've read, the Linux ext2, ext3 and ext4
(at least) distribute files over the entire partition, keeping them as
separated as possible, so when they grow they can grow without
fragmenting them. If this is true, then it makes sense to me that using
very large partitions having only a few small pups in them should
minimize fragmentation.

Now, it so happens that I have no use for 250 gig hard drive space. So
I created three "small" partitions for various purposes, plus one partition
that's 179 gig. My three little pups drown in that huge space.

So my current experiment with this old pentium D class machine is to
see if any fragmentation of significance of the pup files eventually
occurs. Nutty, huh? Smile

Do you have reason to think it will? If so, why?

Art
Back to top
View user's profile Send private message 
musher0

Joined: 04 Jan 2009
Posts: 13003
Location: Gatineau (Qc), Canada

PostPosted: Mon 08 Oct 2018, 16:55    Post subject:  

Hello, artsown.

I'm not saying you need my script! Wink Use it or not -- as you wish.

What I'm saying is --
As hard as we try to limit defragmentation, it is a law of nature concerning
magnetized stuff: any large file stored as ones (1) and zeroes (0) on a hard
drive AND used often will eventually become fragmented.

I think you have a logical approach to it, though.

For the record, Linux does have a FILE defrag utility, written in Python:
https://github.com/ThomasCX/defragfs
One advantage is that the partition the file is on can stay mounted.

But be warned, you will have to repeat the process many times before your
big file finally congeals into one single fragment.

Finally, my preferred one-liner for doing an fsck of partitions is
Code:
fsck."$2" -p -f -v /dev/"$1"
where $2 is the ext type and $1 is the partition.
The "-p" parameter sets automatic correction on.

Best of luck with your approach.

BFN.

_________________
musher0
~~~~~~~~~~
Je suis né pour aimer et non pas pour haïr. (Sophocle) /
I was born to love and not to hate. (Sophocles)
Back to top
View user's profile Send private message 
bigpup


Joined: 11 Oct 2009
Posts: 11286
Location: Charleston S.C. USA

PostPosted: Tue 09 Oct 2018, 16:15    Post subject:  

This is only about hard drives.

I tried to do something about fragmentation in Linux formats and never found any program that worked that good.
As you say. Seems you had to run the program many times to really see any improvement.

To eliminate the possibility of fragmenting. (ext3, ext4 formats)
Make sure you always have some free space in a Linux formatted partition.
That gives it room to self adjust the location of files, as needed, to stop fragmentation.

I just checked a partition formatted ext 4 that I have used for a long time and has a lot of stuff on it that has been deleted, added, moved around, etc.........

The check results:
Total/best extents 37004/35921
Average size per extent 990 KB
Fragmentation score 0
[0-30 no problem: 31-55 a little bit fragmented: 56- needs defrag]
This device (/dev/sdc7) does not need defragmentation.

This is already in most Puppies, but it only works on ext4 formats.
The partition has to be mounted to check.
Code:
e4defrag -c /dev/partition ID.

Example:
Code:
e4defrag -c /dev/sdc3
to check partition sdc3.

e4defrag (-v) option will run the DE-fraging process.

1

_________________
I have found, in trying to help people, that the things they do not tell you, are usually the clue to solving the problem.
When I was a kid I wanted to be older.... This is not what I expected Shocked
Back to top
View user's profile Send private message 
artsown

Joined: 12 Sep 2012
Posts: 288

PostPosted: Tue 09 Oct 2018, 19:54    Post subject:  

@bigpup
There's a issue with ext4 when using Fatdog's humungus initrd to get
fast booting. Ext4 does the opposite, causing bootup to be much
slower than when using ext2 or ext3. Jamesbond and Kirk have
talked about this.Kirk prefers ext3.

To me, this is unfortunate since e4defrag looks attractive, and
fatdog 721 is one of my favorite pups. Right now, I've split my little
kennel so I can use ext4 for xenialpup64 and tahrpup64, and ext3 (now)
for fatdog. Just a bit awkward, but no big deal Smile

Art
Back to top
View user's profile Send private message 
bigpup


Joined: 11 Oct 2009
Posts: 11286
Location: Charleston S.C. USA

PostPosted: Tue 09 Oct 2018, 20:40    Post subject:  

Wonder what
Code:
e4defrag -c
would show for that ext4 partition, if Fatdog was installed to it. Idea
_________________
I have found, in trying to help people, that the things they do not tell you, are usually the clue to solving the problem.
When I was a kid I wanted to be older.... This is not what I expected Shocked
Back to top
View user's profile Send private message 
p310don

Joined: 19 May 2009
Posts: 1258
Location: Brisbane, Australia

PostPosted: Tue 09 Oct 2018, 22:52    Post subject:  

I once read a method of defragging linux that apparently works. I haven't used it.

Basically compress the entire hard drive into a tar.gz, wipe (format the partition) and then uncompress the tar.gz file onto the fresh partition.
Back to top
View user's profile Send private message 
tallboy


Joined: 21 Sep 2010
Posts: 1062
Location: Oslo, Norway

PostPosted: Wed 10 Oct 2018, 23:09    Post subject:  

fsck was default in my old Debian distros, I think 30 bootups was the limit before you got the message at bootup that the HDD would be checked first.
_________________
True freedom is a live Puppy on a multisession CD/DVD.
Back to top
View user's profile Send private message 
s243a

Joined: 02 Sep 2014
Posts: 1290

PostPosted: Wed 10 Oct 2018, 23:39    Post subject:  

artsown wrote:
@bigpup
There's a issue with ext4 when using Fatdog's humungus initrd to get
fast booting. Ext4 does the opposite, causing bootup to be much
slower than when using ext2 or ext3. Jamesbond and Kirk have
talked about this.Kirk prefers ext3.

To me, this is unfortunate since e4defrag looks attractive, and
fatdog 721 is one of my favorite pups. Right now, I've split my little
kennel so I can use ext4 for xenialpup64 and tahrpup64, and ext3 (now)
for fatdog. Just a bit awkward, but no big deal Smile

Art


Sorry to sidetrack things but what was the reason for using the large initrd rather than using a "base sfs" file like a typically puppy?

P.S. Why not put the fatdog64 save file on the ext4 partition and put the initrd file on a different partition? Heck, you could put the initrd on a fat partition if you wanted (I think).
Back to top
View user's profile Send private message 
artsown

Joined: 12 Sep 2012
Posts: 288

PostPosted: Thu 11 Oct 2018, 07:06    Post subject:  

@ s43a
The humungous initrd can speed up bootup times, as I had said.
In my experiences, the speedups have been quite impressive.

Your idea of placing the fatdog save file only on a ext4 partition
makes sense. I implemented this and so far so good Smile I'm considering
the idea of placing only save files/folders in a partition that's set aside
for fragmentation checks and possible defrags.

Thanks for the idea!

Art
Back to top
View user's profile Send private message 
Display posts from previous:   Sort by:   
Page 1 of 1 [11 Posts]  
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » House Training » Beginners Help ( Start Here)
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.0665s ][ Queries: 11 (0.0131s) ][ GZIP on ]