Attention! Using Puppy4.3x with ext3 is dangerous by default
- Abnormalter
- Posts: 95
- Joined: Sun 31 Jan 2010, 10:56
- Location: Russia
Attention! Using Puppy4.3x with ext3 is dangerous by default
This is dangerous because default value of current kernel now is data=writeback
And the cause as sudden loosing power, the file system does not write to hard disk very huge array of information.
This happens because somewhere between kernels 2.6.25 and 2.6.30 (in kernel 2.6.30.5 - exactly) kernel programmers to do a change of values, who uses default setting for caching the file system.
Unfortunately, the creators of the kernel do this and pass as default change it from data=ordered to data=writeback
Requires a patch to the kernel initialization (or prescription data=ordered in fstab for ext3).
( On Russian, see topic here: http://forum.puppyrus.org/index.php/topic,3609 )
And the cause as sudden loosing power, the file system does not write to hard disk very huge array of information.
This happens because somewhere between kernels 2.6.25 and 2.6.30 (in kernel 2.6.30.5 - exactly) kernel programmers to do a change of values, who uses default setting for caching the file system.
Unfortunately, the creators of the kernel do this and pass as default change it from data=ordered to data=writeback
Requires a patch to the kernel initialization (or prescription data=ordered in fstab for ext3).
( On Russian, see topic here: http://forum.puppyrus.org/index.php/topic,3609 )
- BarryK
- Puppy Master
- Posts: 9392
- Joined: Mon 09 May 2005, 09:23
- Location: Perth, Western Australia
- Contact:
Hmmm, yes, I'll consider that next time I compile the kernel. 2.6.30+ allows the default to be set to "ordered":
http://www.mail-archive.com/linuxkernel ... 01904.html
http://www.mail-archive.com/linuxkernel ... 01904.html
[url]https://bkhome.org/news/[/url]
from: http://ext4.wiki.kernel.org/index.php/E ... _tradeoffs
"Ordered mode is the mode used by most distributions, but can introduce latency problems in some workloads, especially if there is a combination of high bandwidth background writes and foreground processes calling fsync(). In worst case scenarios, the fsync() call can take 500ms to multiple seconds to return. In applications such as firefox, which called fsync() out of its main UI thread, the application can appear to have crashed since it is no longer responsive to the user's mouse or keyboard input."
"(Note, however, that Ordered Mode does not guarantee that the file will be consistent at an application level; the application must use fsync() at appropriate commit points in order to guarantee application-level consistency.)"
"Ordered mode is the mode used by most distributions, but can introduce latency problems in some workloads, especially if there is a combination of high bandwidth background writes and foreground processes calling fsync(). In worst case scenarios, the fsync() call can take 500ms to multiple seconds to return. In applications such as firefox, which called fsync() out of its main UI thread, the application can appear to have crashed since it is no longer responsive to the user's mouse or keyboard input."
"(Note, however, that Ordered Mode does not guarantee that the file will be consistent at an application level; the application must use fsync() at appropriate commit points in order to guarantee application-level consistency.)"
From link posted by Barry above:
If there is truly a problem with unsaved data loss when power is interrupted, this could be an interim fix, while waiting on an updated kernel.Filesystems can be forced to use a specific data journalling mode by specifying a mount option on the command line, or in /etc/fstab, or by using the "tune2fs -o journal_data_ordered" or "tune2fs -o journal_data_writeback" to specify a default mount option in the filesystem superblock.
Re: Using Puppy4.3x with ext3 is dangerous by default
while i know none of the technical side. i have been using 431 for hours daily since it's creation and my laptop undergoes many battery empty related power failures and my 431 frugal install on internal SSD at sda2 with ext3 file system has NEVER given me ANY problems of any sort when the power out happens. sorry.Abnormalter wrote:This is dangerous because default value of current kernel now is data=writeback
And the cause as sudden loosing power, the file system does not write to hard disk very huge array of information.
This happens because somewhere between kernels 2.6.25 and 2.6.30 (in kernel 2.6.30.5 - exactly) kernel programmers to do a change of values, who uses default setting for caching the file system.
Unfortunately, the creators of the kernel do this and pass as default change it from data=ordered to data=writeback
Requires a patch to the kernel initialization (or prescription data=ordered in fstab for ext3).
( On Russian, see topic here: http://forum.puppyrus.org/index.php/topic,3609 )
i plug in the cable restart and my opera browser restarts exactly where it left off. no problem at all, never noticed any loss of data elsewhere either.
- Abnormalter
- Posts: 95
- Joined: Sun 31 Jan 2010, 10:56
- Location: Russia
Dear Barry, thank you for your understanding!BarryK wrote:Hmmm, yes, I'll consider that next time I compile the kernel. 2.6.30+ allows the default to be set to "ordered":
http://www.mail-archive.com/linuxkernel ... 01904.html
I think any Linux, and even more so PuppyLinux, *should* be reliable OS, but dangerous option and a features, introduced by default, -- this is a compromise of idea the robust operating systems...
(N.B. Ever since MS-DOS, at last 1980th, and the initial period of Linux, at 90th, so in BSD, who have very safetly and reliabilitly structure, was taken as a good form is *not use* dangerous values as the default, but giving user the ability to include this as an option...)
IMHO. I believe that all users should know these basics, the more the creators of software (such as who wrote the kernel) should not neglect the experience accumulated over decades... But now more than ever have the opportunity to introduce gui-configurator tools, that would all users to quickly and visually configure own copy of the OS under their real need and demand. (As soon as I have free time, I'll try to write the appropriate gui-configurator for available among others in the Puppy.)
I apologize for some sententious of my message...
Sorry for the delay in response.
- Abnormalter
- Posts: 95
- Joined: Sun 31 Jan 2010, 10:56
- Location: Russia
Re: Using Puppy4.3x with ext3 is dangerous by default
Firstly, my usage statistics of the Puppy (including as a small server) by configuration under the 2.6.26.16 kernel is 24 hours at day, 7 days at week, and 365 days at year - and I use it without any problems. (Similarly, and under other clones of Linux.) Note: due to smallness of my network, I did not use any UPS...aarf wrote: while i know none of the technical side. i have been using 431 for hours daily since it's creation and my laptop undergoes many battery empty related power failures and my 431 frugal install on internal SSD at sda2 with ext3 file system has NEVER given me ANY problems of any sort when the power out happens. sorry.
i plug in the cable restart and my opera browser restarts exactly where it left off. no problem at all, never noticed any loss of data elsewhere either.
However, since the installation in January 2010 on my computer puppy-431 (who under kernel 2.6.30.5), and until I described the cause of data loss, it took no more than 2 weeks. But do comparison it please with some annual maintenance by use the puppy-42x without any problems. That is fact for me overriding.
Secondly, as a factually rule, all laptops have a more reliable "at birth" to power loss causes, it is dependent not on the degree of efficiency own battery. Unlike a typical computer desktop (without UPS, of course).
Thirdly, all described in the topic for me is the case only for (a) use the ext3 FS on HDD; (b) because prescribed as default values in the corresponding HDD' fstab, alike:
Puppy-431 (Linux kernel version: 2.6.30.5):
/dev/sda1 /mnt/sda1 ext3 defaults 0 0
/dev/sda2 /mnt/home ext3 defaults 0 0
/dev/sda4 /mnt/sda4 ext3 defaults 0 0
It is the conditions that create the preconditions for the incident, who described above.
Regards!
- Dingo
- Posts: 1437
- Joined: Tue 11 Dec 2007, 17:48
- Location: somewhere at the end of rainbow...
- Contact:
using reiserfs may be a solution? (waiting for fix in kernel)
since in puppy we can mount via pmount I think also pmount needs a tweak
since in puppy we can mount via pmount I think also pmount needs a tweak
replace .co.cc with .info to get access to stuff I posted in forum
dropbox 2GB free
OpenOffice for Puppy Linux
dropbox 2GB free
OpenOffice for Puppy Linux
- Abnormalter
- Posts: 95
- Joined: Sun 31 Jan 2010, 10:56
- Location: Russia
"However, the problem with using a default of Writeback Mode is that after a system crash or a power failure, files that were written right before the system went down could contain previously written data or other garbage... Using Writeback Mode extends the time from when a file is written to when it is pushed out to disk to 30 seconds. This can be surprising for some users; however, it should be noted that such problems can still be an issue with Ordered Mode..." (http://www.mail-archive.com/linuxkernel ... 01904.html)
-- It is true!
Need to permit an user's configuration - as prefer it, but do not as default...
-- It is true!
Need to permit an user's configuration - as prefer it, but do not as default...
- Abnormalter
- Posts: 95
- Joined: Sun 31 Jan 2010, 10:56
- Location: Russia
- Abnormalter
- Posts: 95
- Joined: Sun 31 Jan 2010, 10:56
- Location: Russia
Additional mind
By little more thinking about written on the link, I believe that all you see above: "Ordered mode is the mode used by most distributions, but can introduce latency problems in some workloads, especially if there is a combination of high bandwidth background writes and foreground processes calling fsync(). In worst case scenarios, the fsync() call can take 500ms to multiple seconds to return. In applications such as Firefox..." - it is has not relation to environment reality of PuppyLinux! This is true because any operational data, including the cache and personal settings by Firefox (as example), stored in the virtual filesystem (and worked into very fast DRAM). And periodically only (by default every 30 seconds) this data can be dropped into user's hard disk, -- to the save file pup_save-xxxx.2fs
UPD: for others HDD or partitions it have not corresponding too...
UPD: for others HDD or partitions it have not corresponding too...
-
- Posts: 282
- Joined: Wed 16 Dec 2009, 21:38
- Location: Earth
apparently the file system inside a frugal install pup_save is ext2 and when this pupsave is located on a ext3 partition there doesn't seem to be a problem in my experience. many people have reported problems with full installs when power failure occurs perhaps this ext3 thing was their problem. i still see the solution as frugal install.
- Abnormalter
- Posts: 95
- Joined: Sun 31 Jan 2010, 10:56
- Location: Russia
Of course, but need specialized tools for itPANZERKOPF wrote:No, use "data=ordered" option in fstab . Also this option can be used directly.Dingo wrote:using reiserfs may be a solution? (waiting for fix in kernel)
For example: mount /dev/sda2 /mnt/sda2 -o data=ordered
I create it now, please see it here: http://murga-linux.com/puppy/viewtopic.php?t=52109
Last edited by Abnormalter on Tue 02 Feb 2010, 20:03, edited 1 time in total.
- Abnormalter
- Posts: 95
- Joined: Sun 31 Jan 2010, 10:56
- Location: Russia
No, for me cause it is not. I don't use full install, but I use frugal permanently (for many computers). This problem is enviroment changing on 2.6.30 kernel default option of mount... I was fix it now.aarf wrote:apparently the file system inside a frugal install pup_save is ext2 and when this pupsave is located on a ext3 partition there doesn't seem to be a problem in my experience. many people have reported problems with full installs when power failure occurs perhaps this ext3 thing was their problem. i still see the solution as frugal install.
Test the Tools, please: "Data Loss Defender Tools" - http://murga-linux.com/puppy/viewtopic.php?t=52109
- Abnormalter
- Posts: 95
- Joined: Sun 31 Jan 2010, 10:56
- Location: Russia
Utility for tuning mount mode
I wrote it. It's finally release...
See updated attach on http://murga-linux.com/puppy/viewtopic.php?t=52109
Package: dldt.RC1.pet
Test it please, guys!
See updated attach on http://murga-linux.com/puppy/viewtopic.php?t=52109
Package: dldt.RC1.pet
Test it please, guys!
Data loss twice
I'm just a newbie who started playing with puppy two weeks ago and I ran into sad story. I already lost data twice with 2.6.30.5 in two different computers and different scenarios:
In a Dell L400 PIII while transferring 3 text files from a USB Stick to the internal hard drive. I have an old battery and unit went off while transferring data. After booting again, no data in the usb stick nor internal hard drive. I'd moved some other files before successfully.
Today in a EEEPC 701 4G, I wanted to move some other text files from the internal hard drive to a SD Card (FAT). The SD card had a simple folder structure. So I moved the files, unmount the SD memory and shutdown the computer right after (regular shutdown). When I opened the SD card in a regular windoze box, the card was clean. No trace of any file or folder, new or old, put it back in the EEEPC check the memory with the terminal, clean as mod. I moved lots of files before same way, but kept the computer running for a while after.
My files are gone and nothing I could do, well yes retype my files. . But I want to fix my pupps, I really like it this distro. I don't know if I should be doing something different or I need puppy for dummies. Could this be related to this thread?
In a Dell L400 PIII while transferring 3 text files from a USB Stick to the internal hard drive. I have an old battery and unit went off while transferring data. After booting again, no data in the usb stick nor internal hard drive. I'd moved some other files before successfully.
Today in a EEEPC 701 4G, I wanted to move some other text files from the internal hard drive to a SD Card (FAT). The SD card had a simple folder structure. So I moved the files, unmount the SD memory and shutdown the computer right after (regular shutdown). When I opened the SD card in a regular windoze box, the card was clean. No trace of any file or folder, new or old, put it back in the EEEPC check the memory with the terminal, clean as mod. I moved lots of files before same way, but kept the computer running for a while after.
My files are gone and nothing I could do, well yes retype my files. . But I want to fix my pupps, I really like it this distro. I don't know if I should be doing something different or I need puppy for dummies. Could this be related to this thread?
Puppy 431 frugal kernel 2.6.30.5 3 gigs of memory on PC.
Installed to 40gig ext2 formatted partition along with other Puppy versions also frugal.
I mainly run Puppy 431 SCSI with above kernel and have lately been getting severe file system corruption.
It still boots to desktop, but files come up missing that I did not delete and Puppy Browser quit working on not being able to fine support files.
Also my Floppy Formater program failed to do a low level format of a floppy with an error of "sg* not found".
So it looks like I am off to do a complete reinstall of Puppy complete with a new pupsave file.
I backed up my .mozilla directory and also backed up "user installed packages" list in preparation.
What corrupted the Pupsave fie?
Probably unclean umounts on shutdown.
Who knows for sure.
But it is sure a pain in the A** to have to restore all programs and utilities I had installed.
All my Pet and *.tar.gz files are on another partition so at least I do not have to download them again.
Before I do this, I need a second opinion as to if I should stay with 431 or move to Dpup or back a few Puppy versions.
Installed to 40gig ext2 formatted partition along with other Puppy versions also frugal.
I mainly run Puppy 431 SCSI with above kernel and have lately been getting severe file system corruption.
It still boots to desktop, but files come up missing that I did not delete and Puppy Browser quit working on not being able to fine support files.
Also my Floppy Formater program failed to do a low level format of a floppy with an error of "sg* not found".
So it looks like I am off to do a complete reinstall of Puppy complete with a new pupsave file.
I backed up my .mozilla directory and also backed up "user installed packages" list in preparation.
What corrupted the Pupsave fie?
Probably unclean umounts on shutdown.
Who knows for sure.
But it is sure a pain in the A** to have to restore all programs and utilities I had installed.
All my Pet and *.tar.gz files are on another partition so at least I do not have to download them again.
Before I do this, I need a second opinion as to if I should stay with 431 or move to Dpup or back a few Puppy versions.
Barry, the problems with the shutdown (plus a fix) are mentioned here
http://www.murga-linux.com/puppy/viewtopic.php?t=47668
http://www.murga-linux.com/puppy/viewtopic.php?t=39132
http://www.murga-linux.com/puppy/viewtopic.php?t=50304
Over half of the puppy 4 installs I have "in the field" have suffered from this problem so far
http://www.murga-linux.com/puppy/viewtopic.php?t=47668
http://www.murga-linux.com/puppy/viewtopic.php?t=39132
http://www.murga-linux.com/puppy/viewtopic.php?t=50304
Over half of the puppy 4 installs I have "in the field" have suffered from this problem so far
Puppy Linux's [url=http://www.murga-linux.com/puppy/viewtopic.php?p=296352#296352]Mission[/url]
Sorry, my server is down atm!
Sorry, my server is down atm!