UDF, as a read-write filesystem for USB flash drive
UDF, as a read-write filesystem for USB flash drive
http://tanguy.ortolo.eu/blog/article93/usb-udf
Fat32 is your regular filesystem for USB flash drive. Did you know you can switch to UDF instead of FAT32, or exfat, or ext 2,3,4? It has a lot for it.
Below, there is a post about a script called format-udf.sh, which i have tested. It works, runs fast, and the resulting UDF format is stable, enabling read-write.
Note :
You need to be careful with this script, it is so fast you can wipe and format your internal drive in a few seconds. It happened to a friend of mine, using sda, instead of sdb.
USB sticks are traditionally formatted with FAT 32, because this file system is implemented by almost every operating system and device. Unfortunately, it sucks, as it cannot use more than 2 TiB, store files larger than 2 GiB or store symbolic links for instance. In a word, it is an obsolete and deficient file system.
exFATis a stinking proprietary, secret and patented file system. There are free implementations of that shit, but it is safer to stay away from it.
UDF
Good news: there is one file system that is implemented almost everywhere as well, and which does not suffer from such limitations. UDF, the Universal Disk Format, is an ISO standard originally designed for DVDs, but it is perfectly usable for USB sticks. It also supports POSIX permissions, with one killer feature for removable media: a file can belong to no specific person or group.
So, to use it, assuming you USB stick is /dev/sdc:
$ dd if=/dev/zero of=/dev/sdc bs=1M count=1
$ mkudffs -b 512 --media-type=hd /dev/sdc
Conclusion :
After that, your USB stick will be usable for reading and writing with GNU/Linux and the other free operating systems of course, but also with current versions of Windows (read-only with the outdated version XP) and with MacOS.
Tested and working, showing on Trinitydog as LinuxUDF (but unstable, using a SD card, maybe caused by wrong block size).
Note :
you need to have udftools installed.
If you run gparted, don't expect it to show an UDF partition.
https://www.quora.com/Why-doesnt-GParte ... UDF-format
Patch available for UDF support :
https://bugzilla.gnome.org/show_bug.cgi?id=784533
comparison chart between FAT32 and UDF :
https://j0nam1el.wordpress.com/2015/02/ ... -with-udf/
Format-UDF script :
https://github.com/JElchison/format-udf
Fat32 is your regular filesystem for USB flash drive. Did you know you can switch to UDF instead of FAT32, or exfat, or ext 2,3,4? It has a lot for it.
Below, there is a post about a script called format-udf.sh, which i have tested. It works, runs fast, and the resulting UDF format is stable, enabling read-write.
Note :
You need to be careful with this script, it is so fast you can wipe and format your internal drive in a few seconds. It happened to a friend of mine, using sda, instead of sdb.
USB sticks are traditionally formatted with FAT 32, because this file system is implemented by almost every operating system and device. Unfortunately, it sucks, as it cannot use more than 2 TiB, store files larger than 2 GiB or store symbolic links for instance. In a word, it is an obsolete and deficient file system.
exFATis a stinking proprietary, secret and patented file system. There are free implementations of that shit, but it is safer to stay away from it.
UDF
Good news: there is one file system that is implemented almost everywhere as well, and which does not suffer from such limitations. UDF, the Universal Disk Format, is an ISO standard originally designed for DVDs, but it is perfectly usable for USB sticks. It also supports POSIX permissions, with one killer feature for removable media: a file can belong to no specific person or group.
So, to use it, assuming you USB stick is /dev/sdc:
$ dd if=/dev/zero of=/dev/sdc bs=1M count=1
$ mkudffs -b 512 --media-type=hd /dev/sdc
Conclusion :
After that, your USB stick will be usable for reading and writing with GNU/Linux and the other free operating systems of course, but also with current versions of Windows (read-only with the outdated version XP) and with MacOS.
Tested and working, showing on Trinitydog as LinuxUDF (but unstable, using a SD card, maybe caused by wrong block size).
Note :
you need to have udftools installed.
If you run gparted, don't expect it to show an UDF partition.
https://www.quora.com/Why-doesnt-GParte ... UDF-format
Patch available for UDF support :
https://bugzilla.gnome.org/show_bug.cgi?id=784533
comparison chart between FAT32 and UDF :
https://j0nam1el.wordpress.com/2015/02/ ... -with-udf/
Format-UDF script :
https://github.com/JElchison/format-udf
Last edited by labbe5 on Wed 16 Aug 2017, 19:53, edited 2 times in total.
What about this statement from the link you posted?
In Puppy, it does everything on a USB stick that you would do on a hard drive.UDF is, in principle, a write-once-and-read-only filesystem. It would never be suitable for use on a hard drive.
The things they do not tell you, are usually the clue to solving the problem.
When I was a kid I wanted to be older.... This is not what I expected
YaPI(any iso installer)
When I was a kid I wanted to be older.... This is not what I expected
YaPI(any iso installer)
I was under the impression that this was the best file system for USB sticks.
F2FS
https://en.wikipedia.org/wiki/F2FS
F2FS
https://en.wikipedia.org/wiki/F2FS
The things they do not tell you, are usually the clue to solving the problem.
When I was a kid I wanted to be older.... This is not what I expected
YaPI(any iso installer)
When I was a kid I wanted to be older.... This is not what I expected
YaPI(any iso installer)
Hello all.
A couple of interesting articles:
This one has a table of everything you want in a Linux file system, and
UDF comes out on top:
http://duncanlock.net/blog/2013/05/13/u ... ash-drives
This one is interesting for the benchmarks:
http://florian-berger.de/en/articles/fi ... nilfs-ext4
Ext4 is the overall winner here, but UDF is "okay". On a flash drive you'd
want to cancel ext4's history capacity to avoid too much wear and tear.
I can't find a technical URL validating this ATM, but from experience, I
noticed that UDF takes +/- 200 Mg's less overhead on the partition than
ext? filesystems. Meaning you get +/- 200 Mg's more as storage space
on your partition. This by itself may make UDF worth using on a smaller-
sized flash-drive.
The only problem with UDF is that you can't boot from it. (Need a URL for that, too...)
BFN.
A couple of interesting articles:
This one has a table of everything you want in a Linux file system, and
UDF comes out on top:
http://duncanlock.net/blog/2013/05/13/u ... ash-drives
This one is interesting for the benchmarks:
http://florian-berger.de/en/articles/fi ... nilfs-ext4
Ext4 is the overall winner here, but UDF is "okay". On a flash drive you'd
want to cancel ext4's history capacity to avoid too much wear and tear.
I can't find a technical URL validating this ATM, but from experience, I
noticed that UDF takes +/- 200 Mg's less overhead on the partition than
ext? filesystems. Meaning you get +/- 200 Mg's more as storage space
on your partition. This by itself may make UDF worth using on a smaller-
sized flash-drive.
The only problem with UDF is that you can't boot from it. (Need a URL for that, too...)
BFN.
musher0
~~~~~~~~~~
"You want it darker? We kill the flame." (L. Cohen)
~~~~~~~~~~
"You want it darker? We kill the flame." (L. Cohen)
- perdido
- Posts: 1528
- Joined: Mon 09 Dec 2013, 16:29
- Location: ¿Altair IV , Just north of Eeyore Junction.?
From the article labbe5 linked tolabbe5 wrote:
Tested and working as intended, showing on Trinitydog as LinuxUDF.
Code: Select all
It also supports POSIX permissions, with one killer feature for removable media: a file can belong to no specific person or group.
.
Re: UDF, as a read-write filesystem for USB flash drive
labbe5 wrote:....
you need to have udftools installed.
If you run gparted, don't expect it to show an UDF partition.
https://www.quora.com/Why-doesnt-GParte ... UDF-format
Patch available for UDF support :
https://bugzilla.gnome.org/show_bug.cgi?id=784533
Straight from the latest developers & release of Gparted 0.29:
........ 8 August 2017: GParted Live 0.29.0-1 Stable Release
The GParted team is pleased to announce a new stable release of GParted Live.
This release includes GParted 0.29.0, patches for libparted for FAT file system operations, and other improvements.
Items of note include:
Includes GParted 0.29.0 which adds the following enhancements since the last stable release:
Add support for UDF file system
Fix segmentation fault on disk with corrupted FAT file system
Fix snap-to-alignment of operations creating partitions
Based on the Debian Sid repository (as of 2017/Aug/08)
Linux kernel updated to 4.11.11-1
Package udftools added
Includes patched version of libparted which fixes:
check FAT32 crashes (bug 762448)
resized FAT32 not recognized by Windows (bug 759916)
This release of GParted Live has been successfully tested on VirtualBox, VMware, BIOS, UEFI, and physical computers with AMD/ATI, NVidia, and Intel graphics.
- perdido
- Posts: 1528
- Joined: Mon 09 Dec 2013, 16:29
- Location: ¿Altair IV , Just north of Eeyore Junction.?
Re: UDF, as a read-write filesystem for USB flash drive
That is interesting and coincides nicely with this thread. I can now burn the ISO to CD and have the tool available in GParted if'n I wants to try a UDF partition using my favorite partition manager,belham2 wrote:labbe5 wrote:....
you need to have udftools installed.
If you run gparted, don't expect it to show an UDF partition.
https://www.quora.com/Why-doesnt-GParte ... UDF-format
Patch available for UDF support :
https://bugzilla.gnome.org/show_bug.cgi?id=784533
Straight from the latest developers & release of Gparted 0.29:
........ 8 August 2017: GParted Live 0.29.0-1 Stable Release
The GParted team is pleased to announce a new stable release of GParted Live.
This release includes GParted 0.29.0, patches for libparted for FAT file system operations, and other improvements.
Items of note include:
Includes GParted 0.29.0 which adds the following enhancements since the last stable release:
Add support for UDF file system
Fix segmentation fault on disk with corrupted FAT file system
Fix snap-to-alignment of operations creating partitions
Based on the Debian Sid repository (as of 2017/Aug/08)
Linux kernel updated to 4.11.11-1
Package udftools added
Includes patched version of libparted which fixes:
check FAT32 crashes (bug 762448)
resized FAT32 not recognized by Windows (bug 759916)
This release of GParted Live has been successfully tested on VirtualBox, VMware, BIOS, UEFI, and physical computers with AMD/ATI, NVidia, and Intel graphics.
Link to downloads for various media -->http://gparted.org/livecd.php
That would ~probably mean that the GParted 0.29.0 is going to be updated with the patch GParted Live 0.29.0.1 has received maybe?
.
-
- Posts: 902
- Joined: Mon 22 Jun 2009, 01:36
- Location: Philadelphia, PA
Re: UDF, as a read-write filesystem for USB flash drive
Thanks.labbe5 wrote:you need to have udftools installed.
Micro Center 16GB microSD card
Slacko64 Puppy 6.9.9.9
Code: Select all
# blkid
/dev/mmcblk0p1: SEC_TYPE="msdos" UUID="239E-D6FA" TYPE="vfat" PARTUUID="2652418e-01"
/dev/mmcblk0p2: UUID="598e4371LinuxUDF" LABEL="LinuxUDF" TYPE="udf" PARTUUID="2652418e-02"
Dell E6410: BusterPup, BionicPup64, Xenial, etc
Intel DQ35JOE, Dell Vostro 430
Dell Inspiron, Acer Aspire One, EeePC 1018P
Intel DQ35JOE, Dell Vostro 430
Dell Inspiron, Acer Aspire One, EeePC 1018P
-
- Posts: 902
- Joined: Mon 22 Jun 2009, 01:36
- Location: Philadelphia, PA
Re: UDF, as a read-write filesystem for USB flash drive
I'd formatted the 16GB microSD card to UDF, and indeed MS-Windows and Puppy Linux can read and write it.labbe5 wrote:Fat32 is your regular filesystem for USB flash drive. Did you know you can switch to UDF instead of FAT32, or exfat, or ext 2,3,4? It has a lot for it.
But my Sansa Clip+ media player doesn't recognize it.
Nor does the LG Rebel phone.
At http://tanguy.ortolo.eu/blog/article93/usb-udf
the author had said, in part:
Well, since those limitations haven't caused me any problems, I guess I'll use FAT32.USB sticks are traditionally formatted with FAT 32, because this file system is implemented by almost every operating system and device. Unfortunately, it sucks, as it cannot use more than 2 TiB, store files larger than 2 GiB or store symbolic links for instance. In a word, it is an obsolete and deficient file system.
Dell E6410: BusterPup, BionicPup64, Xenial, etc
Intel DQ35JOE, Dell Vostro 430
Dell Inspiron, Acer Aspire One, EeePC 1018P
Intel DQ35JOE, Dell Vostro 430
Dell Inspiron, Acer Aspire One, EeePC 1018P
Hello all.
Five-six posts above I talked about udf's overhead on a thumb drive. The
attached pic describes the overhead taken by the udf format on an 8Mb
SD card: not quite 2 Mb's. Linux' ext2 would occupy 130 Mb's on a drive of
the same size. A difference of 128 Mb's.
I formatted the card with the mkudffs utility. Please notice the drive name:
sdd. I think udf cannot do partitions like sd1, sd2, etc.
I still haven't found a serious article comparing file systems overhead , but
there you go: empirical data.
BFN.
Five-six posts above I talked about udf's overhead on a thumb drive. The
attached pic describes the overhead taken by the udf format on an 8Mb
SD card: not quite 2 Mb's. Linux' ext2 would occupy 130 Mb's on a drive of
the same size. A difference of 128 Mb's.
I formatted the card with the mkudffs utility. Please notice the drive name:
sdd. I think udf cannot do partitions like sd1, sd2, etc.
I still haven't found a serious article comparing file systems overhead , but
there you go: empirical data.
BFN.
- Attachments
-
- Less_than_2Mb_overhead_udf.jpg
- (87.35 KiB) Downloaded 1247 times
musher0
~~~~~~~~~~
"You want it darker? We kill the flame." (L. Cohen)
~~~~~~~~~~
"You want it darker? We kill the flame." (L. Cohen)
-
- Posts: 902
- Joined: Mon 22 Jun 2009, 01:36
- Location: Philadelphia, PA
I formatted my 8GB SD card to ext4 (had read somewhere that works well).musher0 (in part) wrote:The attached pic describes the overhead taken by the udf format on an 8Mb SD card: not quite 2 Mb's. Linux' ext2 would occupy 130 Mb's on a drive of
the same size. A difference of 128 Mb's
It does seem to use up a few hundred MB, even though there were no files on it.
With no journal.
(Oh, I will go try some other tune2fs options)
Anyhow, the cell phone can't mount it.
Dell E6410: BusterPup, BionicPup64, Xenial, etc
Intel DQ35JOE, Dell Vostro 430
Dell Inspiron, Acer Aspire One, EeePC 1018P
Intel DQ35JOE, Dell Vostro 430
Dell Inspiron, Acer Aspire One, EeePC 1018P
format-UDF
Bash script to format a block device (hard drive or Flash drive) in UDF. The output is a drive that can be used for reading/writing across multiple operating system families: Windows, macOS, and Linux. This script should be capable of running in macOS or in Linux.
Installation :
format-udf is a self-contained script. Simply copy format-udf.sh to a directory of your choosing. Don't forget to make it executable:
chmod +x format-udf.sh
Prerequesites :
sudo apt-get install udftools coreutils vim-common
Note :
instead of Vim, use Geany, already installed on all Dogs.
Link here for complete information about format-udf (tested, working and stable) :
https://github.com/JElchison/format-udf
Block Size
If's extremely important that format-udf use the correct block size when formatting your drive. format-udf will attempt to detect and use the correct (logical) block size. If you know what you're doing, the format-udf -b BLOCK_SIZE option can be used to explicitly override the detected block size value.
If the wrong block size is used while formatting (i.e. one that doesn't match the logical block size of your drive), the resultant drive will likely have OS compatibility issues and suffer from non-optimal performance issues.
Maximum UDF File System Capacity
The UDF format has a maximum of 2^32 blocks. With format-udf, these blocks equate to logical blocks.
If your drive's logical block size is 512 bytes, then your maximum UDF file system capacity will be 2 TiB
If your drive's logical block size is 4096 bytes, then your maximum UDF file system capacity will be 16 TiB
Installation :
format-udf is a self-contained script. Simply copy format-udf.sh to a directory of your choosing. Don't forget to make it executable:
chmod +x format-udf.sh
Prerequesites :
sudo apt-get install udftools coreutils vim-common
Note :
instead of Vim, use Geany, already installed on all Dogs.
Link here for complete information about format-udf (tested, working and stable) :
https://github.com/JElchison/format-udf
Block Size
If's extremely important that format-udf use the correct block size when formatting your drive. format-udf will attempt to detect and use the correct (logical) block size. If you know what you're doing, the format-udf -b BLOCK_SIZE option can be used to explicitly override the detected block size value.
If the wrong block size is used while formatting (i.e. one that doesn't match the logical block size of your drive), the resultant drive will likely have OS compatibility issues and suffer from non-optimal performance issues.
Maximum UDF File System Capacity
The UDF format has a maximum of 2^32 blocks. With format-udf, these blocks equate to logical blocks.
If your drive's logical block size is 512 bytes, then your maximum UDF file system capacity will be 2 TiB
If your drive's logical block size is 4096 bytes, then your maximum UDF file system capacity will be 16 TiB