Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Wed 23 Aug 2017, 06:15
All times are UTC - 4
 Forum index » Advanced Topics » Additional Software (PETs, n' stuff) » Utilities
UDF, as a read-write filesystem for USB flash drive
Post new topic   Reply to topic View previous topic :: View next topic
Page 1 of 1 [15 Posts]  
Author Message
labbe5

Joined: 13 Nov 2013
Posts: 862
Location: Canada

PostPosted: Tue 08 Aug 2017, 15:06    Post subject:  UDF, as a read-write filesystem for USB flash drive
Subject description: Universal Disk Format as a replacement for FAT32 and exFAT as a universal read-write filesystem
 

http://tanguy.ortolo.eu/blog/article93/usb-udf

Fat32 is your regular filesystem for USB flash drive. Did you know you can switch to UDF instead of FAT32, or exfat, or ext 2,3,4? It has a lot for it.

Below, there is a post about a script called format-udf.sh, which i have tested. It works, runs fast, and the resulting UDF format is stable, enabling read-write.

Note :
You need to be careful with this script, it is so fast you can wipe and format your internal drive in a few seconds. It happened to a friend of mine, using sda, instead of sdb.

USB sticks are traditionally formatted with FAT 32, because this file system is implemented by almost every operating system and device. Unfortunately, it sucks, as it cannot use more than 2 TiB, store files larger than 2 GiB or store symbolic links for instance. In a word, it is an obsolete and deficient file system.

exFATis a stinking proprietary, secret and patented file system. There are free implementations of that shit, but it is safer to stay away from it.

UDF
Good news: there is one file system that is implemented almost everywhere as well, and which does not suffer from such limitations. UDF, the Universal Disk Format, is an ISO standard originally designed for DVDs, but it is perfectly usable for USB sticks. It also supports POSIX permissions, with one killer feature for removable media: a file can belong to no specific person or group.

So, to use it, assuming you USB stick is /dev/sdc:

$ dd if=/dev/zero of=/dev/sdc bs=1M count=1
$ mkudffs -b 512 --media-type=hd /dev/sdc

Conclusion :
After that, your USB stick will be usable for reading and writing with GNU/Linux and the other free operating systems of course, but also with current versions of Windows (read-only with the outdated version XP) and with MacOS.

Tested and working, showing on Trinitydog as LinuxUDF (but unstable, using a SD card, maybe caused by wrong block size).

Note :
you need to have udftools installed.
If you run gparted, don't expect it to show an UDF partition.
https://www.quora.com/Why-doesnt-GParted-support-the-UDF-format
Patch available for UDF support :
https://bugzilla.gnome.org/show_bug.cgi?id=784533
comparison chart between FAT32 and UDF :
https://j0nam1el.wordpress.com/2015/02/20/sharing-a-hardflash-drive-across-windows-os-x-and-linux-with-udf/
Format-UDF script :
https://github.com/JElchison/format-udf

Last edited by labbe5 on Wed 16 Aug 2017, 15:53; edited 2 times in total
Back to top
View user's profile Send private message 
bigpup


Joined: 11 Oct 2009
Posts: 9091
Location: Charleston S.C. USA

PostPosted: Tue 08 Aug 2017, 18:27    Post subject:  

What about this statement from the link you posted?

Quote:
UDF is, in principle, a write-once-and-read-only filesystem. It would never be suitable for use on a hard drive.

In Puppy, it does everything on a USB stick that you would do on a hard drive.

_________________
I have found, in trying to help people, that the things they do not tell you, are usually the clue to solving the problem.
When I was a kid I wanted to be older.... This is not what I expected Shocked
Back to top
View user's profile Send private message 
bigpup


Joined: 11 Oct 2009
Posts: 9091
Location: Charleston S.C. USA

PostPosted: Tue 08 Aug 2017, 18:39    Post subject:  

I was under the impression that this was the best file system for USB sticks.

F2FS
https://en.wikipedia.org/wiki/F2FS

_________________
I have found, in trying to help people, that the things they do not tell you, are usually the clue to solving the problem.
When I was a kid I wanted to be older.... This is not what I expected Shocked
Back to top
View user's profile Send private message 
musher0


Joined: 04 Jan 2009
Posts: 10664
Location: Gatineau (Qc), Canada

PostPosted: Tue 08 Aug 2017, 23:25    Post subject:  

Hello all.

A couple of interesting articles:

This one has a table of everything you want in a Linux file system, and
UDF comes out on top:
http://duncanlock.net/blog/2013/05/13/using-udf-as-an-improved-filesystem-for-usb-flash-drives

This one is interesting for the benchmarks:
http://florian-berger.de/en/articles/file-system-benchmarks-udf-btrfs-nilfs-ext4
Ext4 is the overall winner here, but UDF is "okay". On a flash drive you'd
want to cancel ext4's history capacity to avoid too much wear and tear.

I can't find a technical URL validating this ATM, but from experience, I
noticed that UDF takes +/- 200 Mg's less overhead on the partition than
ext? filesystems. Meaning you get +/- 200 Mg's more as storage space
on your partition. This by itself may make UDF worth using on a smaller-
sized flash-drive.

The only problem with UDF is that you can't boot from it. (Need a URL for that, too...)

BFN.

_________________
musher0
~~~~~~~~~~
« Un insensé sur le trône n'est qu'un singe sur le haut d'un toit. » / "A madman
on the throne is just a monkey on top of a roof." (Bernard de Clervaux)
Back to top
View user's profile Send private message 
perdido


Joined: 09 Dec 2013
Posts: 587
Location: Altair IV , Just north of Eeyore Junction.

PostPosted: Wed 09 Aug 2017, 04:01    Post subject:  

labbe5 wrote:


Tested and working as intended, showing on Trinitydog as LinuxUDF.

From the article labbe5 linked to
Code:
It also supports POSIX permissions, with one killer feature for removable media: a file can belong to no specific person or group.

Hey labbe5, have you checked if file permissions for user and group are available on files saved on your LinuxUDF partition?

.

_________________
.
Back to top
View user's profile Send private message 
belham2

Joined: 15 Aug 2016
Posts: 911

PostPosted: Wed 09 Aug 2017, 12:15    Post subject: Re: UDF, as a read-write filesystem for USB flash drive
Subject description: Universal Disk Format as a replacement for FAT32 and exFAT as a universal read-write filesystem
 

labbe5 wrote:
....
you need to have udftools installed.
If you run gparted, don't expect it to show an UDF partition.
https://www.quora.com/Why-doesnt-GParted-support-the-UDF-format
Patch available for UDF support :
https://bugzilla.gnome.org/show_bug.cgi?id=784533



Straight from the latest developers & release of Gparted 0.29:

........ 8 August 2017: GParted Live 0.29.0-1 Stable Release

The GParted team is pleased to announce a new stable release of GParted Live.

This release includes GParted 0.29.0, patches for libparted for FAT file system operations, and other improvements.

Items of note include:

Includes GParted 0.29.0 which adds the following enhancements since the last stable release:
Add support for UDF file system
Fix segmentation fault on disk with corrupted FAT file system
Fix snap-to-alignment of operations creating partitions
Based on the Debian Sid repository (as of 2017/Aug/0Cool
Linux kernel updated to 4.11.11-1
Package udftools added
Includes patched version of libparted which fixes:
check FAT32 crashes (bug 762448)
resized FAT32 not recognized by Windows (bug 759916)

This release of GParted Live has been successfully tested on VirtualBox, VMware, BIOS, UEFI, and physical computers with AMD/ATI, NVidia, and Intel graphics.
Back to top
View user's profile Send private message 
perdido


Joined: 09 Dec 2013
Posts: 587
Location: Altair IV , Just north of Eeyore Junction.

PostPosted: Wed 09 Aug 2017, 16:36    Post subject: Re: UDF, as a read-write filesystem for USB flash drive
Subject description: Universal Disk Format as a replacement for FAT32 and exFAT as a universal read-write filesystem
 

belham2 wrote:
labbe5 wrote:
....
you need to have udftools installed.
If you run gparted, don't expect it to show an UDF partition.
https://www.quora.com/Why-doesnt-GParted-support-the-UDF-format
Patch available for UDF support :
https://bugzilla.gnome.org/show_bug.cgi?id=784533



Straight from the latest developers & release of Gparted 0.29:

........ 8 August 2017: GParted Live 0.29.0-1 Stable Release

The GParted team is pleased to announce a new stable release of GParted Live.

This release includes GParted 0.29.0, patches for libparted for FAT file system operations, and other improvements.

Items of note include:

Includes GParted 0.29.0 which adds the following enhancements since the last stable release:
Add support for UDF file system
Fix segmentation fault on disk with corrupted FAT file system
Fix snap-to-alignment of operations creating partitions
Based on the Debian Sid repository (as of 2017/Aug/0Cool
Linux kernel updated to 4.11.11-1
Package udftools added
Includes patched version of libparted which fixes:
check FAT32 crashes (bug 762448)
resized FAT32 not recognized by Windows (bug 759916)

This release of GParted Live has been successfully tested on VirtualBox, VMware, BIOS, UEFI, and physical computers with AMD/ATI, NVidia, and Intel graphics.


That is interesting and coincides nicely with this thread. I can now burn the ISO to CD and have the tool available in GParted if'n I wants to try a UDF partition using my favorite partition manager, Smile
Link to downloads for various media -->http://gparted.org/livecd.php

That would ~probably mean that the GParted 0.29.0 is going to be updated with the patch GParted Live 0.29.0.1 has received maybe?

.

_________________
.
Back to top
View user's profile Send private message 
sheldonisaac

Joined: 21 Jun 2009
Posts: 670
Location: Philadelphia, PA

PostPosted: Fri 11 Aug 2017, 20:28    Post subject: Re: UDF, as a read-write filesystem for USB flash drive
Subject description: Universal Disk Format as a replacement for FAT32 and exFAT as a universal read-write filesystem
 

labbe5 wrote:
you need to have udftools installed.

Thanks.

Micro Center 16GB microSD card
Slacko64 Puppy 6.9.9.9
Code:

# blkid
/dev/mmcblk0p1: SEC_TYPE="msdos" UUID="239E-D6FA" TYPE="vfat" PARTUUID="2652418e-01"
/dev/mmcblk0p2: UUID="598e4371LinuxUDF" LABEL="LinuxUDF" TYPE="udf" PARTUUID="2652418e-02"

I don't understand some of that blkid info.

_________________
Dell E6410: Slim Slacko 6, LuPu Super 2, Tahr
Dell D610: Windows XP, Puppy Linux 5.2
Intel D865GBF: Tiny Windows 7, Puppy Linux 5.2
Acer Aspire One: Windows XP, Puppy Linux 5.2
Back to top
View user's profile Send private message 
musher0


Joined: 04 Jan 2009
Posts: 10664
Location: Gatineau (Qc), Canada

PostPosted: Fri 11 Aug 2017, 23:02    Post subject:  

Hi Sheldon.

Me neither ! Laughing Don't worry about it! Wink

BFN.

_________________
musher0
~~~~~~~~~~
« Un insensé sur le trône n'est qu'un singe sur le haut d'un toit. » / "A madman
on the throne is just a monkey on top of a roof." (Bernard de Clervaux)
Back to top
View user's profile Send private message 
sheldonisaac

Joined: 21 Jun 2009
Posts: 670
Location: Philadelphia, PA

PostPosted: Sat 12 Aug 2017, 16:53    Post subject: Re: UDF, as a read-write filesystem for USB flash drive
Subject description: Universal Disk Format as a replacement for FAT32 and exFAT as a universal read-write filesystem
 

labbe5 wrote:
Fat32 is your regular filesystem for USB flash drive. Did you know you can switch to UDF instead of FAT32, or exfat, or ext 2,3,4? It has a lot for it.

I'd formatted the 16GB microSD card to UDF, and indeed MS-Windows and Puppy Linux can read and write it.
But my Sansa Clip+ media player doesn't recognize it.
Nor does the LG Rebel phone.
At http://tanguy.ortolo.eu/blog/article93/usb-udf
the author had said, in part:
Quote:
USB sticks are traditionally formatted with FAT 32, because this file system is implemented by almost every operating system and device. Unfortunately, it sucks, as it cannot use more than 2 TiB, store files larger than 2 GiB or store symbolic links for instance. In a word, it is an obsolete and deficient file system.
Well, since those limitations haven't caused me any problems, I guess I'll use FAT32.
_________________
Dell E6410: Slim Slacko 6, LuPu Super 2, Tahr
Dell D610: Windows XP, Puppy Linux 5.2
Intel D865GBF: Tiny Windows 7, Puppy Linux 5.2
Acer Aspire One: Windows XP, Puppy Linux 5.2
Back to top
View user's profile Send private message 
musher0


Joined: 04 Jan 2009
Posts: 10664
Location: Gatineau (Qc), Canada

PostPosted: Sat 12 Aug 2017, 21:18    Post subject:  

@Sheldon:

One can count on popular apps to use ancient technology! Twisted Evil
It's beyond logic, but the boss is a conservative! Wink

BFN.

_________________
musher0
~~~~~~~~~~
« Un insensé sur le trône n'est qu'un singe sur le haut d'un toit. » / "A madman
on the throne is just a monkey on top of a roof." (Bernard de Clervaux)
Back to top
View user's profile Send private message 
musher0


Joined: 04 Jan 2009
Posts: 10664
Location: Gatineau (Qc), Canada

PostPosted: Sun 13 Aug 2017, 08:42    Post subject:  

Hello all.

Five-six posts above I talked about udf's overhead on a thumb drive. The
attached pic describes the overhead taken by the udf format on an 8Mb
SD card: not quite 2 Mb's. Linux' ext2 would occupy 130 Mb's on a drive of
the same size. A difference of 128 Mb's.

I formatted the card with the mkudffs utility. Please notice the drive name:
sdd. I think udf cannot do partitions like sd1, sd2, etc.

I still haven't found a serious article comparing file systems overhead , but
there you go: empirical data.

BFN.
Less_than_2Mb_overhead_udf.jpg
 Description   
 Filesize   87.35 KB
 Viewed   85 Time(s)

Less_than_2Mb_overhead_udf.jpg


_________________
musher0
~~~~~~~~~~
« Un insensé sur le trône n'est qu'un singe sur le haut d'un toit. » / "A madman
on the throne is just a monkey on top of a roof." (Bernard de Clervaux)
Back to top
View user's profile Send private message 
musher0


Joined: 04 Jan 2009
Posts: 10664
Location: Gatineau (Qc), Canada

PostPosted: Sun 13 Aug 2017, 09:01    Post subject:  

Hi all.

I found a previous forum thread on UDF here.

BFN.

_________________
musher0
~~~~~~~~~~
« Un insensé sur le trône n'est qu'un singe sur le haut d'un toit. » / "A madman
on the throne is just a monkey on top of a roof." (Bernard de Clervaux)
Back to top
View user's profile Send private message 
sheldonisaac

Joined: 21 Jun 2009
Posts: 670
Location: Philadelphia, PA

PostPosted: Sun 13 Aug 2017, 10:29    Post subject:  

musher0 (in part) wrote:
The attached pic describes the overhead taken by the udf format on an 8Mb SD card: not quite 2 Mb's. Linux' ext2 would occupy 130 Mb's on a drive of
the same size. A difference of 128 Mb's

I formatted my 8GB SD card to ext4 (had read somewhere that works well).
It does seem to use up a few hundred MB, even though there were no files on it.
With no journal.
(Oh, I will go try some other tune2fs options)
Anyhow, the cell phone can't mount it.

_________________
Dell E6410: Slim Slacko 6, LuPu Super 2, Tahr
Dell D610: Windows XP, Puppy Linux 5.2
Intel D865GBF: Tiny Windows 7, Puppy Linux 5.2
Acer Aspire One: Windows XP, Puppy Linux 5.2
Back to top
View user's profile Send private message 
labbe5

Joined: 13 Nov 2013
Posts: 862
Location: Canada

PostPosted: Tue 15 Aug 2017, 13:46    Post subject: format-UDF
Subject description: a bash script to automate the UDF formating of hard drives or flash drives
 

Bash script to format a block device (hard drive or Flash drive) in UDF. The output is a drive that can be used for reading/writing across multiple operating system families: Windows, macOS, and Linux. This script should be capable of running in macOS or in Linux.

Installation :

format-udf is a self-contained script. Simply copy format-udf.sh to a directory of your choosing. Don't forget to make it executable:

chmod +x format-udf.sh

Prerequesites :
sudo apt-get install udftools coreutils vim-common

Note :
instead of Vim, use Geany, already installed on all Dogs.

Link here for complete information about format-udf (tested, working and stable) :
https://github.com/JElchison/format-udf

Block Size

If's extremely important that format-udf use the correct block size when formatting your drive. format-udf will attempt to detect and use the correct (logical) block size. If you know what you're doing, the format-udf -b BLOCK_SIZE option can be used to explicitly override the detected block size value.

If the wrong block size is used while formatting (i.e. one that doesn't match the logical block size of your drive), the resultant drive will likely have OS compatibility issues and suffer from non-optimal performance issues.


Maximum UDF File System Capacity

The UDF format has a maximum of 2^32 blocks. With format-udf, these blocks equate to logical blocks.

If your drive's logical block size is 512 bytes, then your maximum UDF file system capacity will be 2 TiB
If your drive's logical block size is 4096 bytes, then your maximum UDF file system capacity will be 16 TiB
Back to top
View user's profile Send private message 
Display posts from previous:   Sort by:   
Page 1 of 1 [15 Posts]  
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » Advanced Topics » Additional Software (PETs, n' stuff) » Utilities
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.0643s ][ Queries: 15 (0.0052s) ][ GZIP on ]