busybox extras

For discussions about programming, programming questions/advice, and projects that don't really have anything to do with Puppy.
Message
Author
User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

busybox extras

#1 Post by technosaurus »

I have previously posted some busybox applets here in various spots, but I am starting this thread to consolidate them.

fbvnc - todo? add support for compression

It could also be a good place to discuss porting additional applets to busybox. What puppy utils _should_ be added to busybox?
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

jamesbond
Posts: 3433
Joined: Mon 26 Feb 2007, 05:02
Location: The Blue Marble

#2 Post by jamesbond »

guess_fstype?
I have a patch that will make use of volume_id/* filesystem detection code to compile as a standalone guess_fstype.
Fatdog64 forum links: [url=http://murga-linux.com/puppy/viewtopic.php?t=117546]Latest version[/url] | [url=https://cutt.ly/ke8sn5H]Contributed packages[/url] | [url=https://cutt.ly/se8scrb]ISO builder[/url]

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#3 Post by technosaurus »

that is one of the ones I patch for busybox before, but since then they have changed a few things and I patched in support for new filesystems as did some other, so it needs to be redone - note that busybox already has blkid which does the same thing if configured, but is much slower because it does a read for each fs type instead of a single batch of sequential reads using a union of structs (excellent fast technique to use for guessing mime types too if someone had the time)
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

goingnuts
Posts: 932
Joined: Sun 07 Dec 2008, 13:33
Contact:

#4 Post by goingnuts »

Some of the apps from mut2-src-1.3.2.1238 (mutd mutdaemon mutforeground mutclient probepart probedisk probedisk2 probepci drivelist guess_fstype mounts mediaplug)?

User avatar
Karl Godt
Posts: 4199
Joined: Sun 20 Jun 2010, 13:52
Location: Kiel,Germany

#5 Post by Karl Godt »

losetup should be patched to be able to use it as losetup-FULL .
ash should have ASH_ARGV and ASH_ARGC for to show in set output .

klibc-2.0.1/usr/kinit/fstype could be merged into guess_fstype .

I remember that you ported some kind of media player.
Perhaps start ffplay if dropping to initial ramdisk console as lollypop for new users :) Would make the initrd.lz bigger . SneekyLinux videos come in mind.

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#6 Post by technosaurus »

Karl Godt wrote:losetup should be patched to be able to use it as losetup-FULL .
ash should have ASH_ARGV and ASH_ARGC for to show in set output .

klibc-2.0.1/usr/kinit/fstype could be merged into guess_fstype .

I remember that you ported some kind of media player.
Perhaps start ffplay if dropping to initial ramdisk console as lollypop for new users :) Would make the initrd.lz bigger . SneekyLinux videos come in mind.
argc == $#
argv == $@
argv[1] == $1 ...

that was minimp3, but I'd like to do something with stb_vorbis to play ogg files

what parts of fstype are missing from guessfstype?
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

User avatar
Karl Godt
Posts: 4199
Joined: Sun 20 Jun 2010, 13:52
Location: Kiel,Germany

ARGV

#7 Post by Karl Godt »

in bash :

Code: Select all

ARGV=`set | grep -m1 BASH_ARGV | sed 's%.*(\(.*\))%\1%'`
echo "ARGV=$ARGV"
ARGC=`set | grep -m1 BASH_ARGC | sed 's%.*"\(.*\)")%\1%'`
echo "ARGC=$ARGC"
c=$ARGC 
while read p ; do
p=${p#*=}
p=${p/#\"/}
p=${p/%\"/}
P[$c]="$p"
(( c-- ))
done<<EOI
$(echo "$ARGV" | sed 's%" \[%"\n\[%g')
EOI
echo "${P[@]}"
to pass arguments with spaces
ie in mount script for mount-FULL "${P[@]}" .

For guess_fstype i am thinking of trying to use klibc fstype.c and main.c instead of that Puppy one ( cleaner, easier to work with ) :
/*
* Note:
*
* Minix test needs to come after ext3/ext2, since it's possible for
* ext3/ext2 to look like minix by pure random chance.
*
* LVM comes after all other filesystems since it's possible
* that an old lvm signature is left on the disk if pvremove
* is not used before creating the new fs.
*
* The same goes for LUKS as for LVM.
*/
static struct imagetype images[] = {
{0, "gzip", gzip_image},
{0, "cramfs", cramfs_image},
{0, "romfs", romfs_image},
{0, "xfs", xfs_image},
{0, "squashfs", squashfs_image},
{1, "ext4dev", ext4dev_image},
{1, "ext4", ext4_image},
{1, "ext3", ext3_image},
{1, "ext2", ext2_image},
{1, "minix", minix_image},
{1, "nilfs2", nilfs2_image},
{2, "ocfs2", ocfs2_image},
{8, "reiserfs", reiserfs_image},
{64, "reiserfs", reiserfs_image},
{64, "reiser4", reiser4_image},
{64, "gfs2", gfs2_image},
{64, "btrfs", btrfs_image},
{32, "jfs", jfs_image},
{32, "iso9660", iso_image},
{0, "luks", luks_image},
{0, "lvm2", lvm2_image},
{1, "lvm2", lvm2_image},
{-1, "swap", swap_image},
{-1, "suspend", suspend_image},
{0, "", NULL}
};

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#8 Post by sunburnt »

guess_fstype reports unknown for Puppy SFS files.

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

Re: ARGV

#9 Post by technosaurus »

Karl Godt wrote:in bash :...
in other words it should support arrays - it has been a requested feature for a while, I think Rob Landley has it stubbed out in the toybox shell. I use set to store an array as $@
for example
set --
set -- $@ `find .`
#now I have an "array" of all files/directories in the pwd
#downside is that it is only 1 at a time

@sunburnt - it shouldn't be too difficult to add squashfs support
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

jamesbond
Posts: 3433
Joined: Mon 26 Feb 2007, 05:02
Location: The Blue Marble

Re: ARGV

#10 Post by jamesbond »

Karl Godt wrote:For guess_fstype i am thinking of trying to use klibc fstype.c and main.c instead of that Puppy one ( cleaner, easier to work with )
I was looking for this initially in klibc before I worked on busybox patch but somehow I missed it.
But now that you pointed this out to me, I see that it is missing important filesystems: fat, ntfs, udf, etc.

@sunburnt - I thought I already posted the squashfs patch for puppy's original guess_fstype, somewhere on the forum too?
Fatdog64 forum links: [url=http://murga-linux.com/puppy/viewtopic.php?t=117546]Latest version[/url] | [url=https://cutt.ly/ke8sn5H]Contributed packages[/url] | [url=https://cutt.ly/se8scrb]ISO builder[/url]

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#11 Post by sunburnt »

jamesbond; Yes, but I thought it was important that it make it into the exec.
It seemed that technosaurus wasn`t aware of the failure.

technosaurus; Now it seems you`re unaware that guess_fstype use to work on squash files! :lol:
When the new compression was used in Puppy, guess_fstype quit working ( I think ).
.

jamesbond
Posts: 3433
Joined: Mon 26 Feb 2007, 05:02
Location: The Blue Marble

#12 Post by jamesbond »

No worries sunburnt, busybox's filesystem probe already contains squashfs detection.

Attached is a patch that will make bb_guess_fstype (which can be renamed to guess_fstype) from plain vanilla busybox 1.21. Included in the patch is f2fs probe.

How to use:
1. Extract vanilla busybox 1.21 source
2. Apply the patch (gunzip first)
3. Go to the volume-id directory
4. Type "make"
5. You will end-up with a bb_guess_fstype which can be renamed to guess_fstype for proper use.

You can fine-tune which probes to be included by editing fake_libbb.h (removing the defines) as well as the Makefile (removing unused object code from being linked in).
From here to busybox applet shouldn't be too difficult, I suppose.

Enjoy.

EDIT: Attachment deleted as it was buggy. Please get it from this post instead: http://murga-linux.com/puppy/viewtopic. ... 895#703895.
Last edited by jamesbond on Thu 16 May 2013, 17:30, edited 1 time in total.
Fatdog64 forum links: [url=http://murga-linux.com/puppy/viewtopic.php?t=117546]Latest version[/url] | [url=https://cutt.ly/ke8sn5H]Contributed packages[/url] | [url=https://cutt.ly/se8scrb]ISO builder[/url]

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#13 Post by technosaurus »

jamesbond wrote:Attached is a patch that will make bb_guess_fstype
I bet Denys would accept it as a replacement for blkid if we added UUID and label - they are just at different offsets and most seem to already be documented in the source already.
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

musher0
Posts: 14629
Joined: Mon 05 Jan 2009, 00:54
Location: Gatineau (Qc), Canada

#14 Post by musher0 »

To whom it may concern:

At least reintroduce the prompt function and the command function in busybox's less (-P and !<command> parameters).

Or remove less from busybox altogether. Because, as it is now, busybox's less is a joke.

It's a shame that such an excellent pager/reader with so many useful options be reduced to not even a workable skeleton of itself in busybox.

It's also misrepresentation, on an ethical level, but that is another subject.

musher0
musher0
~~~~~~~~~~
"You want it darker? We kill the flame." (L. Cohen)

jamesbond
Posts: 3433
Joined: Mon 26 Feb 2007, 05:02
Location: The Blue Marble

#15 Post by jamesbond »

technosaurus wrote:
jamesbond wrote:Attached is a patch that will make bb_guess_fstype
I bet Denys would accept it as a replacement for blkid if we added UUID and label - they are just at different offsets and most seem to already be documented in the source already.
I'm not so sure, I thought that's what blkid is for. The code in volume_id is actually meant for blkid, and in all the filesystem probes they actually stores both the label and the UUID if available (in fact, some of the unused probes seem to work right but they don't have UUID and labels identified properly and my guess is that why Denys left them out atm). I would be happy to drop guess_fstype altogether if only busybox blkid can specify which device to probe (instead of probing all) - but alas they are not.

Anyway, as it is now, busybox guess_fstype takes 2.5 times the time of the original guess_fstype :shock:, but I'll take use it anyway because it is still in milliseconds range and it is more maintainable. I am re-posting the patch because the original one I posted was buggy (forgot to return pointer value in xzalloc :evil: - but it worked previously as long as I didn't turn on optimisation ...).
Attachments
bb_guess_fstype.patch.gz
gunzip then patch.
(4.35 KiB) Downloaded 471 times
Fatdog64 forum links: [url=http://murga-linux.com/puppy/viewtopic.php?t=117546]Latest version[/url] | [url=https://cutt.ly/ke8sn5H]Contributed packages[/url] | [url=https://cutt.ly/se8scrb]ISO builder[/url]

Ibidem
Posts: 549
Joined: Wed 26 May 2010, 03:31
Location: State of Jefferson

#16 Post by Ibidem »

jamesbond wrote:
technosaurus wrote:
jamesbond wrote:Attached is a patch that will make bb_guess_fstype
I bet Denys would accept it as a replacement for blkid if we added UUID and label - they are just at different offsets and most seem to already be documented in the source already.
I'm not so sure, I thought that's what blkid is for. The code in volume_id is actually meant for blkid, and in all the filesystem probes they actually stores both the label and the UUID if available (in fact, some of the unused probes seem to work right but they don't have UUID and labels identified properly and my guess is that why Denys left them out atm). I would be happy to drop guess_fstype altogether if only busybox blkid can specify which device to probe (instead of probing all) - but alas they are not.
Do you mean like "busybox blkid /dev/sda1" ?
If so, it works with busybox 1.21.
But if you mean like "busybox blkid /dev/sda", that doesn't work; you must use "for d in /dev/sda*; do blkid $d; done"
blkid does need to print at least TYPE and LABEL as well to match the standard version.

jamesbond
Posts: 3433
Joined: Mon 26 Feb 2007, 05:02
Location: The Blue Marble

#17 Post by jamesbond »

Ibidem wrote:Do you mean like "busybox blkid /dev/sda1" ?
If so, it works with busybox 1.21.
But if you mean like "busybox blkid /dev/sda", that doesn't work; you must use "for d in /dev/sda*; do blkid $d; done"
blkid does need to print at least TYPE and LABEL as well to match the standard version.
Actually, I was wrong and you are right - even my current 1.19.4 busybox already does that :oops:

The real problem is this: if I pass a devpath for something that blkid doesn't understand (try passing an "extended partition" device)
a) util-linux blkid will produce no output.
b) busybox blkid will dump output for *ALL* devices.
Of course we can get around that in scripts, but that's *annoying*.
Fatdog64 forum links: [url=http://murga-linux.com/puppy/viewtopic.php?t=117546]Latest version[/url] | [url=https://cutt.ly/ke8sn5H]Contributed packages[/url] | [url=https://cutt.ly/se8scrb]ISO builder[/url]

Ibidem
Posts: 549
Joined: Wed 26 May 2010, 03:31
Location: State of Jefferson

#18 Post by Ibidem »

jamesbond wrote:
Ibidem wrote:Do you mean like "busybox blkid /dev/sda1" ?
If so, it works with busybox 1.21.
But if you mean like "busybox blkid /dev/sda", that doesn't work; you must use "for d in /dev/sda*; do blkid $d; done"
blkid does need to print at least TYPE and LABEL as well to match the standard version.
Actually, I was wrong and you are right - even my current 1.19.4 busybox already does that :oops:

The real problem is this: if I pass a devpath for something that blkid doesn't understand (try passing an "extended partition" device)
a) util-linux blkid will produce no output.
b) busybox blkid will dump output for *ALL* devices.
Of course we can get around that in scripts, but that's *annoying*.
With busybox 1.21.0, it will produce no output (tested with /dev/sda4)
This is proper behavior, since that device has no UUID.
Also, it seems LABEL is now supported. TYPE, however, isn't.

Hmm...I wonder if an mdev hook could use blkid for /dev/disks/by-*

And I believe busybox has code to detect filesystems, but good luck finding it.

Ibidem
Posts: 549
Joined: Wed 26 May 2010, 03:31
Location: State of Jefferson

#19 Post by Ibidem »

musher0 wrote:To whom it may concern:

At least reintroduce the prompt function and the command function in busybox's less (-P and !<command> parameters).

Or remove less from busybox altogether. Because, as it is now, busybox's less is a joke.

It's a shame that such an excellent pager/reader with so many useful options be reduced to not even a workable skeleton of itself in busybox.

It's also misrepresentation, on an ethical level, but that is another subject.

musher0
From "reintroduce" I get the impression that you are saying it was in busybox at one point...if that is correct, when was that? (I couldn't find any indications in the git log.)

By the way, for my own purposes, (current) busybox less is quite adequate in a way that more falls far short of.

jamesbond
Posts: 3433
Joined: Mon 26 Feb 2007, 05:02
Location: The Blue Marble

#20 Post by jamesbond »

Ibidem wrote:With busybox 1.21.0, it will produce no output (tested with /dev/sda4)
This is proper behavior, since that device has no UUID.
Also, it seems LABEL is now supported. TYPE, however, isn't.
Well :shock: It wasn't obvious from the code. Perhaps it's time to upgrade :evil: I'm quite sure it supports type because in 1.19.4 it does show type. Perhaps you didn't enable a compile-time option.
Hmm...I wonder if an mdev hook could use blkid for /dev/disks/by-*
Probably.
And I believe busybox has code to detect filesystems, but good luck finding it.
It does, in util-linux/volume_id. That's what my patch is for :wink: to make a standalone executable out of these filesystem detection code.
Fatdog64 forum links: [url=http://murga-linux.com/puppy/viewtopic.php?t=117546]Latest version[/url] | [url=https://cutt.ly/ke8sn5H]Contributed packages[/url] | [url=https://cutt.ly/se8scrb]ISO builder[/url]

Post Reply