212,213b USB flash names not retained in Pmount, Univ Inst

Please post any bugs you have found
Message
Author
User avatar
BarryK
Puppy Master
Posts: 9392
Joined: Mon 09 May 2005, 09:23
Location: Perth, Western Australia
Contact:

#121 Post by BarryK »

Okay, I think I fixed probedisk...
Attachments
probedisk4-1.gz
(527 Bytes) Downloaded 478 times

User avatar
pakt
Posts: 1157
Joined: Sat 04 Jun 2005, 16:54
Location: Sweden

#122 Post by pakt »

Same hardware as earlier:
# ./probepart-bk3
/dev/hdb1|vfat|2040192
/dev/hdb2|ext2|2008124
/dev/sr0|iso9660|0
/dev/sda1|vfat|249824
#
# ./probedisk4-1
/dev/hdb|disk|KINGSTON
/dev/sr0|cdrom|TSSTcorpCD/DVDW SH-W162C
/dev/sda|Direct-Access|SanDisk Cruzer Mini
#
Paul
Methinks Raspberry Pi were ideal for runnin' Puppy Linux

GuestToo
Puppy Master
Posts: 4083
Joined: Wed 04 May 2005, 18:11

#123 Post by GuestToo »

busybox's expr has a range of -2147483648 to 2147483647

bash's $((...)) has a range of -9223372036854775808 to 9223372036854775807, and the syntax is better looking, to my eye anyway

bc has arbitrary (unlimited) precision

i don't know about busybox's dc ... i think it switches from integers to floating point, and integer mode can't be forced using 0k ... busybox's dc has differences from the real dc, a script that was written using the real dc would probably not work with busybox's dc and vice versa

$((...)) also seems to work in busybox's ash, at least when ash is executed from a bash shell:

# ash

BusyBox v1.01 (2006.08.20-09:28+0000) Built-in shell (ash)
Enter 'help' for a list of built-in commands.

# echo $((3+4))
7
#

i'm not sure if bash is intercepting the $((...)) syntax before it gets to the ash shell or not ... i wouldn't think bash should do that

your script using $((...)) would look something like this:

[ $BSIZE ] && SIZE=$(($BSIZE/1024)) #KB


[ "$SUNITS" = "" ] && SIZE=$(($SIZE*2)) #512 byte blocks.
[ "$SUNITS" = '-m' ] && SIZE=$(($SIZE/1024)) #MB

your script using bc would look something like this:

[ $BSIZE ] && SIZE=`echo "$BSIZE/1024"|bc` #KB


[ "$SUNITS" = "" ] && SIZE=`echo "$SIZE*2"|bc` #512 byte blocks.
[ "$SUNITS" = '-m' ] && SIZE=`echo "$SIZE/1024"|bc` #MB

also, your script prints sizes in mb for -m, 512 byte blocks for $1="" ... any other parameter will print the size in 1k blocks ... i assume that's what you intended

John Doe
Posts: 1681
Joined: Mon 01 Aug 2005, 04:46
Location: Michigan, US

HFS/HFSPlus support

#124 Post by John Doe »

Yesterday another Intel-Mac user arrived.

I recall Barry modified 'libhardware' to enable NTFS recognition. Could the same be done for HFS and HFS+? Thereby enabling it to show up in probepart. Which would only leave some tiny modification to MUT and PMOUNT to enable mounting (example).

GUIDs (couldn't figure out which one is plain HFS):

Code: Select all

Hierarchical File System (HFS+) partition 	48465300-0000-11AA-AA11-00306543ECAC
reference:
http://developer.apple.com/technotes/tn ... #SECADVICE
http://en.wikipedia.org/wiki/GUID_Partition_Table

Here is the line for modules.dep for the zdrive and the squash, I put it right after hfs.ko:

Code: Select all

/lib/modules/2.6.18.1/kernel/fs/hfsplus/hfsplus.ko:
mut.tcl would be something like this at around line 285 (right after 'set results ""' in grab_fs_vol):

Code: Select all

if {$fs == "hfs"} {set fs "hfs"}
if {$fs == "hfsplus"} {set fs "hfsplus"}
pmount would be something like this at around line 20 (right after 'FSTYPE="$1"'):

Code: Select all

[ "$FSTYPE" = "HFS" ] && FSTYPE="hfs"
[ "$FSTYPE" = "HFSPlus" ] && FSTYPE="hfsplus"
Also the respective kernel module would need loaded.

Code: Select all

modprobe hfs
modprobe hfsplus
That's about it I think. I have the modules in my zdrive and one hfs+ partition, so I could test the new scrips and libhardware if you upload it here.

*Edit- THat wasn't it, one other thought I forgot. Saving and booting could be a possible after that. :-)
Last edited by John Doe on Sat 16 Jun 2007, 18:57, edited 1 time in total.

User avatar
Dougal
Posts: 2502
Joined: Wed 19 Oct 2005, 13:06
Location: Hell more grotesque than any medieval woodcut

#125 Post by Dougal »

Oops… I just read Barry's blog.
When I posted that probepart script I made some last-minute change and must have had a typo. I discovered it later but since no-one complained I assumed the script I posted didn't have that modification.

Here're my latest versions of the script, in case they are of any help.
Attachments
probaprt-probedisk-test-scsi.tar.gz
(1.94 KiB) Downloaded 410 times
What's the ugliest part of your body?
Some say your nose
Some say your toes
But I think it's your mind

User avatar
BarryK
Puppy Master
Posts: 9392
Joined: Mon 09 May 2005, 09:23
Location: Perth, Western Australia
Contact:

#126 Post by BarryK »

Heh heh, Jesse may be making all of this history with his "mutt on steroids":
http://www.puppyos.net/forum/?1181868790

(the word "mutt" is slang for a dog, actually a mixed-breed: http://en.wikipedia.org/wiki/Mixed-breed_dog , which kind of describes the multiple-nature of his new MUT -- hey Jess, why not call it "Mutt"?!)

User avatar
BarryK
Puppy Master
Posts: 9392
Joined: Mon 09 May 2005, 09:23
Location: Perth, Western Australia
Contact:

#127 Post by BarryK »

G2, thanks for that advice about $((...)). Yes, where I have the SIZE in bytes there is potential trouble using expr. I've modified the script.

jeffrey
Posts: 168
Joined: Mon 16 Jan 2006, 04:20
Location: Christchurch, New Zealand

awk is a powerful tool too...

#128 Post by jeffrey »

Just as an aside, you can often replace several commands with awk.
For example,

Code: Select all

ALLDRVS="`grep -E '^ .*[^0-9]$|^ .*mmcblk[0-9]$' /proc/partitions | tr -s ' ' | cut -f 5 -d ' ' | tr '\n' ' '`"
could probably be replaced with

Code: Select all

ALLDRVS=`awk 'NR>2 && $4~/[^0-9]$|mmcblk[0-9]$/ {print $4}' /proc/partitions`
Which means IF the line number is greater than 2 (ie skip the header) AND (field 4 is EITHER a string ending in something other than a digit OR the string "mmcblk" followed by a digit) THEN print field 4.
If you really need ALLDRVS to be on one line use

Code: Select all

ALLDRVS=`awk 'NR>2 && $4~/.*[^0-9]$|mmcblk[0-9]$/ {printf "%s ",$4}' /proc/partitions`
Similarly, the following code

Code: Select all

ALLDRVS="`grep -E ' hd| scd| sd| mmc| sr' /proc/diskstats | tr -s ' ' | cut -f 4 -d ' ' | grep -vE 'hd.*[0-9]$|sd.*[0-9]$' | tr '\n' ' '`"
could be replaced by

Code: Select all

ALLDRVS=`awk '$3~/^hd.*[^0-9]$|^scd|^sd.*[^0-9]$|^mmc|^sr/ {print $3}' /proc/diskstats`
As you can guess, I quite like awk...

PaulBx1
Posts: 2312
Joined: Sat 17 Jun 2006, 03:11
Location: Wyoming, USA

#129 Post by PaulBx1 »

Heh heh, Jesse may be making all of this history with his "mutt on steroids"
Maybe as far as MUT and pmount are concerned, but there are a lot of other scripts that use probedisk and probepart, so keep on plugging away at it... :)

Jesse
Posts: 466
Joined: Sun 08 May 2005, 16:07
Location: Auckland, NZ

#130 Post by Jesse »

Hi,

The new mutt will be able to simulate probepart and probedisk through a symlink to the mutt executable, like how busybox does multiple cli apps. If you're not sure what I'm saying there then consider it as another script candidate, but its a compiled binary (from c code).
Its currently working, it does probedisk and probepart, but there are some internal aspects still to tidy up before its ready for alpha/beta testing.

Jesse

Jesse
Posts: 466
Joined: Sun 08 May 2005, 16:07
Location: Auckland, NZ

#131 Post by Jesse »

Hi Barry,

I was just checkining out your probepart-bk3,
there is a minor glitch in it, to do with detecting superfloppy format and/or media change.
I recently discovered this in the new mutt, and fixed it there, I'd seen it previously but not realised what was going on.

The problem comes about when the user swaps the media, e.g. usb compact flash is swapped for another compact flash.
The problem can show as an incorrect size for the device, incorect partition name, or as a really confusing diferent filesystem name.
The good news is that on a second execution of the script the hardware is detected/reported correctly.
The problem comes about because the media has been changed and the kernel has not re-read the partition table from the device to update kernel interfaces like /proc/partitions.

Examples: I have a 256Mb CF that was once an ext3 superfloppy, but then I partitioned it to a 256MB part of vfat. I also have a 32MB CF of superfloppy vfat.

(start with 32MB)
sh-3.00# ./probepart-bk3 | grep sdb
/dev/sdb|vfat|62592

(swap to 256MB)
sh-3.00# ./probepart-bk3 | grep sdb
/dev/sdb|ext3|510464
sh-3.00# ./probepart-bk3 | grep sdb
/dev/sdb1|vfat|509992

(back to 32MB)
sh-3.00# ./probepart-bk3 | grep sdb
/dev/sdb1|none|509992
sh-3.00# ./probepart-bk3 | grep sdb
/dev/sdb|vfat|62592

Jesse

User avatar
BarryK
Puppy Master
Posts: 9392
Joined: Mon 09 May 2005, 09:23
Location: Perth, Western Australia
Contact:

#132 Post by BarryK »

Jesse, so do you mean if probepart is run too soon after a usb drive is changed, it can read /proc/partitions before it has properly updated?
/proc/partitions is not a normal file, we can't read its modify date. How to work around this problem? ...how did you do it in mutt?

I suppose it could be read twice, with one or two second delay, then check that they are identical -- this could be in a while-loop.

jeffrey
Posts: 168
Joined: Mon 16 Jan 2006, 04:20
Location: Christchurch, New Zealand

#133 Post by jeffrey »

I don't understand what happens in Puppy when a device is added, but /sbin/hotplug may be called which may create a short-lived 'loading' file via the /etc/hotplug/firmware.agent script, although I couldn't spot one by running "find /sys /proc -name loading" in a loop while plugging a USB drive in.
When a device is plugged in I'd expect /var/log/messages to be updated so perhaps as a rough solution you could check whether its timestamp is the same as the current time.
(Just a few poor thoughts - not very clean I'm afraid...)

Jesse
Posts: 466
Joined: Sun 08 May 2005, 16:07
Location: Auckland, NZ

#134 Post by Jesse »

Hi,

In mutt, I use an ioctl to ask the kernel to update its info about the mbr of each drive/device, and I read the partition table direct from the MBR.
But it seems to be possible by reading the first sector and attempting to find the filesystem on the drive, which can be scripted for.
I have just done a quick update to your script and it works for me, I tried to optimise out the duplicate fs scans but guessfs_type is needed for both scans.
Attached is my updates to probepart-bk3, you might want to add comments to it?

sh-3.00# ./probepart-bk3 | grep sdb
/dev/sdb|vfat|62592
sh-3.00# ./probepart-bk3 | grep sdb
/dev/sdb1|vfat|509992
sh-3.00# ./probepart-bk3 | grep sdb
/dev/sdb|vfat|62592

Jesse
Attachments
probepart-bk3.tar.gz
(1.1 KiB) Downloaded 384 times

User avatar
BarryK
Puppy Master
Posts: 9392
Joined: Mon 09 May 2005, 09:23
Location: Perth, Western Australia
Contact:

#135 Post by BarryK »

Jesse, I don't quite understand the mods to the script.

1.
What is the effect of just reading and dumping the first 512 bytes of the drive/partition? Does that serve as a wake-up to the kernel?

2.
why do you have to call guess-fstype on both prescan and normal loops?

It would seem 1. and 2. are related somehow.

Apart from the above, note that the case-structure is only entered for drives that do not have partitions. But, isn't this updating problem generic?
In other words, the reading of the first 512 bytes needs to be in a separate case-structure? Like this:

Code: Select all

  if [ ! $SIZE ];then
   [ "`echo "$PARTITIONS" | grep "$ONEDEV"`" != "" ] && continue
   #must be a device without partitions...
   SIZE=0
   DEVICE="$ONEDEV"
   case $DEVICE in
    hd*)
     [ "`cat /proc/ide/$DEVICE/media`" = "cdrom" ] && FSTYPE="iso9660"
     ;;
    scd*|sr*) #usb,sata,scsi cd/dvd drive.
     FSTYPE="iso9660"
     ;;
   esac
   #for hd* or sd* superfloppy, determine size...
   if [ "$PASS" == "normal" -a "$FSTYPE" = "unknown" ];then
    BSIZE=`disktype /dev/$DEVICE 2>/dev/null | grep '^Block device' | cut -f 2 -d '(' | cut -f 1 -d ' '`
    [ $BSIZE ] && SIZE=$(($BSIZE/1024)) #KB
   fi
  fi

  if [ "$FSTYPE" = "unknown" -a "$PASS" = "prescan" ];then
   case $DEVICE in
    sd*|hd*)
     #force a read of the partition/device mbr/boot block in case it has changed
     dd if=/dev/$DEVICE of=/dev/null bs=512 count=1 2>&1 > /dev/null
     ;;
   esac
  fi

User avatar
BarryK
Puppy Master
Posts: 9392
Joined: Mon 09 May 2005, 09:23
Location: Perth, Western Australia
Contact:

#136 Post by BarryK »

Also, can it be further qualified? The code I posted above will read the first 512 bytes of every partition, and the first 512 bytes of drives without partitions. But, in the case of drives with partitions, for example hda with hda1, hda2, etc., isn't it enough to just read the 512 bytes of hda?

User avatar
BarryK
Puppy Master
Posts: 9392
Joined: Mon 09 May 2005, 09:23
Location: Perth, Western Australia
Contact:

#137 Post by BarryK »

I've modified it so the dd runs on the first 512 bytes of the drives (except cd/dvd), not any partitions. Attached.
Attachments
probepart-bk4.gz
(1.09 KiB) Downloaded 488 times

Jesse
Posts: 466
Joined: Sun 08 May 2005, 16:07
Location: Auckland, NZ

#138 Post by Jesse »

Hi Barry,

I just tried your bk4 script with my test, and it is back to the old tricks.

(start with 32MB)
sh-3.00# ./probepart-bk4 | grep sdb
/dev/sdb|vfat|62592
(swap to 256MB)
sh-3.00# ./probepart-bk4 | grep sdb
/dev/sdb|ext3|510464
sh-3.00# ./probepart-bk4 | grep sdb
/dev/sdb1|vfat|509992
(back to 32MB)
sh-3.00# ./probepart-bk4 | grep sdb
/dev/sdb1|none|509992
sh-3.00# ./probepart-bk4 | grep sdb
/dev/sdb|vfat|62592

To be perfectly honest, regarding your #1 and #2 points, I have no idea, I was just trying 'best guess' combinations until I came across a set that worked.

Jesse

User avatar
BarryK
Puppy Master
Posts: 9392
Joined: Mon 09 May 2005, 09:23
Location: Perth, Western Australia
Contact:

#139 Post by BarryK »

Jesse, do you have USB2? I can't duplicate the problem you report. I have tried to swap the usb drives quickly and rerun probepart immediately (the old version before your fix) but I don't get that problem. What I do get is a delay. It seems when I remove a drive, /proc gets updated immediately, but when I plug in a drive there is a delay before it registers in /proc.

So, if I unplug a drive and immediately rerun probepart, no sda drive shows up at all. I wait a second or two, rerun probepart and get the correct sda drive.

So, your fixes don't seem to be doing anything for me. The delay from when a drive is plugged in to registering in /proc seems to be more a function of the usb interface, the time it takes for a pen drive to respond to a query and provide its particulars.

I've been thinking of workarounds, but it doesn't help that you get a different problem, of the previous drive particulars persisting in /proc after the drive is removed. I wonder why I don't have that problem?

Jesse
Posts: 466
Joined: Sun 08 May 2005, 16:07
Location: Auckland, NZ

#140 Post by Jesse »

Hi Barry,

Sorry I should have explained what my test was in a little more detail!
Yes, I have been doing my test case on USB2, and I have just now tried it on USB1.1 and I have the same issue.
You are probably using fully self contained USB drives, which makes all the difference, as there is a hot-plug event with the USB host adapter that lets the kernel know that something has been added or changed.
I am using a "USB Card Reader" which stays plugged into the USB, but the media is removable, removing/inserting the media does not generate a usb-hot-plug event and there is no directly caused kernel update of /proc files.

example of self-contained usb drive:
http://ps.synnex.co.nz/images/FM/50974.jpg

example of compact flash removable media:
http://www.eoneonline.co.nz/site/e-one3 ... ngston.jpg

example of Card Reader (overkill):
http://www.geeks.com/details.asp?invtid ... 25&cat=FLM

example of Card Reader (just like my one):
http://www.geeks.com/details.asp?invtid ... -C&cat=FLM

Over here in NZ, new computers seem to get an internal card reader instead of a floppy drive, from what I have seen. I think it makes sense.

With my reported problem, there is no need to be quick about swapping the CF media, if you wait 10 seconds or 0.5 seconds the /proc files are the same.
I hope that helps clear up what the issue is :)

Jesse

Post Reply