Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Fri 28 Aug 2015, 10:42
All times are UTC - 4
 Forum index » Taking the Puppy out for a walk » Suggestions
'better' usbstoragefunc in the init script
Moderators: Flash, Ian, JohnMurga
Post new topic   Reply to topic View previous topic :: View next topic
Page 1 of 3 [37 Posts]   Goto page: 1, 2, 3 Next
Author Message
Jesse

Joined: 08 May 2005
Posts: 465
Location: Auckland, NZ

PostPosted: Thu 03 Aug 2006, 19:45    Post subject:  'better' usbstoragefunc in the init script  

Hi Barry & co,

I have been poking my nose around the init script and I have found a 'better' way to detect when the usb-storage module has identified the usb-storage devices, so that we can then proceed to mount them.
Basically we scan the dmesg output for the count of devices that are usb-storage, and the count of usbstorage devices that have been scanned, and wait for the number to be the same.
I have split the function into two components, because there is an enforced pause after loading usb-storage, it does a pause so that any device 'just plugged in' has time to 'spin up', in case it is a real hard disk.

To use them:

Code:
usbstoragefunc
sleep 1
usbstoragefunc_completeion


Or to use them and be clever with the waiting period:

Code:
usbstoragefunc
insmod /lib/modules/${KERNVER}/cdrom.ko    #before ide-cd.
insmod /lib/modules/${KERNVER}/ide-cd.ko

#might as well do this too...
insmod /lib/modules/${KERNVER}/sr_mod.ko

#ide zip or ide ls-120 drive?...
[ ! "`dmesg | grep "ATAPI FLOPPY"`" = "" ] && insmod /lib/modules/${KERNVER}/ide-floppy.ko

#I dunno why, a bit further down complains needs nls_cp437 and nls_iso8859-1.
insmod /lib/modules/${KERNVER}/nls_cp437.ko
insmod /lib/modules/${KERNVER}/nls_iso8859-1.ko
#filesystems...
[ -f /lib/modules/${KERNVER}/reiserfs.ko ] && insmod /lib/modules/${KERNVER}/reiserfs.ko
[ -f /lib/modules/${KERNVER}/ntfs.ko ] && insmod /lib/modules/${KERNVER}/ntfs.ko
#...my 2.6.16.7 has reiserfs and ntfs builtin, leave these lines for other kernels.
insmod /lib/modules/${KERNVER}/unionfs.ko
usbstoragefunc_completeion


Here are the functions:

Code:
usbstoragefunc() {
 if [ "$USB" ];then
  #k2.6: annoying, lsmod reports usb-storage.ko as usb_storage if loaded...
  if [ "`lsmod | grep "^usb_storage"`" = "" ];then
   echo "USB-Storage: Loading usb-storage module"
   insmod /lib/modules/${KERNVER}/usb-storage.ko &
   echo "USB-Storage: Detecting connected device(s)..."
  fi
 fi
}

usbstoragefunc_completeion() {
 if [ "$USB" ];then
  #k2.6: annoying, lsmod reports usb-storage.ko as usb_storage if loaded...
  if [ ! "`lsmod | grep "^usb_storage"`" = "" ];then
   SLPCNT="0";
   USBSTORAGES="0";
   USBSTORAGESAVAILABLE="0";
   while true;do
     USBSTORAGES="`/bin/dmesg | grep -o "usb-storage: device found at.*" | wc -l | sed -e 's/ //g'`";
     USBSTORAGESAVAILABLE="`/bin/dmesg | grep -o "usb-storage: device scan complete" | wc -l | sed -e 's/ //g'`";
     if [ $USBSTORAGES -gt 0 ]; then
       if [ $USBSTORAGES -eq $USBSTORAGESAVAILABLE ]; then
         echo "USB-Storage: Device scan complete, found $USBSTORAGESAVAILABLE devices"
         break
       fi
     fi
     if [ $SLPCNT -gt 50 ]; then
       echo "USB-Storage: Device scan time-out $SLPCNT secs, found $USBSTORAGESAVAILABLE of $USBSTORAGES device(s)."
       break
     fi
     sleep 1
     SLPCNT=`expr $SLPCNT + 1`
   done
   echo ""
  fi
 fi
}


I've quite a few reboots on my system and this is working for me.
Would anyone else like to give it a go?

Jesse
Back to top
View user's profile Send private message 
BarryK
Puppy Master


Joined: 09 May 2005
Posts: 7450
Location: Perth, Western Australia

PostPosted: Thu 03 Aug 2006, 20:20    Post subject:  

Jesse, that's great, yes I'll test it
...soon, my PC is rather preoccupied right now (see developer news about T2)!
Back to top
View user's profile Send private message Visit poster's website 
Jesse

Joined: 08 May 2005
Posts: 465
Location: Auckland, NZ

PostPosted: Fri 04 Aug 2006, 01:57    Post subject:  

Hi Barry,

coolies, yes usb still needs a little work, I'm trying out a few ideas for hardware matching too, based on the pci hardware scanning tool I posted a while back, with scripting in init. Looks like it could well be promising.

T2 sounds very interesting, I don't know if I could handle not having my pc for four straight days though! I salute your dedication!

Jesse
Back to top
View user's profile Send private message 
BarryK
Puppy Master


Joined: 09 May 2005
Posts: 7450
Location: Perth, Western Australia

PostPosted: Sun 06 Aug 2006, 14:19    Post subject:  

Jesse,
for comparison, you might like to evaluate 'hwscan2' -- I got it out of the
ROCK Linux project.
I can compile it statically with dietlibc if you like.
Source attached...
hwscan.tar.gz
Description 
gz

 Download 
Filename  hwscan.tar.gz 
Filesize  43 KB 
Downloaded  374 Time(s) 
Back to top
View user's profile Send private message Visit poster's website 
Jesse

Joined: 08 May 2005
Posts: 465
Location: Auckland, NZ

PostPosted: Sun 06 Aug 2006, 19:07    Post subject:  

Hi Barry,

Thanks, your download contains a binary.
hwscan2 looks quite cool, its doing the same sort of thing that I've been looking into for getting modules loaded from initrd.
Though after going inside and outside the process for detecting USB controllers, it seems to be fairly simple, but as far as I can tell, you need an executable to extract the "class subtype" from the /proc/bus/pci/00/00.0 device files, which contains binary, not text info, so you can't really write a bash script file to get it.

Ok, so how can we make this work for us? well PCI class 0c03 is USB Controller (0c is for "serial", 03 is for "USB"), which is Class category and Class Type, lspci shows us some useful info:

sh-3.00# lspci | grep 0c03
00:02.0 Class 0c03: 10de:005a (rev a2)
00:02.1 Class 0c03: 10de:005b (rev a3)

but to get the class subtype I use my program:

sh-3.00# ./elspci -l | grep -i 0c03
00:02.1 0C0320 10DE:005B <>
00:02.0 0C0310 10DE:005A <ohci_hcd>

The angle brackets show the module bound to that pci device (loaded driver).

According to the /lib/modules/2.6.16.7/modues.pcimap there are just three matches for the USB controller modules

uhci-hcd 0xffffffff 0xffffffff 0xffffffff 0xffffffff 0x000c0300 0xffffffff 0x0
ehci-hcd 0xffffffff 0xffffffff 0xffffffff 0xffffffff 0x000c0320 0xffffffff 0x0
ohci-hcd 0xffffffff 0xffffffff 0xffffffff 0xffffffff 0x000c0310 0xffffffff 0x0

Now that we have "class subtype" we can match exactly which modules are needed to any pc, in the init script.

Code:

  [ ! "`elspci -l | grep '0C0300'`" = "" ] && insmod /lib/modules/${KERNVER}/uhci-hcd.ko
  [ ! "`elspci -l | grep '0C0310'`" = "" ] && insmod /lib/modules/${KERNVER}/ohci-hcd.ko
  [ ! "`elspci -l | grep '0C0320'`" = "" ] && insmod /lib/modules/${KERNVER}/ehci-hcd.ko


Potentially this means we do not need the /initrd/pci-usb.ids file to correctly identify what type of controllers we have by description string.

sh-3.00# lspci -i /initrd/pci-usb.ids | grep USB
00:02.0 USB Controller: nVidia Corporation CK804 USB Controller (rev a2)
00:02.1 USB Controller: nVidia Corporation CK804 USB Controller (rev a3)

Look they are both identified as "USB" rather than 00:02.1 being "USB 2".
hmmmm, makes you wonder how many id strings are compatible?

So now we can check to see exactly what we have installed:

sh-3.00# modprobe ehci-hcd
sh-3.00# ./elspci -d -l | grep -i 0c03
00:02.1 0C0320 10DE:005B <ehci_hcd>
00:02.0 0C0310 10DE:005A <ohci_hcd>

Coolies! Now I get USB2 speed rather than having to use the "low speed compatibiliy", now my Flash disk boot is about 60 seconds quicker Smile

Jesse
elspci.tar.gz
Description 
gz

 Download 
Filename  elspci.tar.gz 
Filesize  9.59 KB 
Downloaded  400 Time(s) 
Back to top
View user's profile Send private message 
BarryK
Puppy Master


Joined: 09 May 2005
Posts: 7450
Location: Perth, Western Australia

PostPosted: Mon 07 Aug 2006, 11:33    Post subject:  

That's great !!! Very Happy
Back to top
View user's profile Send private message Visit poster's website 
Jesse

Joined: 08 May 2005
Posts: 465
Location: Auckland, NZ

PostPosted: Mon 07 Aug 2006, 20:41    Post subject:  

Hmm, seems I found a bug on my system when there is no usb devices needed to be identified, i.e. booting from cdrom it would do the 50 second pause.

So here is an update with a few more comments.

Code:
usbstoragefunc() {
 if [ "$USB" ];then
  #k2.6: annoying, lsmod reports usb-storage.ko as usb_storage if loaded...
  if [ "`lsmod | grep "^usb_storage"`" = "" ];then
   echo "USB-Storage: Loading usb-storage module"
   insmod /lib/modules/${KERNVER}/usb-storage.ko &
   echo "USB-Storage: Detecting connected device(s)..."
  fi
 fi
}

usbstoragefunc_completeion() {
 if [ "$USB" ];then
  #k2.6: annoying, lsmod reports usb-storage.ko as usb_storage if loaded...
  if [ ! "`lsmod | grep "^usb_storage"`" = "" ];then
   SLPCNT="0";
   USBSTORAGES="0";   # number of storages that are pending device identification.
   USBSTORAGESAVAILABLE="0";   # number of storages that are up and running
   USBSTORAGES_NEEDED="0";    # Do we need these things for boot process?
   if [ "`echo -n "$PMEDIA" | grep --extended-regexp "ide|sata|scsi"`" = "" ];then
    # booting off a usb drive... so we need to wait till we have found them
    USBSTORAGES_NEEDED="1";
   fi
   while true;do
     #What can we find out?
     USBSTORAGES="`/bin/dmesg | grep -o "usb-storage: device found at.*" | wc -l | sed -e 's/ //g'`";
     USBSTORAGESAVAILABLE="`/bin/dmesg | grep -o "usb-storage: device scan complete" | wc -l | sed -e 's/ //g'`";
     #have we got everything needed and found?
     if [ $USBSTORAGES_NEEDED -eq 1 ]; then
       if [ $USBSTORAGES -eq $USBSTORAGESAVAILABLE ]; then
         echo "USB-Storage: Device scan complete, found $USBSTORAGESAVAILABLE devices"
         break
       fi
     fi
     # Do we really need to hang about? early break out.
     if [ $USBSTORAGES_NEEDED -eq 0 ]; then
       if [ $SLPCNT -ge 2 ]; then
         echo "USB-Storage: Minimal boot delay, found $USBSTORAGES usb storage devices"
         break
       fi
     fi
     #have we timed out?
     if [ $SLPCNT -gt 50 ]; then
       echo "USB-Storage: Device scan time-out $SLPCNT secs, found $USBSTORAGESAVAILABLE of $USBSTORAGES device(s)."
       break
     fi
     sleep 1
     SLPCNT=`expr $SLPCNT + 1`
   done
   echo ""
  fi
 fi
}

Jesse
Back to top
View user's profile Send private message 
BarryK
Puppy Master


Joined: 09 May 2005
Posts: 7450
Location: Perth, Western Australia

PostPosted: Mon 07 Aug 2006, 21:53    Post subject:  

Jesse,
I've announced your work on the Developer News page.
Attached is a modified init script, and elspci compiled statically with dietlibc.
-- it has to be compiled statically to work in the 2.02 initial ramdisk, and it's
only 14K compared with 6.5K for the shared version.

I modified the init script before seeing your last post. Perhaps you would like
to peruse my script and make any mods to it such as the above improvement.
I integrated your suggestions into the init script, with careful (paranoid)
checking that modules do load okay, also note that usbstoragefunc gets
called from two different places, so you will also find a call to
usbstoragecompletionfunc in two places.

Note, I haven't tested it yet.

The init script is for Puppy 2.02, not 2.01.
elspci-stuff.tar.gz
Description 
gz

 Download 
Filename  elspci-stuff.tar.gz 
Filesize  21.91 KB 
Downloaded  363 Time(s) 
Back to top
View user's profile Send private message Visit poster's website 
Jesse

Joined: 08 May 2005
Posts: 465
Location: Auckland, NZ

PostPosted: Tue 08 Aug 2006, 03:47    Post subject:  

Hi Barry,

I noticed that you only coded for one usb driver, they all used the same variable. I've made minor changes to allow them all to load as necessary.
Looks good, fingers crossed! Smile

Jesse
init.tar.gz
Description 
gz

 Download 
Filename  init.tar.gz 
Filesize  13.2 KB 
Downloaded  371 Time(s) 
Back to top
View user's profile Send private message 
RobertB


Joined: 02 Jan 2006
Posts: 145
Location: Big D

PostPosted: Tue 08 Aug 2006, 14:31    Post subject:  

As a fan of Eats, Shoots & Leaves, I must humbly point out that the correct spelling of the function would be "completion", not "completeion".
Absolutely meaningless to the execution of the program, of course -- the functions could be called "Fred" and "Wilma", for all it would care.
Back to top
View user's profile Send private message Visit poster's website 
Jesse

Joined: 08 May 2005
Posts: 465
Location: Auckland, NZ

PostPosted: Tue 08 Aug 2006, 19:48    Post subject:  

Hi RobertB,

Yep, you're completely right Smile My ability to spell correctly is directly proportionaly to either my ability to remember, or energy to check things out on dictionary.com.... my 'typo' for sure. Looks like Barry fixed the spelling in the latest incarnation, all should be good unless I have another go at it Rolling Eyes

Jesse
Back to top
View user's profile Send private message 
Jesse

Joined: 08 May 2005
Posts: 465
Location: Auckland, NZ

PostPosted: Tue 08 Aug 2006, 21:01    Post subject:  

Hi Barry & co,

Boot time checking of is a disk USB or SATA in a bash script.
Prompted by problem reported by losl in thread http://www.murga.org/~puppy/viewtopic.php?p=61994

Still needs some code for getting other SCSI disks into a SCSIDRIVES list.

May need to replace "cat /var/log/messages" with "dmesg"

output example:

SATADRIVES found: sda
USBDRIVES found: sdb sdc sdd sde sdf sdg sdh sdi

Code:

USBDRIVES="";
SATADRIVES="";
SCSIDRIVES="";

SCSI_CTRL="`cat /var/log/messages | grep -o "scsi. : .*" | sed -e 's/SCSI emulation for USB Mass Storage devices/usb_storage/' -e 's/ : /:/' -e 's/ /_/g' `"
#echo SCSI_CTRL found: $SCSI_CTRL

USBCTRLRS="";
SATACTRLRS="";
SCSICTRLRS="";
for ONE in $SCSI_CTRL
do
   SCSI="`echo $ONE | grep -o "scsi[0-9]"`";
   CTRLR="`echo $ONE | grep -o ":.*" | sed -e 's/://' `";
   CTRLRTYPE="SCSI";
   if [ $CTRLR = "usb_storage" ]; then
      CTRLRTYPE="USB";
   fi
   if [ ! "`echo $CTRLR | grep -i sata`" = "" ]; then
      CTRLRTYPE="SATA";
   fi
   CTRLRN="`echo $SCSI | grep -o "[0-9]"`";
   case $CTRLRTYPE in
      USB)   USBCTRLRS="$CTRLRN|$USBCTRLRS";;
      SATA)   SATACTRLRS="$CTRLRN|$SATACTRLRS";;
      SCSI)   SCSICTRLRS="$CTRLRN|$SCSICTRLRS";;
   esac
   echo "Controller $SCSI is type $CTRLRTYPE   ( $CTRLR )"
done
#echo "USB $USBCTRLRS";
#echo "SATA $SATACTRLRS";
#echo "SCSI $SCSICTRLRS";



SATA_CTRL="`cat /var/log/messages | grep -o "scsi. : .*" | grep ata `"
#echo SATA_CTRL found: $SATA_CTRL

#get the list of SATA drives
SATA_SCAN="`cat /var/log/messages | grep "sd .* Attached scsi disk" | grep -o "sd .*" | cut -d ' ' -f 1,2,6 | tr ' ' '.' `";
#looking for:    sd 2:0:0:0: Attached scsi disk sda
#echo SATA_SCAN found: $SATA_SCAN


#get the list of USB drives
for ONE in $SATA_SCAN
do
   DRIVE=`echo $ONE | grep -o "sd[a-z].*"`
   CTRLR=`echo $ONE | cut -d '.' -f 2 | cut -d ':' -f 1`
   #echo $DRIVE on $CTRLR hopefully in $SATACTRLRS
   if [ ! "`echo "|$SATACTRLRS" | grep "|${CTRLR}|"`" = "" ]; then
      SATADRIVES="`echo $SATADRIVES $DRIVE | tr ' ' '\n' | sort -u | tr '\n' ' '`";
   fi
done

echo SATADRIVES found: $SATADRIVES


USB_CTRL="`cat /var/log/messages | grep "scsi. : SCSI emulation for USB Mass Storage devices" | grep -o "scsi."`";
#echo USB_CTRL found: $USB_CTRL

USB_DISKS="";
USB_SCAN="`cat /var/log/messages | grep "sd .* Attached scsi removable disk" | grep -o "sd .*" | cut -d ' ' -f 1,2,7 | tr ' ' '.' `";
# looking for:    sd 4:0:0:0: Attached scsi removable disk sdb
#echo USB_SCAN found: $USB_SCAN

#get the list of USB drives
for ONE in $USB_SCAN
do
   DRIVE=`echo $ONE | grep -o "sd[a-z].*"`
   CTRLR=`echo $ONE | cut -d '.' -f 2 | cut -d ':' -f 1`

   #echo $DRIVE on $CTRLR hopefully in $USBCTRLRS

   if [ ! "`echo "|$USBCTRLRS" | grep "|${CTRLR}|"`" = "" ]; then
      USBDRIVES="`echo $USBDRIVES $DRIVE | tr ' ' '\n' | sort -u | tr '\n' ' '`";
   fi
done

echo USBDRIVES found: $USBDRIVES



Jesse
Back to top
View user's profile Send private message 
Jesse

Joined: 08 May 2005
Posts: 465
Location: Auckland, NZ

PostPosted: Wed 09 Aug 2006, 08:55    Post subject:  

Wha hey! this init script stuff is getting easier!!! Very Happy

I've made it print out a list of devices relative to the PMEDIA setting, if it contains sata or usb
You can also get more info on the screen if you set the pfix to include 'sd'.

This update tested lots of times on my pc, and snapshot below:



Attachment includes the init script and the elspci file.

Jesse
for_initrd.tar.gz
Description 
gz

 Download 
Filename  for_initrd.tar.gz 
Filesize  23.41 KB 
Downloaded  518 Time(s) 
Back to top
View user's profile Send private message 
Jesse

Joined: 08 May 2005
Posts: 465
Location: Auckland, NZ

PostPosted: Wed 09 Aug 2006, 09:05    Post subject:  

Oh the 'pfix=sd' goes in the syslinux.cfg file in the root directory of the flash drive, if you're using the syslinux (default) bootloader.
If this init script gets it wrong, it would be great to see the output when pfix contains 'sd'.

Also, if you can wingle the PUI (Puppy universal installer) to install puppy to SATA hard disk drive, use a PMEDIA=satahd as this will only look in the sd* devices that are actually sata.
If this init script gets it wrong, it would be great to see the output when pfix contains 'sd'.

This update "for_initrd.tar.gz" is NOT official (just like all the rest in this thread) but this one is 'much better', if you're keen and know how to update and try it out, please have a go, and let us know how it went, good and bad feedback are both great to see, so let us know! Wink

Jesse
Back to top
View user's profile Send private message 
BarryK
Puppy Master


Joined: 09 May 2005
Posts: 7450
Location: Perth, Western Australia

PostPosted: Sat 12 Aug 2006, 08:55    Post subject:  

jesse,
I was wondering about timeout for a usb1 interface.
usb1 can take up to 25 seconds for a usb drive to be "ready" after the usb-storage
module is loaded.
I was wondering just where that delay occurs. This dmesg message:
usb-storage: device found at
...does that appear immediately, or in the first second or so?
then:
usb-storage: device scan complete
...this one appears up to 25 seconds later?

...I guessed that is probably how it works, so modified usbstoragecompletionfunc
accordingly, but it's just a guess.

Have you tested on a usb1 system?
Back to top
View user's profile Send private message Visit poster's website 
Display posts from previous:   Sort by:   
Page 1 of 3 [37 Posts]   Goto page: 1, 2, 3 Next
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » Taking the Puppy out for a walk » Suggestions
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.1262s ][ Queries: 13 (0.0266s) ][ GZIP on ]