Author |
Message |
technosaurus

Joined: 18 May 2008 Posts: 4786 Location: Kingwood, TX
|
Posted: Sat 11 May 2013, 17:20 Post subject:
populating /dev from /sys |
|
kernel's shell example (~16.0 seconds)
Code: | #!/bin/bash
# Populate block devices
for i in /sys/block/*/dev /sys/block/*/*/dev
do
if [ -f $i ]
then
MAJOR=$(sed 's/:.*//' < $i)
MINOR=$(sed 's/.*://' < $i)
DEVNAME=$(echo $i | sed -e 's@/dev@@' -e 's@.*/@@')
mknod /dev/$DEVNAME b $MAJOR $MINOR
fi
done
# Populate char devices
for i in /sys/bus/*/devices/*/dev /sys/class/*/*/dev
do
if [ -f $i ]
then
MAJOR=$(sed 's/:.*//' < $i)
MINOR=$(sed 's/.*://' < $i)
DEVNAME=$(echo $i | sed -e 's@/dev@@' -e 's@.*/@@')
mknod /dev/$DEVNAME c $MAJOR $MINOR
fi
done
| Of course it is slow - they are calling sed 3 times for every entry
my reworking
Code: | #!/bin/sh
IFS=":"
DEV=/dev
#char devices
for i in /sys/bus/*/devices/*/dev /sys/class/*/*/dev; do
read MAJOR MINOR < "$i" && i=${i%/dev} && NAME=${i##*/} && \
mknod "$DEV/$NAME" c $MAJOR $MINOR 2>/dev/null
done
#block devices
for i in /sys/block/*/dev /sys/block/*/*/dev; do
read MAJOR MINOR < "$i" && i=${i%/dev} && NAME=${i##*/} && \
mknod "$DEV/$NAME" b $MAJOR $MINOR 2>/dev/null
done
|
my version runs in ~2 seconds with bash 4
pretty good improvement, but ash reduces it to under 1 second
I then tried it with busybox built to:
1)prefer applets, 2)run as a standalone shell and 3)run nofork applets directly
This got it down to just over 1/4 second
but out of curiousity I took a look at mknod and found out that it is only listed as a noexec applet, so I modified include/applets.src.h to change mknode from NOEXEC to NOFORK and that reduced the execution time to just over 1/10th of a second
... great now I can quickly create my containerized $container/dev to use with lxc (which is why I didn't hardcode /dev btw) - kiss my ass udev/systemd - don't need your slow, bloated crap
_________________ Check out my github repositories. I may eventually get around to updating my blogspot.
|
Back to top
|
|
 |
technosaurus

Joined: 18 May 2008 Posts: 4786 Location: Kingwood, TX
|
Posted: Sat 11 May 2013, 18:02 Post subject:
|
|
Next up: handling hotplug events
Process to figure out what to do:
1) need to know environment variables and args that are passed
--->use env >>somefile in a dummy script
--->echo /root/dummy /proc/sys/kernel/hotplug
2)use this data to analyze what we may need to do.
attaching a gzip output from removing then reinserting a usb flash drive
Description |
|

Download |
Filename |
hotplug.events.gz |
Filesize |
747 Bytes |
Downloaded |
672 Time(s) |
_________________ Check out my github repositories. I may eventually get around to updating my blogspot.
|
Back to top
|
|
 |
Karl Godt

Joined: 20 Jun 2010 Posts: 4208 Location: Kiel,Germany
|
Posted: Sat 11 May 2013, 21:13 Post subject:
|
|
Very nice as function for rc.sysinit !
time dyn_mknod
Code: |
#!/bin/ash
#technosaurus, May 2013
#http://murga-linux.com/puppy/viewtopic.php?t=86125
function_dyn_mknod(){
alias mknod='busybox mknod'
alias rm='busybox rm'
alias mkdir='busybox mkdir'
ERR=/dev/stderr
OLDIFS="$IFS"
IFS=":"
DEV=/dev
mkdir -p "$DEV"
#char devices
for i in /sys/bus/*/devices/*/dev /sys/class/*/*/dev; do
MAJOR='';MINOR='';NAME=''
read MAJOR MINOR < "$i" && i=${i%/dev*} || continue
NAME=${i##*/}
echo "'$NAME' c '$MAJOR' '$MINOR'"
[ "$MAJOR" -a "$MINOR" -a "$NAME" ] || continue
#rm -f "$DEV"/"$NAME" || continue
rm ${DEV}/${NAME} || continue
#mknod "$DEV"/"$NAME" c $MAJOR $MINOR 2>$ERR
mknod ${DEV}/${NAME} c $MAJOR $MINOR
done
#block devices
for i in /sys/block/*/dev /sys/block/*/*/dev; do
MAJOR='';MINOR='';NAME=''
read MAJOR MINOR < "$i" && i=${i%/dev*} || continue
NAME=${i##*/}
echo "'$NAME' b '$MAJOR' '$MINOR'"
[ "$MAJOR" -a "$MINOR" -a "$NAME" ] || continue
#rm -f "$DEV"/"$NAME" || continue
rm ${DEV}/${NAME} || continue
#mknod "$DEV"/"$NAME" b $MAJOR $MINOR 2>$ERR
mknod ${DEV}/${NAME} b $MAJOR $MINOR
done
IFS="$OLDIFS"
}
function_dyn_mknod
|
[ .. huge snip .. ]
'sda9' '8' '9'
'sdb1' '8' '17'
real 0m3.964s
user 0m0.128s
sys 0m0.412s
Had strange messages for using double quotes in the
mknod "$DEV"/"$NAME" b $MAJOR $MINOR 2>$ERR line >
'sdb1' '8' '17'
/root/my-applications/bin/dyn_mknod: line 44: can't create : nonexistent directory
Never had such before ..
BB is Lupu BusyBox v1.16.2 (2010-06-19 18:02:46 GMT-8 )
running a luma preview by plazdayz from 2010 ..
Funny the system still runs without the nodes
Anyway nice code . Have such similar in my /etc/init.d/alsa, but much more heavy:
Code: | D=`find /sys -type d -name "*sound*"`
POOL=''
for i in $D ; do
##echo $i ;
DEV=`find $i -type f -name "dev" -exec echo -n {}@ \; -exec cat {} \; | tr ':' '@' | tr ' ' '@' -exec echo \; ` ;
POOL="$POOL $DEV"
done
POOL=`echo $POOL | rev`
for i in $POOL ; do
NODE=`echo $i | cut -f 3 -d '@'`
NODE=`echo $NODE | rev`
NODE=`basename $(dirname $(echo $NODE) )`
MAJ=`echo $i | cut -f 2 -d '@'`
MAJ=`echo $MAJ | rev`
Min=`echo $i | cut -f 1 -d '@'`
Min=`echo $Min | rev`
echo "$NODE" "$MAJ" "$Min"
rm /dev/$NODE
mknod /dev/$NODE c $MAJ $Min
rm /dev/snd/$NODE
mknod /dev/snd/$NODE c $MAJ $Min
done |
|
Back to top
|
|
 |
technosaurus

Joined: 18 May 2008 Posts: 4786 Location: Kingwood, TX
|
Posted: Sat 11 May 2013, 22:11 Post subject:
|
|
there is no need to prepend busybox ... (if busybox has it ash will use it)
and ERR-... should be ERR=... (probably the error source)
and what was the point of adding the rm ... ?
its usually better to use [ ! -f "$VAR" ] && command "$VAR"
... as it is, your rework will remove all existing nodes that are found and not recreate it (bad when Puppy uses a static /dev), but rerunning mknod on an existing device is quick and fails harmlessly (thus the 2>/dev/null)
keep in mind this script should only run once, so there shouldn't be any preexisting nodes (aside from the static ones that could be wrong)
also unnecessary echoes in loops really slow down scripts and should only be used for debugging - you can use OUTPUT="$OUTPUT
$NEWDATA" in the loop and then echo $OUTPUT afterwards
but Puppy's busybox is pretty conservatively built, if you're building it for an initrd, the following should be enabled 1)prefer applets, 2)standalone shell and 3)run nofork applets directly
This alone makes a 5x speed improvement.
Another related tip
"$VAR1/$VAR2" makes the whole thing one string because all variables are expanded inside double quotes (but not in single quotes) - using "$A"/"$B" is unnecessary and just slows it down
FYI - this makes it possible to construct a large command with many arguments (including quoted ones) - especially useful with programs like dialog Xdialog
Code: | [ "$DISPLAY" ] && DIALOG=Xdialog || DIALOG=dialog
MSG="hello world"
mycommand="$DIALOG --msgbox '$MSG' 0 0"
eval $mycommand
#this technique is useful for iteratively building a command in a loop |
_________________ Check out my github repositories. I may eventually get around to updating my blogspot.
|
Back to top
|
|
 |
Karl Godt

Joined: 20 Jun 2010 Posts: 4208 Location: Kiel,Germany
|
Posted: Sat 11 May 2013, 22:46 Post subject:
|
|
Quote: | and ERR-... should be ERR=... (probably the error source) |
was the failure =- typo . Thanks .
|
Back to top
|
|
 |
technosaurus

Joined: 18 May 2008 Posts: 4786 Location: Kingwood, TX
|
Posted: Sat 11 May 2013, 22:58 Post subject:
|
|
start of a hotplug script:
Code: | #!/bin/ash
DEV=/dev
#todo? maybe modify to "$ACTION$SUBSYSTEM" ???
add(){
SUBDIR=${DEVNAME%/*}
[ "$MAJOR" ] && [ "$MINOR" ] || return
[ "$SUBDIR" ] && [ ! -d "$SUBDIR" ] && mkdir -p "$SUBDIR"
case $SUBSYSTEM in
block)mknod "$DEV/$DEVNAME" b $MAJOR $MINOR;;
*)mknod "$DEV/$DEVNAME" c $MAJOR $MINOR;;
esac
}
remove(){
[ "$DEVNAME" ] && rm "$DEV/$DEVNAME"
}
$ACTION
#notes
#ACTION=add,remove,???
#SEQNUM=integer
#DEVPATH=subdirectory of /sys
#SUBSYSTEM=scsi,bsg,...
#MODALIAS=scsi:t-0x00
#DEVTYPE=scsi_device (same as $1 ??)
#DEVNAME=bsg/8:0:0:0 (@ /dev/... ??)
#ACTION=add
#MAJOR=integer
#MINOR=integer
|
_________________ Check out my github repositories. I may eventually get around to updating my blogspot.
|
Back to top
|
|
 |
Karl Godt

Joined: 20 Jun 2010 Posts: 4208 Location: Kiel,Germany
|
Posted: Sat 11 May 2013, 23:03 Post subject:
|
|
For bsg device node path I could only think of reading the uevent files
like :
Code: |
#!/bin/ash
ERR=/dev/null
CHAR_DEVICES=`sed -n '/Character devices:/,/Block devices:/p' /proc/devices`
BLOK_DEVICES=`sed -n '/Block devices:/,//p' /proc/devices`
OLDIFS="$IFS"
find /sys -type f -name uevent |
while read uev_file
do
MAJ='';Min='';DEV=''
IFS='='
while read name val
do
case $name in
MAJOR) MAJ=$val;;
MINOR) Min=$val;;
DEVNAME) DEV=$val;;
*) :;;
esac
done<$uev_file
IFS="$OLDIFS"
[ "$MAJ" -a "$Min" -a "$DEV" ] || continue
KINDC=`echo "$CHAR_DEVICES" | grep -w "$MAJ"`
[ "$KINDC" ] && KIND=c || {
KINDB=`echo "$BLOK_DEVICES" | grep -w "$MAJ"`
[ "$KINDB" ] && KIND=b ; }
[ "$KIND" ] || continue
#echo "'$DEV' '$KIND' '$MAJ' '$Min'"
#rm /dev/$DEV
mknod /dev/$DEV $KIND $MAJ $Min
done 2>$ERR
|
but that takes 10 seconds here .
|
Back to top
|
|
 |
technosaurus

Joined: 18 May 2008 Posts: 4786 Location: Kingwood, TX
|
Posted: Sat 11 May 2013, 23:57 Post subject:
|
|
I used the uevent files in one of my jwm tools, but all you have to do is
. pathto/uevent
and it will source the variables in it
find will recurse the giant barrage of symlinks many levels, plus find is external, so if the path can be narrowed down to:
/sys/something/something/*/whatever
you can use a Code: | for x in /sys/something/something/*/whatever; do
something_with "$x"
done |
what are the results of find?
looks like /sys/class/bsg/<devicename>
with a symlink @
/sys/block/<devname>/queue/bsg
WTF is it a char or block device?
_________________ Check out my github repositories. I may eventually get around to updating my blogspot.
|
Back to top
|
|
 |
Karl Godt

Joined: 20 Jun 2010 Posts: 4208 Location: Kiel,Germany
|
Posted: Sun 12 May 2013, 11:08 Post subject:
|
|
Don't know .. had to read https://en.wikipedia.org/wiki/Device_file
What seems to be that there are some doublettes there for both
Character devices:
1 mem
128 ptm
254 bsg
Block devices:
1 ramdisk
128 sd
254 mdp
|
Back to top
|
|
 |
Karl Godt

Joined: 20 Jun 2010 Posts: 4208 Location: Kiel,Germany
|
Posted: Sun 12 May 2013, 12:46 Post subject:
|
|
Quote: | find will recurse the giant barrage of symlinks many levels |
from man find :
-P Never follow symbolic links. This is the default behaviour.
Think that find in /sys is quiet fast .
But of course finds a lot of unallowed uevent files .
-readable
Matches files which are readable.
-perm mode
File's permission bits
-readable did not work for me but
Code: | find -P /sys -type f -name uevent -perm /u+r |
find itself is fast but processing of the uevent files is slow .
Version 2 for events ie pup_event_frontend_d detects sdc in /sys/block
Code: |
#!/bin/ash
[ "$*" ] && WANTED="${*// /|}"
WANTED="${WANTED#|}"
WANTED="${WANTED%|}"
ERR=/dev/null;QUIET='-q'
#ERR=/dev/stderr
mount -t proc none /proc &>$ERR
mount -t sysfs none /sys &>$ERR
[ -L /dev/fd ] || ln -sf /proc/self/fd /dev/fd
ln -sf /dev/fd/0 /dev/stdin
ln -sf /dev/fd/1 /dev/stdout
ln -sf /dev/fd/2 /dev/stderr
CHAR_DEVICES=`sed -n '/Character devices:/,/Block devices:/p' /proc/devices`
BLOK_DEVICES=`sed -n '/Block devices:/,$ p' /proc/devices`
OLDIFS="$IFS"
function_read_uev(){
while read uev_file
do
MAJ='';Min='';DEV='';KINDB='';KINDC='';KIND=''
IFS='='
while read name val
do
case $name in
MAJOR) MAJ=$val;;
MINOR) Min=$val;;
DEVNAME) DEV=$val;;
*) :;;
esac
done<$uev_file
IFS="$OLDIFS"
[ "$MAJ" -a "$Min" -a "$DEV" ] || continue
Dev="${DEV##*/}"
KINDC=`echo "$CHAR_DEVICES" | grep -w "${MAJ}" | grep "${Dev:0:2}"`
[ "$KINDC" ] && KIND=c || {
KINDB=`echo "$BLOK_DEVICES" | grep -w "${MAJ}" | grep "${Dev:0:2}"`
[ "$KINDB" -o "$MAJ" = 259 ] && KIND=b ; }
## 259 is blkext but nodes are still /dev/sda not /dev/blkexta
[ "$KIND" ] || continue
#echo "'$DEV' '$KIND' '$MAJ' '$Min'"
#rm -f /dev/$DEV || rmdir /dev/$DEV
DEV="/$DEV"
mkdir -p "/dev/${DEV%/*}"
mknod "/dev/$DEV" $KIND $MAJ $Min
done 2>$ERR
}
if [ ! "$WANTED" ] ; then
find -P /sys -type f -name uevent -perm /u+r | function_read_uev
else
for p in ${WANTED//|/ }
do
find -P /sys -path "*${p}*" -type f -name uevent -perm /u+r | function_read_uev
done
fi
|
# time sys-ev2 sdc
real 0m1.413s
user 0m1.185s
sys 0m0.460s
|
Back to top
|
|
 |
technosaurus

Joined: 18 May 2008 Posts: 4786 Location: Kingwood, TX
|
Posted: Sun 12 May 2013, 15:38 Post subject:
|
|
the while read blocks like that will skip the last line unless you do something like
while read LINE || [ "$LINE" ];do
guess we've been lucky the last thing is not important
in this case the LINE is a uevent file with shell formatted contents that can be sourced as a file with variable names, (but we may need to NULL out previous VARs if it is found that some uevent files do not contain all of MAJOR, MINOR, DEVNAME and DEVTYPE) such as:
Code: | while read LINE || [ "$LINE" ];do
MAJOR="" MINOR="" DEVNAME="" DEVTYPE=""
. "$LINE" #this is the full path of the uevent file
#now we can do stuff with MAJOR, MINOR, DEVNAME and DEVTYPE, no parsing required
done |
EDIT: ... wow - I need to look into what kernel version got /sys/dev/{char,block} - that would be so much easier (has symlinks to ALL char and block devices in appropriate subdirectories) I guess the kernel docs are more outdated than I even thought
_________________ Check out my github repositories. I may eventually get around to updating my blogspot.
|
Back to top
|
|
 |
technosaurus

Joined: 18 May 2008 Posts: 4786 Location: Kingwood, TX
|
Posted: Sun 12 May 2013, 16:19 Post subject:
|
|
here is my rework using /sys/dev/{char,block}/*/uevent
Code: | #!/bin/ash
IFS=": "
DEV=/dev #change this for testing
#char devices
for i in /sys/dev/char/*/uevent; do
. "$i"
[ "${DEVNAME%/*}" != "${DEVNAME}" ] && [ ! -d "$DEV/${DEVNAME%/*}" ] && mkdir -p "$DEV/${DEVNAME%/*}"
mknod "$DEV/$DEVNAME" c $MAJOR $MINOR 2>/dev/null
done
#block devices
for i in /sys/dev/char/*/uevent; do
. "$i"
[ "${DEVNAME%/*}" != "${DEVNAME}" ] && [ ! -d "$DEV/${DEVNAME%/*}" ] && mkdir -p "$DEV/${DEVNAME%/*}"
mknod "$DEV/$DEVNAME" b $MAJOR $MINOR 2>/dev/null
done
#todo create symlinks? |
_________________ Check out my github repositories. I may eventually get around to updating my blogspot.
|
Back to top
|
|
 |
technosaurus

Joined: 18 May 2008 Posts: 4786 Location: Kingwood, TX
|
Posted: Sun 12 May 2013, 22:35 Post subject:
|
|
I uncovered a bug ... mknod will only accept major and minor 0..255, but the kernel supports higher values, so the mknod in busybox fails on /dev/bus/usb/004/001 (amongst others) because it has a minor value of 384 ... so I patched busybox mknod to use MINOR&0xff making it have an actual minor value of 128 - IDK if this is OK or not - probably not
musl-libc doesn't account for it, but it looks like uclibc and glibc do (prepends the extra bit in an unsigned int vs using just an unsigned short) - posting to musl-libc list
_________________ Check out my github repositories. I may eventually get around to updating my blogspot.
|
Back to top
|
|
 |
Karl Godt

Joined: 20 Jun 2010 Posts: 4208 Location: Kiel,Germany
|
Posted: Mon 13 May 2013, 12:09 Post subject:
|
|
The idea of sourceing is really fine !
Would probably come to it after several years due to not using eval gtkdialog or source /etc/rc.d/PUPSTATE much .
Most sourced files are actualy scripts containing functions .
*
Just did a
BusyBox v1.16.2 (2010-06-19 18:02:46 GMT-8 ) multi-call binary.
# busybox mknod /dev/test c 499 499
# ls -l /dev/test
crw-r--r-- 1 root root 499, 499 2013-05-13 17:12 /dev/test
# busybox mknod /dev/testb b 500 500
# ls -l /dev/testb
brw-r--r-- 1 root root 500, 500 2013-05-13 17:15 /dev/testb
seems to work OK.
|
Back to top
|
|
 |
technosaurus

Joined: 18 May 2008 Posts: 4786 Location: Kingwood, TX
|
Posted: Mon 13 May 2013, 13:33 Post subject:
|
|
I figured out a way to allow other packages to provide their own configurations for a shell script. this one is an example for a hotplugger shell script replacement (just the subsystem section)
Code: | subsystem="case $SUBSYSTEM in "
for x in /etc/mdev.d/subsystem.*; do
subsystem="$subsystem $(< ${x})"
done
subsystem="$subsystem esac"
eval "$subsystem" |
so if you wanted to do something for all block subsystems you would
Code: | echo "block)dosomething $some $args ;;" > /etc/mdev.d/subsystem.block | but it could be more complex
I may find some use for this technique in jwm_tools so that packages can provide their owns configs - how do I come up with this crap?
_________________ Check out my github repositories. I may eventually get around to updating my blogspot.
|
Back to top
|
|
 |
|