bash library

[technosaurus]

What is it?
Maybe it has already been done (Barry's functions4puppy for example), but what I would like to start is a bash script that contains a set of useful functions that can be included in scripts for ease of use the way ".h" files are included in C.

This has already been discussed with regard to gtkdialog and I prototyped what I called "bbgui" to handle the most common use cases and is not the purpose of "bashlib

The largest subset, ashlib, would consist of any useful scripts (in function format) that work with busybox ash and any program that can be included in busybox.

The superset, bashlib, would contain any functions that either do not work with ash or require a program that is not included in busybox.

Why do it?
Not only will this save space by compressing better and not having to add the same snippets several times in several scripts,
but it will also make script development easier and somewhat standardized.
It can also be used to call "functions" directly via a symlink with the function name and a `basename $0` snippet.
These "functions" can consist of what were formerly entire scripts and can all be contained in the same script.
This removes the cost of having to locate and load additional files (at the cost of reading one larger file).
The biggest advantage though is that once the file is read once, all of the functions are available with almost no cost;
this is even more advantageous on the ashlib side because the new nofork/noexec feature in busybox can run many functions with almost no startup penalties.

How?
Start with the most commonly used scripts and convert them to function format - preferably with ash-friendly formating

Sources:
Bash scripting howto
Various scripting sites & linux forums
Puppy scripts

Examples: bbgui, functions4puppy...

[sunburnt]

At what level of code are you thinking about? It sounds like fix patches for ash and bash.
I was thinking of libraries for: Files, FileSystems, Lists, Program Control, and Gtkdialog.
Example of 2 simple very useful functions:

Code: Select all

  getPS() { ps |grep $1 |egrep -v '(grep|geany|getPS)'										    # get process line
  } ; export -f getPS
  killPS() { PS=`getPS POPMENU_$1` ; [ "$PS" ]&& kill `echo $PS |awk '{print POPMENU_$1}'`			# kill process ID
  } ; export -f killPS

getPS is useful for testing if an app. is running and seeing what arguments it was run with.

Give a few examples of what you have in mind...

[Lobster]

Sounds like a good idea
Would it be Puppy specific or open to other open source groups?
Good documentation would be useful to future developers.

As well as 4.4 it would work with other woof builds
and earlier Puppy projects?

[technosaurus]

@Lobster - It would work with any distro that has bash/ash/sh ... If there is one without a shell I must test it out

At what level of code are you thinking about? It sounds like fix patches for ash and bash.

maybe library is not the exact term - maybe bash header? basically just useful code snippets like your example.

[technosaurus]

Aha! I knew there had to be a similar project - Marco's Bash Function Library
http://gna.org/projects/mbfl/

and these:
http://www.codelibary.com/Bash/

I still think they should be divided into ash-capable and bash-required categories, but Marco's work is a good starting point for the bash specific side. Busybox also has some good script examples in the source code for the busybox/ash side.

[technosaurus]

here is a primitive script that I started on to generate an xpm image of a horizontal percent bar... still need to make it into a function with parameters (instead of just a script)... maybe add bgcolor instead of always translucent

Could be an alternative to Barry's use of ppmtogif and ppmrough in partview

Edit: More conformant xpm images

Code: Select all

#!/bin/ash
FILENAME=barimage
FGCOLOR=\#000
BGCOLOR=None
PERCENT=50
WIDTH=100
HEIGHT=24
BARSTRING=""
q=0 #should have used for loops
y=0 #not while loops - oops
#get values from input
for x in $@; do
case $x in
   width*)
      WIDTH=`echo $x |cut -d = -f 2`
      ;;
   height*)
      HEIGHT=`echo $x |cut -d = -f 2`
      ;;
   fgcolor*)
      FGCOLOR=`echo $x |cut -d = -f 2`
      [ "$FGCOLOR" != "None" ] && FGCOLOR=\#$FGCOLOR
      ;;
   bgcolor*)
      BGCOLOR=`echo $x |cut -d = -f 2`
      [ "$BGCOLOR" != "None" ] && BGCOLOR=\#$BGCOLOR
      ;;
   name*)
      FILENAME=`echo $x |cut -d = -f 2`
      ;;
   *%)
      PERCENT=`echo $x |cut -d % -f 1`
      ;;
   *)
      echo "usage "$0" XX% name=filename fgcolor=XXX bgcolor=XXX height=XXX width=XXX
      colors can be 3 or 6 digit hexadecimal or None" && exit
      ;;
esac
done


#write header to file
echo '/* XPM */' >${FILENAME}.xpm
echo 'static char *'$FILENAME'_xpm[] = {' >>${FILENAME}.xpm
echo '"'$WIDTH $HEIGHT '2 1",' >>${FILENAME}.xpm
echo '"0 c '$BGCOLOR'",' >>${FILENAME}.xpm
echo '"1 c '$FGCOLOR'",' >>${FILENAME}.xpm

#generate one line of the image
BARMAX=$(($WIDTH*$PERCENT/100))
while [ "$q" -lt $BARMAX ]; do
    q=$(($q + 1))
    BARSTRING=${BARSTRING}1
done
while [ "$q" -le $WIDTH ]; do
    BARSTRING=${BARSTRING}0
    q=$(($q + 1))
done
#write a line for each pixel of height
while [ "$y" -le $HEIGHT ]; do
echo '"'$BARSTRING'"' >>${FILENAME}.xpm
    y=$(($y + 1))
done

echo '};' >>${FILENAME}.xpm
#defaultimageeditor ${FILENAME}.xpm

[Lobster]

Are you planning to make use of this in Puppy 4.4?
Puppy Lucid has the Lucid update so should be able to incorporate.
Other version would maybe use a 5.1 upgrade path.
Anyway I notice upgrades and improvements throughout the Puppyverse due to your innovations

Technosaurus=Puppy Cyber-Dino power

[technosaurus]

the last one was for a horizontal bar ... this one is for a vertical bar (they share a lot of code so could easily be combined)

Code: Select all

#!/bin/ash
FILENAME=barimage
FGCOLOR=000
BGCOLOR=None
PERCENT=50
WIDTH=100
HEIGHT=24
BARSTRING=""
#get values from input
for x in $@; do
case $x in
	width*)
		WIDTH=`echo $x |cut -d = -f 2`
		;;
	height*)
		HEIGHT=`echo $x |cut -d = -f 2`
		;;
	fgcolor*)
		FGCOLOR=`echo $x |cut -d = -f 2`
		;;
	bgcolor*)
		BGCOLOR=`echo $x |cut -d = -f 2`
		;;
	name*)
		FILENAME=`echo $x |cut -d = -f 2`
		;;
	*%)
		PERCENT=`echo $x |cut -d % -f 1`
		;;
	*)
		echo "usage "$0" XX% name=filename fgcolor=XXX bgcolor=XXX height=XXX width=XXX
		colors can be 3 or 6 digit hexadecimal or None" && exit
		;;
esac
done

#generate one line of the image
BARMAX=$(($HEIGHT*$PERCENT/100))
BARTOP=$(($HEIGHT-$BARMAX))
#write to file
echo '/* XPM */' >${FILENAME}.xpm
echo 'static char *'$FILENAME'_xpm[] = {' >>${FILENAME}.xpm
echo '"'$WIDTH $HEIGHT '2 1",' >>${FILENAME}.xpm
echo '"0 c #'$BGCOLOR'",' >>${FILENAME}.xpm
echo '"1 c #'$FGCOLOR'",' >>${FILENAME}.xpm

while [ "$q" -lt $HEIGHT ]; do
q=$(($q + 1))
	if [ "$q" -lt $BARTOP ];then
		while [ "$z" -lt $WIDTH ]; do
			BARSTRING=${BARSTRING}0
			z=$(($z + 1))
		done
	else
		while [ "$z" -lt $WIDTH ]; do
			BARSTRING=${BARSTRING}1
			z=$(($z + 1))
		done
	fi
echo '"'$BARSTRING'"' >>${FILENAME}.xpm
BARSTRING=""
z=0
done

echo '};' >>${FILENAME}.xpm
defaultimageviewer ${FILENAME}.xpm

[technosaurus]

@Lobster - This last one was something I had stuck in my head since Barry posted his battery monitor and from all the tray applets that make use of these types of images. Just about every linux system can use xpm and busybox ash so it is ultra-portable... but then again so is a handkerchief full of holes

As for the big picture - this is more of a long term project. I have this crazy idea that all (or at least most) scripts can be combined into a single script as functions that rely heavily on busybox and LDPRELOAD ... the script stays running (similar to init but after X loading) so that there is no wait time to read scripts (only call already read functions) or libraries (via LDPRELOAD) This is an extension of the busybox gui (bbgui) concept so that more code can be shared and there is less wait to read files.

I know this is counter to what everyone advises with regard to writing scripts, but I have a hard head. x86 can be an embedded platform too dammit

[technosaurus]

Ok, combined the two functions into one - default is horizontal bars unless vert or vertical (vert*) is specified

Code: Select all

#!/bin/ash
FILENAME=barimage
FGCOLOR=\#000
BGCOLOR=None
PERCENT=50
WIDTH=100
HEIGHT=24
BARSTRING=""
q=0 #should have used for loops
y=0 #not while loops - oops
#get values from input
for x in $@; do
case $x in
	width*)WIDTH=`echo $x |cut -d = -f 2`;;
	height*)HEIGHT=`echo $x |cut -d = -f 2`;;
	fg*)FGCOLOR=`echo $x |cut -d = -f 2`
		[ "$FGCOLOR" != "None" ] && FGCOLOR=\#$FGCOLOR;;
	bg*)BGCOLOR=`echo $x |cut -d = -f 2`
		[ "$BGCOLOR" != "None" ] && BGCOLOR=\#$BGCOLOR;;
	name*)FILENAME=`echo $x |cut -d = -f 2`;;
	*%)PERCENT=`echo $x |cut -d % -f 1`;;
	vert*)VERTICAL="true";;
	*)	echo "usage "$0" XX% name=filename fgcolor=XXX bgcolor=XXX height=XXX width=XXX [vertical]
		colors can be 3 or 6 digit hexadecimal or None ... default is horizontal bars" && exit
		;;
esac
done


#write header to file
echo '/* XPM */' >${FILENAME}.xpm
echo 'static char *'$FILENAME'_xpm[] = {' >>${FILENAME}.xpm
echo '"'$WIDTH $HEIGHT '2 1",' >>${FILENAME}.xpm
echo '"0 c '$BGCOLOR'",' >>${FILENAME}.xpm
echo '"1 c '$FGCOLOR'",' >>${FILENAME}.xpm

if [ "$VERTICAL" == "true" ];then #vertical bars
BARTOP=$(($HEIGHT*$PERCENT/100))
#generate each line of the image
	while [ "$q" -lt $HEIGHT ]; do
	q=$(($q + 1))
		if [ "$q" -lt $BARTOP ];then
			while [ "$z" -lt $WIDTH ]; do
				BARSTRING=${BARSTRING}0
				z=$(($z + 1))
			done
		else
			while [ "$z" -lt $WIDTH ]; do
				BARSTRING=${BARSTRING}1
				z=$(($z + 1))
			done
		fi
	echo '"'$BARSTRING'"' >>${FILENAME}.xpm
	BARSTRING=""
	z=0
	done
else #horizontal bars
#generate one line of the image
	BARMAX=$(($WIDTH*$PERCENT/100))
	while [ "$q" -lt $BARMAX ]; do
		q=$(($q + 1))
		BARSTRING=${BARSTRING}1
	done
	while [ "$q" -le $WIDTH ]; do
		BARSTRING=${BARSTRING}0
		q=$(($q + 1))
	done
#write a line for each pixel of height
	while [ "$y" -le $HEIGHT ]; do
	echo '"'$BARSTRING'"' >>${FILENAME}.xpm
		y=$(($y + 1))
	done
fi

echo '};' >>${FILENAME}.xpm
#defaultimageeditor ${FILENAME}.xpm

[technosaurus]

Ok, I haven't looked at this in a while, but I touched base back on the subject recently while trying to unclutter a rootfs. Here is what I did:

1. dumped all bash scripts into a single script and turned them into functions.
2. stripped full paths manually (to prevent accidental breakage)
3. removed duplicate functionality by separating into sub-functions
4. moved non-code comments to a separate file (not really necessary, just loads a slight bit faster)
5. the only non-function command is $0 $@
6. the $0 $@ command will run the function with the same name as the symlink/hardlink to the main script.


After doing a couple of tests using a script containing a function called abiword in rxvt to open up another script that had rxvt -e abiword , I got some encouragement to look into this further when the terminal echoed "I am not the real abiword" (from the function) rather than opening abiword. It appears that subprocesses kept access to those functions, so it may be beneficial to for instance add it to the init script (or .xinitrc?) so that it would only need to be loaded once.


Here are the benefits of doing it this way:
1. If multiple "scripts" are to be called, it only has to seek once since it will immediately call the function and not even have to find the symlink. (seeking a file is often slower than actually reading it)
2. Better compressibility (only 1 header and more text to compare)
3. Links can be created when needed (reduces clutter for easier development and also shrinks image size slightly from having fewer nodes)
4. Easier to find the code you are looking for (use just your text editor instead of pfind etc... and then your editor)
5. Programs calling functions in an already loaded script run way faster (eliminates seek and read... really useful to gtkdialog type scripting guis)
6. Code centrality = lots of examples for the beginner and lower likelihood for developers to accidentally recreate the wheel.
7. Portability is nice - one or two files is easier to move around than 1000
8. Ultra simple version control = diff + patch (even the busybox versions should work for this)
9. More hackability - access to more functions makes it easier to create something new and useful
10. More eyes to fix bugs, optimize code and eliminate dead code (How many scripts are in each distro that are no longer particularly useful.. or outdated?)
11. Extremely easy to fork and re-merge and develop new work in parallel in order to walk the line between stability and cutting edge.


Downsides:
1. will override the actual app if called from within the script if a function of the same name exists (can be worked around using "which")
2. slightly slower to run a single small script (has to open a bigger file - not very noticeable to the end user)
3. more... but just thought of a code snippet to record for later

#untested code to auto create links for every function

Code: Select all

make_links(){
    for x in `cat $0 |grep "(){" |sed "s/(){//g"` do ln -s $0 $x; done
}

[seaside]

Ok, I haven't looked at this in a while, but I touched base back on the subject recently while trying to unclutter a rootfs. Here is what I did:

1. dumped all bash scripts into a single script and turned them into functions.

technosaurus,

Wow, it looks like you did alot of work on this! Anything you could show us?

I find this area very interesting and a bit puzzling.

The "busybox" concept of linking to a single binary seems to be a good idea for the reasons stated and yet here's a quote from the busybox site under "What is the design of busybox?"

The "one binary to rule them all" approach is primarily for size reasons: a single multi-purpose executable is smaller then many small files could be. This way busybox only has one set of ELF headers, it can easily share code between different apps even when statically linked, it has better packing efficiency by avoding gaps between files or compression dictionary resets, and so on.

Work is underway on new options such as "make standalone" to build separate binaries for each applet , and a "libbb.so" to make the busybox common code available as a shared library. Neither is ready yet at the time of this writing.

Perhaps my understanding of this is incomplete, but it implies that "standalone binaries" might be better for some uses.

Also, experimenting with puppy by renaming or deleting the busybox links, shows that the busybox functions still work when called both from the interactive shell and scripts.

This brings up some questions.

Apparently busybox is loaded somewhere in memory and available. How is that done in puppy?

Why are links necessary at all?

Thanks and regards,
s

[technosaurus]

links aren't necessary - just convenient. In my bbgui I checked to see if $0 was bbgui and if so used $1 to call the function. It works as long as you don't have parameters to pass ... I think I just need to echo $@ through sed or cut and remove $1

Something like:

Code: Select all

[ "$0" == "megascript" ] && $1 `echo $@ |cut -d " " -f1` #<<I forget the parameter do the opposite of this>>

I have recently posted a busybox daemons package in additional software - this is a time when it is better to have individual applets because they use less memory and are always running. Run top to see what I mean klogd, syslogd, getty, init - they only use 100-200kb separately but between 1 and 2 MB in a defconfig/allyesconfig busybox.

[technosaurus]

Ok - I have now created bashbox. It uses many of the same principles as busybox and those listed here - only applied to bash.

The largest improvement is the main_dialog() function that I ported from my bbgui, which allows you to create a gui in about 5 minutes for most basic command line programs.

this version is only a single script, but it should be able to automatically generate symlinks for all of its functions by running bashbox createlinks (assumes bashbox is in /usr/bin for now)
Later versions will likely be a "function library" included with:
. /usr/share/bashbox/libbashbox.so (fake library)

Here is the current growing list of functions:

• addgroupgui() { #uses main_dialog() -> #addgroup [-g GID] [user_name] group_name
  bashbox() { #todo make this more robust
  blockads() { #tweaks /etc/hosts to ban certain sites
  brokenlinks() { #find broken links in given input dir or pwd if not given
  createlinks() { #make executable links for functions matching "()_{"
  for x in `grep "() {" bashbox |cut -d"(" -f1 |grep -v " " |grep -v "#"` do ln -s bashbox $x; done
  fixflash() { #mask your flash version as 10.1 rXXXXXX
  getflash() { #install latest flash
  getflash9() { #flash9 is smaller, fewer deps and still updated
  getinput() { #DIALOG box to return user input to stdout
  google() { #searches google for $@ or if blank uses $DIALOG
  launcher() { #gtkdialog basic launcher program
  main_dialog() { #parses template of vars to gen and eval a dialog then return the results
  maindialogtemplate() {
  percentbar() { #usage: "$0" XX% name=filename fgcolor=XXX bgcolor=XXX height=XXX width=XXX [vertical]
  pget() { #download $1 to HOME/my-documents with axel
  pickdate() { #choose a date - optional input: $1="text" $2=day $3=month $4=year
  picktime() { #output a time - optional input: $1="text" $2=hours $3=minutes $4=seconds
  pickfile() { #DIALOG box to return a user chosen file to stdout
  pickXcolor() { #DIALOG box to return a user chosen X color to stdout
  reset_mixers() { #from /usr/sbin/alsaconf... was called from rc.local0.
  setDIALOG() { #sets $DIALOG to Xdialog if in X or dialog if in console
  setjwmbg() { #TODO tweaks ... include $HOME/.jwmbg in the jwmrc template
  setXCOLORS() { #sets $XCOLORS from ...rgb.txt
  webcam2avi() { #capture your webcam to HOME/webcam.avi - requires ffmpeg & rxvt
  X2avi() { #capture your desktop to HOME/x11-session.avi - ffmpeg needs x11grab
  Xsetvolume() { #set volume with slider
  Xshowimage() { #quick view of image

please post your comments/extensions/improvements