Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Mon 24 Nov 2014, 01:46
All times are UTC - 4
 Forum index » Off-Topic Area » Programming
bash library
Post new topic   Reply to topic View previous topic :: View next topic
Page 1 of 1 [14 Posts]  
Author Message
technosaurus


Joined: 18 May 2008
Posts: 4376

PostPosted: Tue 20 Apr 2010, 14:31    Post subject:  bash library
Subject description: one script to rule them all
 

What is it?
Maybe it has already been done (Barry's functions4puppy for example), but what I would like to start is a bash script that contains a set of useful functions that can be included in scripts for ease of use the way ".h" files are included in C.

This has already been discussed with regard to gtkdialog and I prototyped what I called "bbgui" to handle the most common use cases and is not the purpose of "bashlib

The largest subset, ashlib, would consist of any useful scripts (in function format) that work with busybox ash and any program that can be included in busybox.

The superset, bashlib, would contain any functions that either do not work with ash or require a program that is not included in busybox.

Why do it?
Not only will this save space by compressing better and not having to add the same snippets several times in several scripts,
but it will also make script development easier and somewhat standardized.
It can also be used to call "functions" directly via a symlink with the function name and a `basename $0` snippet.
These "functions" can consist of what were formerly entire scripts and can all be contained in the same script.
This removes the cost of having to locate and load additional files (at the cost of reading one larger file).
The biggest advantage though is that once the file is read once, all of the functions are available with almost no cost;
this is even more advantageous on the ashlib side because the new nofork/noexec feature in busybox can run many functions with almost no startup penalties.

How?
Start with the most commonly used scripts and convert them to function format - preferably with ash-friendly formating

Sources:
Bash scripting howto
Various scripting sites & linux forums
Puppy scripts

Examples: bbgui, functions4puppy...

_________________
Web Programming - Pet Packaging 100 & 101
Back to top
View user's profile Send private message 
sunburnt


Joined: 08 Jun 2005
Posts: 5042
Location: Arizona, U.S.A.

PostPosted: Tue 20 Apr 2010, 20:06    Post subject:  

At what level of code are you thinking about? It sounds like fix patches for ash and bash.
I was thinking of libraries for: Files, FileSystems, Lists, Program Control, and Gtkdialog.
Example of 2 simple very useful functions:
Code:
  getPS() { ps |grep $1 |egrep -v '(grep|geany|getPS)'                                  # get process line
  } ; export -f getPS
  killPS() { PS=`getPS POPMENU_$1` ; [ "$PS" ]&& kill `echo $PS |awk '{print POPMENU_$1}'`         # kill process ID
  } ; export -f killPS

getPS is useful for testing if an app. is running and seeing what arguments it was run with.

Give a few examples of what you have in mind...
Back to top
View user's profile Send private message 
Lobster
Official Crustacean


Joined: 04 May 2005
Posts: 15117
Location: Paradox Realm

PostPosted: Tue 20 Apr 2010, 22:24    Post subject:  

Sounds like a good idea Smile
Would it be Puppy specific or open to other open source groups?
Good documentation would be useful to future developers.

As well as 4.4 it would work with other woof builds
and earlier Puppy projects? Cool

_________________
Puppy WIKI
Back to top
View user's profile Send private message Visit poster's website 
technosaurus


Joined: 18 May 2008
Posts: 4376

PostPosted: Wed 21 Apr 2010, 07:21    Post subject:  

@Lobster - It would work with any distro that has bash/ash/sh ... If there is one without a shell I must test it out

sunburnt wrote:
At what level of code are you thinking about? It sounds like fix patches for ash and bash.


maybe library is not the exact term - maybe bash header? basically just useful code snippets like your example.

_________________
Web Programming - Pet Packaging 100 & 101
Back to top
View user's profile Send private message 
technosaurus


Joined: 18 May 2008
Posts: 4376

PostPosted: Thu 22 Apr 2010, 18:21    Post subject:  

Aha! I knew there had to be a similar project - Marco's Bash Function Library
http://gna.org/projects/mbfl/

and these:
http://www.codelibary.com/Bash/

I still think they should be divided into ash-capable and bash-required categories, but Marco's work is a good starting point for the bash specific side. Busybox also has some good script examples in the source code for the busybox/ash side.

_________________
Web Programming - Pet Packaging 100 & 101
Back to top
View user's profile Send private message 
technosaurus


Joined: 18 May 2008
Posts: 4376

PostPosted: Thu 29 Apr 2010, 20:11    Post subject:  

here is a primitive script that I started on to generate an xpm image of a horizontal percent bar... still need to make it into a function with parameters (instead of just a script)... maybe add bgcolor instead of always translucent

Could be an alternative to Barry's use of ppmtogif and ppmrough in partview

Edit: More conformant xpm images
Code:
#!/bin/ash
FILENAME=barimage
FGCOLOR=\#000
BGCOLOR=None
PERCENT=50
WIDTH=100
HEIGHT=24
BARSTRING=""
q=0 #should have used for loops
y=0 #not while loops - oops
#get values from input
for x in $@; do
case $x in
   width*)
      WIDTH=`echo $x |cut -d = -f 2`
      ;;
   height*)
      HEIGHT=`echo $x |cut -d = -f 2`
      ;;
   fgcolor*)
      FGCOLOR=`echo $x |cut -d = -f 2`
      [ "$FGCOLOR" != "None" ] && FGCOLOR=\#$FGCOLOR
      ;;
   bgcolor*)
      BGCOLOR=`echo $x |cut -d = -f 2`
      [ "$BGCOLOR" != "None" ] && BGCOLOR=\#$BGCOLOR
      ;;
   name*)
      FILENAME=`echo $x |cut -d = -f 2`
      ;;
   *%)
      PERCENT=`echo $x |cut -d % -f 1`
      ;;
   *)
      echo "usage "$0" XX% name=filename fgcolor=XXX bgcolor=XXX height=XXX width=XXX
      colors can be 3 or 6 digit hexadecimal or None" && exit
      ;;
esac
done


#write header to file
echo '/* XPM */' >${FILENAME}.xpm
echo 'static char *'$FILENAME'_xpm[] = {' >>${FILENAME}.xpm
echo '"'$WIDTH $HEIGHT '2 1",' >>${FILENAME}.xpm
echo '"0 c '$BGCOLOR'",' >>${FILENAME}.xpm
echo '"1 c '$FGCOLOR'",' >>${FILENAME}.xpm

#generate one line of the image
BARMAX=$(($WIDTH*$PERCENT/100))
while [ "$q" -lt $BARMAX ]; do
    q=$(($q + 1))
    BARSTRING=${BARSTRING}1
done
while [ "$q" -le $WIDTH ]; do
    BARSTRING=${BARSTRING}0
    q=$(($q + 1))
done
#write a line for each pixel of height
while [ "$y" -le $HEIGHT ]; do
echo '"'$BARSTRING'"' >>${FILENAME}.xpm
    y=$(($y + 1))
done

echo '};' >>${FILENAME}.xpm
#defaultimageeditor ${FILENAME}.xpm
hbar.png
 Description   
 Filesize   9.55 KB
 Viewed   1219 Time(s)

hbar.png


_________________
Web Programming - Pet Packaging 100 & 101

Last edited by technosaurus on Sun 02 May 2010, 18:03; edited 2 times in total
Back to top
View user's profile Send private message 
Lobster
Official Crustacean


Joined: 04 May 2005
Posts: 15117
Location: Paradox Realm

PostPosted: Thu 29 Apr 2010, 21:28    Post subject:  

Are you planning to make use of this in Puppy 4.4?
Puppy Lucid has the Lucid update so should be able to incorporate.
Other version would maybe use a 5.1 upgrade path.
Anyway I notice upgrades and improvements throughout the Puppyverse due to your innovations Smile

Technosaurus=Puppy Cyber-Dino power

_________________
Puppy WIKI
Back to top
View user's profile Send private message Visit poster's website 
technosaurus


Joined: 18 May 2008
Posts: 4376

PostPosted: Thu 29 Apr 2010, 21:56    Post subject:  

the last one was for a horizontal bar ... this one is for a vertical bar (they share a lot of code so could easily be combined)

Code:

#!/bin/ash
FILENAME=barimage
FGCOLOR=000
BGCOLOR=None
PERCENT=50
WIDTH=100
HEIGHT=24
BARSTRING=""
#get values from input
for x in $@; do
case $x in
   width*)
      WIDTH=`echo $x |cut -d = -f 2`
      ;;
   height*)
      HEIGHT=`echo $x |cut -d = -f 2`
      ;;
   fgcolor*)
      FGCOLOR=`echo $x |cut -d = -f 2`
      ;;
   bgcolor*)
      BGCOLOR=`echo $x |cut -d = -f 2`
      ;;
   name*)
      FILENAME=`echo $x |cut -d = -f 2`
      ;;
   *%)
      PERCENT=`echo $x |cut -d % -f 1`
      ;;
   *)
      echo "usage "$0" XX% name=filename fgcolor=XXX bgcolor=XXX height=XXX width=XXX
      colors can be 3 or 6 digit hexadecimal or None" && exit
      ;;
esac
done

#generate one line of the image
BARMAX=$(($HEIGHT*$PERCENT/100))
BARTOP=$(($HEIGHT-$BARMAX))
#write to file
echo '/* XPM */' >${FILENAME}.xpm
echo 'static char *'$FILENAME'_xpm[] = {' >>${FILENAME}.xpm
echo '"'$WIDTH $HEIGHT '2 1",' >>${FILENAME}.xpm
echo '"0 c #'$BGCOLOR'",' >>${FILENAME}.xpm
echo '"1 c #'$FGCOLOR'",' >>${FILENAME}.xpm

while [ "$q" -lt $HEIGHT ]; do
q=$(($q + 1))
   if [ "$q" -lt $BARTOP ];then
      while [ "$z" -lt $WIDTH ]; do
         BARSTRING=${BARSTRING}0
         z=$(($z + 1))
      done
   else
      while [ "$z" -lt $WIDTH ]; do
         BARSTRING=${BARSTRING}1
         z=$(($z + 1))
      done
   fi
echo '"'$BARSTRING'"' >>${FILENAME}.xpm
BARSTRING=""
z=0
done

echo '};' >>${FILENAME}.xpm
defaultimageviewer ${FILENAME}.xpm
vbar.png
 Description   
 Filesize   9.62 KB
 Viewed   1227 Time(s)

vbar.png


_________________
Web Programming - Pet Packaging 100 & 101

Last edited by technosaurus on Thu 29 Apr 2010, 22:49; edited 1 time in total
Back to top
View user's profile Send private message 
technosaurus


Joined: 18 May 2008
Posts: 4376

PostPosted: Thu 29 Apr 2010, 22:23    Post subject:  

@Lobster - This last one was something I had stuck in my head since Barry posted his battery monitor and from all the tray applets that make use of these types of images. Just about every linux system can use xpm and busybox ash so it is ultra-portable... but then again so is a handkerchief full of holes

As for the big picture - this is more of a long term project. I have this crazy idea that all (or at least most) scripts can be combined into a single script as functions that rely heavily on busybox and LDPRELOAD ... the script stays running (similar to init but after X loading) so that there is no wait time to read scripts (only call already read functions) or libraries (via LDPRELOAD) This is an extension of the busybox gui (bbgui) concept so that more code can be shared and there is less wait to read files.

I know this is counter to what everyone advises with regard to writing scripts, but I have a hard head. x86 can be an embedded platform too dammit

_________________
Web Programming - Pet Packaging 100 & 101
Back to top
View user's profile Send private message 
technosaurus


Joined: 18 May 2008
Posts: 4376

PostPosted: Sun 02 May 2010, 18:26    Post subject:  

Ok, combined the two functions into one - default is horizontal bars unless vert or vertical (vert*) is specified
Code:

#!/bin/ash
FILENAME=barimage
FGCOLOR=\#000
BGCOLOR=None
PERCENT=50
WIDTH=100
HEIGHT=24
BARSTRING=""
q=0 #should have used for loops
y=0 #not while loops - oops
#get values from input
for x in $@; do
case $x in
   width*)WIDTH=`echo $x |cut -d = -f 2`;;
   height*)HEIGHT=`echo $x |cut -d = -f 2`;;
   fg*)FGCOLOR=`echo $x |cut -d = -f 2`
      [ "$FGCOLOR" != "None" ] && FGCOLOR=\#$FGCOLOR;;
   bg*)BGCOLOR=`echo $x |cut -d = -f 2`
      [ "$BGCOLOR" != "None" ] && BGCOLOR=\#$BGCOLOR;;
   name*)FILENAME=`echo $x |cut -d = -f 2`;;
   *%)PERCENT=`echo $x |cut -d % -f 1`;;
   vert*)VERTICAL="true";;
   *)   echo "usage "$0" XX% name=filename fgcolor=XXX bgcolor=XXX height=XXX width=XXX [vertical]
      colors can be 3 or 6 digit hexadecimal or None ... default is horizontal bars" && exit
      ;;
esac
done


#write header to file
echo '/* XPM */' >${FILENAME}.xpm
echo 'static char *'$FILENAME'_xpm[] = {' >>${FILENAME}.xpm
echo '"'$WIDTH $HEIGHT '2 1",' >>${FILENAME}.xpm
echo '"0 c '$BGCOLOR'",' >>${FILENAME}.xpm
echo '"1 c '$FGCOLOR'",' >>${FILENAME}.xpm

if [ "$VERTICAL" == "true" ];then #vertical bars
BARTOP=$(($HEIGHT*$PERCENT/100))
#generate each line of the image
   while [ "$q" -lt $HEIGHT ]; do
   q=$(($q + 1))
      if [ "$q" -lt $BARTOP ];then
         while [ "$z" -lt $WIDTH ]; do
            BARSTRING=${BARSTRING}0
            z=$(($z + 1))
         done
      else
         while [ "$z" -lt $WIDTH ]; do
            BARSTRING=${BARSTRING}1
            z=$(($z + 1))
         done
      fi
   echo '"'$BARSTRING'"' >>${FILENAME}.xpm
   BARSTRING=""
   z=0
   done
else #horizontal bars
#generate one line of the image
   BARMAX=$(($WIDTH*$PERCENT/100))
   while [ "$q" -lt $BARMAX ]; do
      q=$(($q + 1))
      BARSTRING=${BARSTRING}1
   done
   while [ "$q" -le $WIDTH ]; do
      BARSTRING=${BARSTRING}0
      q=$(($q + 1))
   done
#write a line for each pixel of height
   while [ "$y" -le $HEIGHT ]; do
   echo '"'$BARSTRING'"' >>${FILENAME}.xpm
      y=$(($y + 1))
   done
fi

echo '};' >>${FILENAME}.xpm
#defaultimageeditor ${FILENAME}.xpm

_________________
Web Programming - Pet Packaging 100 & 101
Back to top
View user's profile Send private message 
technosaurus


Joined: 18 May 2008
Posts: 4376

PostPosted: Tue 05 Oct 2010, 15:49    Post subject:  

Ok, I haven't looked at this in a while, but I touched base back on the subject recently while trying to unclutter a rootfs. Here is what I did:

1. dumped all bash scripts into a single script and turned them into functions.
2. stripped full paths manually (to prevent accidental breakage)
3. removed duplicate functionality by separating into sub-functions
4. moved non-code comments to a separate file (not really necessary, just loads a slight bit faster)
5. the only non-function command is $0 $@
6. the $0 $@ command will run the function with the same name as the symlink/hardlink to the main script.


After doing a couple of tests using a script containing a function called abiword in rxvt to open up another script that had rxvt -e abiword , I got some encouragement to look into this further when the terminal echoed "I am not the real abiword" (from the function) rather than opening abiword. It appears that subprocesses kept access to those functions, so it may be beneficial to for instance add it to the init script (or .xinitrc?) so that it would only need to be loaded once.


Here are the benefits of doing it this way:
1. If multiple "scripts" are to be called, it only has to seek once since it will immediately call the function and not even have to find the symlink. (seeking a file is often slower than actually reading it)
2. Better compressibility (only 1 header and more text to compare)
3. Links can be created when needed (reduces clutter for easier development and also shrinks image size slightly from having fewer nodes)
4. Easier to find the code you are looking for (use just your text editor instead of pfind etc... and then your editor)
5. Programs calling functions in an already loaded script run way faster (eliminates seek and read... really useful to gtkdialog type scripting guis)
6. Code centrality = lots of examples for the beginner and lower likelihood for developers to accidentally recreate the wheel.
7. Portability is nice - one or two files is easier to move around than 1000
8. Ultra simple version control = diff + patch (even the busybox versions should work for this)
9. More hackability - access to more functions makes it easier to create something new and useful
10. More eyes to fix bugs, optimize code and eliminate dead code (How many scripts are in each distro that are no longer particularly useful.. or outdated?)
11. Extremely easy to fork and re-merge and develop new work in parallel in order to walk the line between stability and cutting edge.


Downsides:
1. will override the actual app if called from within the script if a function of the same name exists (can be worked around using "which")
2. slightly slower to run a single small script (has to open a bigger file - not very noticeable to the end user)
3. more... but just thought of a code snippet to record for later

#untested code to auto create links for every function
Code:
make_links(){
    for x in `cat $0 |grep "(){" |sed "s/(){//g"` do ln -s $0 $x; done
}

_________________
Web Programming - Pet Packaging 100 & 101
Back to top
View user's profile Send private message 
seaside

Joined: 11 Apr 2007
Posts: 887

PostPosted: Thu 07 Oct 2010, 12:19    Post subject:  

technosaurus wrote:
Ok, I haven't looked at this in a while, but I touched base back on the subject recently while trying to unclutter a rootfs. Here is what I did:

1. dumped all bash scripts into a single script and turned them into functions.


technosaurus,

Wow, it looks like you did alot of work on this! Anything you could show us?

I find this area very interesting and a bit puzzling.

The "busybox" concept of linking to a single binary seems to be a good idea for the reasons stated and yet here's a quote from the busybox site under "What is the design of busybox?"


Quote:
The "one binary to rule them all" approach is primarily for size reasons: a single multi-purpose executable is smaller then many small files could be. This way busybox only has one set of ELF headers, it can easily share code between different apps even when statically linked, it has better packing efficiency by avoding gaps between files or compression dictionary resets, and so on.

Work is underway on new options such as "make standalone" to build separate binaries for each applet , and a "libbb.so" to make the busybox common code available as a shared library. Neither is ready yet at the time of this writing.


Perhaps my understanding of this is incomplete, but it implies that "standalone binaries" might be better for some uses.

Also, experimenting with puppy by renaming or deleting the busybox links, shows that the busybox functions still work when called both from the interactive shell and scripts.

This brings up some questions.

Apparently busybox is loaded somewhere in memory and available. How is that done in puppy?

Why are links necessary at all?

Thanks and regards,
s
Back to top
View user's profile Send private message 
technosaurus


Joined: 18 May 2008
Posts: 4376

PostPosted: Thu 07 Oct 2010, 17:59    Post subject:  

links aren't necessary - just convenient. In my bbgui I checked to see if $0 was bbgui and if so used $1 to call the function. It works as long as you don't have parameters to pass ... I think I just need to echo $@ through sed or cut and remove $1

Something like:
Code:
[ "$0" == "megascript" ] && $1 `echo $@ |cut -d " " -f1` #<<I forget the parameter do the opposite of this>>


I have recently posted a busybox daemons package in additional software - this is a time when it is better to have individual applets because they use less memory and are always running. Run top to see what I mean klogd, syslogd, getty, init - they only use 100-200kb separately but between 1 and 2 MB in a defconfig/allyesconfig busybox.

_________________
Web Programming - Pet Packaging 100 & 101
Back to top
View user's profile Send private message 
technosaurus


Joined: 18 May 2008
Posts: 4376

PostPosted: Sun 10 Oct 2010, 22:07    Post subject:  

Ok - I have now created bashbox. It uses many of the same principles as busybox and those listed here - only applied to bash.

The largest improvement is the main_dialog() function that I ported from my bbgui, which allows you to create a gui in about 5 minutes for most basic command line programs.

this version is only a single script, but it should be able to automatically generate symlinks for all of its functions by running bashbox createlinks (assumes bashbox is in /usr/bin for now)
Later versions will likely be a "function library" included with:
. /usr/share/bashbox/libbashbox.so (fake library)

Here is the current growing list of functions:
    addgroupgui() { #uses main_dialog() -> #addgroup [-g GID] [user_name] group_name
    bashbox() { #todo make this more robust
    blockads() { #tweaks /etc/hosts to ban certain sites
    brokenlinks() { #find broken links in given input dir or pwd if not given
    createlinks() { #make executable links for functions matching "()_{"
    for x in `grep "() {" bashbox |cut -d"(" -f1 |grep -v " " |grep -v "#"` do ln -s bashbox $x; done
    fixflash() { #mask your flash version as 10.1 rXXXXXX
    getflash() { #install latest flash
    getflash9() { #flash9 is smaller, fewer deps and still updated
    getinput() { #DIALOG box to return user input to stdout
    google() { #searches google for $@ or if blank uses $DIALOG
    launcher() { #gtkdialog basic launcher program
    main_dialog() { #parses template of vars to gen and eval a dialog then return the results
    maindialogtemplate() {
    percentbar() { #usage: "$0" XX% name=filename fgcolor=XXX bgcolor=XXX height=XXX width=XXX [vertical]
    pget() { #download $1 to HOME/my-documents with axel
    pickdate() { #choose a date - optional input: $1="text" $2=day $3=month $4=year
    picktime() { #output a time - optional input: $1="text" $2=hours $3=minutes $4=seconds
    pickfile() { #DIALOG box to return a user chosen file to stdout
    pickXcolor() { #DIALOG box to return a user chosen X color to stdout
    reset_mixers() { #from /usr/sbin/alsaconf... was called from rc.local0.
    setDIALOG() { #sets $DIALOG to Xdialog if in X or dialog if in console
    setjwmbg() { #TODO tweaks ... include $HOME/.jwmbg in the jwmrc template
    setXCOLORS() { #sets $XCOLORS from ...rgb.txt
    webcam2avi() { #capture your webcam to HOME/webcam.avi - requires ffmpeg & rxvt
    X2avi() { #capture your desktop to HOME/x11-session.avi - ffmpeg needs x11grab
    Xsetvolume() { #set volume with slider
    Xshowimage() { #quick view of image


please post your comments/extensions/improvements
bashbox.bz2
Description 
bz2

 Download 
Filename  bashbox.bz2 
Filesize  6.08 KB 
Downloaded  250 Time(s) 

_________________
Web Programming - Pet Packaging 100 & 101
Back to top
View user's profile Send private message 
Display posts from previous:   Sort by:   
Page 1 of 1 [14 Posts]  
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » Off-Topic Area » Programming
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.1277s ][ Queries: 12 (0.0041s) ][ GZIP on ]