Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Mon 11 Nov 2019, 17:11
All times are UTC - 4
 Forum index » Off-Topic Area » Programming
Some Bash Trap Madness
Post new topic   Reply to topic View previous topic :: View next topic
Page 1 of 1 [8 Posts]  
Author Message
s243a

Joined: 02 Sep 2014
Posts: 2144

PostPosted: Mon 08 Apr 2019, 01:00    Post subject:  Some Bash Trap Madness
Subject description: doing try/catch like stuff in bash
 

Say your writing a bash script and you want to ensure proper cleanup in the event that the process is interrupted or their is an error. One way to do this is with trap statements.

An analogy to the trap statement is the try/catch in java (and other languages) but the problem with the trap statement is that bash doesn't have buit-in methods to nest them.

Bash is a funny language. It is great for writing short scripts and it is easy to work with file systems and external command line utilities but if we try to mimic with bash design patterns which are common in more heavy duty languages than the simplicity of bash is quickly lost.

With a touch of madness I head down this rabbit hole.

There a few discussion on this which one can find via a google search but I found this one about a trap stack particular interesting.

One function given in this link retrieves information about the current trap:
Code:

get_trap() {
  eval echo $(extract_trap `trap -p $1`)
}


To get an idea of what this does here is an example of the "trap -p" statement.
Code:

$ trap -p
trap -- 'echo "SIGINT caught!"' SIGINT

https://linuxconfig.org/how-to-modify-scripts-behavior-on-signals-using-bash-traps

We see that the third field that is printed is the function (or command) which is called when there is a signal of type SIGINT.

In the above get_trap function the function "extract_trap" is called to get the third field (in most cases):
Code:

extract_trap() {
  echo ${@:3:$(($#-3))}
}

The thing to keep in mind here is that in the array of input arguments (i.e. $@) the first index is ("1") one, while most arrays in bash have their first index as ("0") zero, unless specified otherwise. The first ("3") three in the above statement means start at index ("3") three. The value after the next colon (i.e. $(($#-3))) says how many elements to take. In the above example example "trap -p" returns four fields, so the function has four input arguments. $# returns the number of input arguments to the function. When you subtract ("3") three from this you get one.

Therefore, in this example we are taking exactly the third element from the array. The point is though that we want to take the fields after the "--" and before the signal name (i.e. SIGINT). In this example there is only one such field like this. If we had used wither the trap_append or trap_prepend (see below) then we would be taking more than one field.

Related bash array example:
Quote:

For completeness, I'd like to mention how way take elements from arrays in general in bash. Here is a code example
Code:

bla=( [1]=>a [2]=>b [3]=>c [4]=>d )
echo ${bla[@]:3:$((${#bla[@]}-3))}
bla2=( a b c d )
echo ${bla2[@]:2:$((${#bla2[@]}-3))}

In this above snippet ${#bla[@]} gives the number of elements in the array "bla" and we are either echoing the element at index ("3") three from the array "bla" (i,e, echo ${bla[@]:3:1}) or echoing element at index ("2") two (i,e, echo ${bla2[@]:2:1}). In both cases we are echoing the third element of the array. The difference is for the array named "bla" the first index is ("1") one but for the array named "bla2", the first index is ("0") zero .

To get a ("1") one as the first index we had to use associative-arrays-like semantics.



Now I would like to look at the proposed function to push something onto the trap stack linked to above.

Code:

trap_push() {
  local new_trap=$1
  shift
  local sigs=$*
  for sig in $sigs; do
    local stack_name=`trap_stack_name "$sig"`
    local old_trap=$(get_trap $sig)
    eval "${stack_name}"'[${#'"${stack_name}"'[@]}]=$old_trap'
    trap "${new_trap}" "$sig"
  done
}


this seems to be implemented functionally different than I would have done so. However, this is only partly true. In the above function (i.e. trap_push the previous trap is pushed onto the trap stack and replaced by the new trap function. The previous trap is never used unless it is first popped off the stack. How I imagined things should work is that all previous traps should be called. The author did also do this but instead called the functions trap_prepend and trap_append:

Code:

trap_prepend() {
  local new_trap=$1
  shift
  local sigs=$*
  for sig in $sigs; do
    if [[ -z $(get_trap $sig) ]]; then
      trap_push "$new_trap" "$sig"
    else
      trap_push "$new_trap ; $(get_trap $sig)" "$sig"
    fi
  done
}
trap_append() {
  local new_trap=$1
  shift
  local sigs=$*
  for sig in $sigs; do
    if [[ -z $(get_trap $sig) ]]; then
      trap_push "$new_trap" "$sig"
    else
      trap_push "$(get_trap $sig) ; $new_trap" "$sig"
    fi
  done
}


In my mind both prepend and append could have been the functions which use stack semantics. The author (i.e.
Iron Savior) instead chose to use strings, which still could emulate stacks. To make something more like what I had in moned one could write function that:
1. Push the previous trap onto the stack
2. Adds a new trap
3. appends to the new trap a function that calls the previous trap in the trap stack.

we could call this function trap_push_append
Back to top
View user's profile Send private message Visit poster's website 
musher0

Joined: 04 Jan 2009
Posts: 14451
Location: Gatineau (Qc), Canada

PostPosted: Mon 08 Apr 2019, 02:38    Post subject:  

Thanks, s243a,

for sharing your insights on the trap function in bash,
Food for thought, certainly.

BFN.

_________________
musher0
~~~~~~~~~~
Je suis né pour aimer et non pas pour haïr. (Sophocle) /
I was born to love and not to hate. (Sophocles)
Back to top
View user's profile Send private message 
nosystemdthanks


Joined: 03 May 2018
Posts: 696

PostPosted: Mon 08 Apr 2019, 05:10    Post subject:  

never knew there was a trap statement. very cool.
_________________
"microsoft is unique among proprietary software companies: they are the only ones who have actively tried to kill [floss]. it’s not often someone wants to be your friend after trying to kill you for ten years" -- bradley m. kuhn
Back to top
View user's profile Send private message Visit poster's website 
wiak

Joined: 11 Dec 2007
Posts: 1827
Location: not Bulgaria

PostPosted: Mon 08 Apr 2019, 07:18    Post subject:  

Yes, I use bash trap to catch and clean up and also whenever any code error is detected in several of my programs, which incidentally forces me to find the error and correct it during development (in order to get past the trap...). So it thus also helps ensure more 'correct' code published.

I only use very simple trap calls in bash however and bash is certainly a very weird language really. Best thing about it is that it is so close to the underlying C language (as is Linux itself really), but bash syntax sometimes gets more than a bit odd/wild/hard-to-remember. C is actually straighforward in comparison (despite pointers, which I actually love for their efficiency).

wiak

_________________
Tiny Linux Blog: http://www.tinylinux.info/
Check Firmware: www.murga-linux.com/puppy/viewtopic.php?p=1022797
tinycore/slitaz: http://www.murga-linux.com/puppy/viewtopic.php?p=990130#990130
Back to top
View user's profile Send private message 
s243a

Joined: 02 Sep 2014
Posts: 2144

PostPosted: Sat 13 Apr 2019, 03:45    Post subject:  

Thankyou musher0 and nosystemdthanks

wiak wrote:
Best thing about it is that it is so close to the underlying C language (as is Linux itself really)


That really surprises me! I suppose I'm going to have to look at the bash source code someday to see what you mean!
Back to top
View user's profile Send private message Visit poster's website 
s243a

Joined: 02 Sep 2014
Posts: 2144

PostPosted: Sat 13 Apr 2019, 03:51    Post subject:  

I wrote some code to test the two new functions I added:
Code:

#trap_stack_test
rm trap_stack_test.log
touch trap_stack_test.log
source trap_stack.sh
rm trap_test_out.log
touch trap_test_out.log
trap_1(){ echo "trap_1" | tee -a trap_stack_test.log; }
trap_2(){ echo "trap 2" | tee -a trap_stack_test.log; }
trap trap_1 SIGINT
trap_push_append "trap_2" SIGINT
read -p "Press enter to continue"

https://pastebin.com/24Vj7R53

and the functions which I wrote are (discussed above in previous posts):
Code:

call_last_trap(){ #Function added by s243a
  local sig=$1
  trap_pop $sig
  local curr_trap="$(get_trap $sig)"
  eval $curr_trap
  #read -p "Press enter to continue"
}
trap_push_append(){ #Function added by s243a
  local new_trap=$1
  shift
  local sigs=$*
  for sig in $sigs; do
    local stack_name=`trap_stack_name "$sig"`
    local old_trap=$(get_trap $sig)
    eval "${stack_name}"'[${#'"${stack_name}"'[@]}]=$old_trap'
    trap "${new_trap} ; call_last_trap $sig" "$sig"
  done
}

https://pastebin.com/JBbF6Q1H

I also added the following if statment to trap_pop
Code:

  if [ -z "$sigs" ]; then #Avoids inifinite loops
    #Untested. I think we can use numers instead of signal names
    sigs="0 1 2 3 6 9 14 15"
  fi


The reason is that if you leave off the signal name when you call trap_pop then you won't pop anything off the trap stack and this could result in an infinite loop. However, on second thoughts here I realize that this might pop off things that one doesn't want to pop off so perhaps there is a way to get the name of the current signal and use that rather than popping off everything.
Back to top
View user's profile Send private message Visit poster's website 
disciple

Joined: 20 May 2006
Posts: 6988
Location: Auckland, New Zealand

PostPosted: Mon 15 Apr 2019, 03:56    Post subject:  

I think this is kind of related:
I have a script that I use to do a bunch of different file processing tasks, in parallel. It does a bunch of stuff like this:
[code]cd "$PROCESSING/processDAT"
for i in *.dat
do
( iconv -f utf-16 -t 'ASCII//TRANSLIT' "$i" > "${i%.*}-ansi.xyzs" ) &
done
… other tasks in different subfolders using a similar pattern ...
wait
[code]
Paralleling isn't really that important, because I actually generally only use it to perform one of the tasks at a time; it is just convenient to have them all in one script.
I have a wait at the end of the script to make sure all the tasks are complete before it exits.
But there are some tasks which need an internal wait, and I found the behaviour of wait interesting. It doesn't care if I use it inside a for loop or something - it waits for every* previous task in the script to complete. If I want to limit it then I have to put it inside a subshell along with the commands that I want it to wait for.
*Except that if you've piped something through several commands, there is an odd limitation in that it only waits for I think two of the commands in the pipeline to finish..

_________________
Do you know a good gtkdialog program? Please post a link here

Classic Puppy quotes

ROOT FOREVER
GTK2 FOREVER
Back to top
View user's profile Send private message 
wiak

Joined: 11 Dec 2007
Posts: 1827
Location: not Bulgaria

PostPosted: Mon 15 Apr 2019, 07:54    Post subject:  

s243a wrote:
wiak wrote:
Best thing about it is that it is so close to the underlying C language (as is Linux itself really)


That really surprises me! I suppose I'm going to have to look at the bash source code someday to see what you mean!


Since UNIX is written in C, a lot of stdlib and other system C libs are directly to do with UNIX operation/control. The shell is written as a higher level user programming interface to UNIX (or Linux from Puppy point of view) so has functionality to use fifos and pipes more generally, redirection, ttys and so on, which can similarly be interfaced directly using C (albeit in a more fine-grained, more lines of code, manner).

That's what I mean when I say that using bash for system level programming is closely related to using C for same purpose.

I'm not really talking about the code constructs per se, (since Bourne shell more like Algol in many ways), but still it is common to write first system program in shell and then convert it to C for efficiency gains, and relatively easy since C was really written for UNIX as was shell so their functionality/use ends up similar (much more so than when using other language such as Python or Lua or whatever). When we think 'Linux' commands, we tend to think 'shell commandlines' but using C for system programming is really similar conceptually since both are really 'talking' to Linux using the same underlying C system calls. Other languages tend to call up the underlying shell utilities (via something like 'system' command - C can do that too of course) whereas C of course can avoid that go-between and interface to underlying C functions/pipes/fifos/ttys etc etc directly.

https://sites.harvard.edu/~lib113/reference/c/c_history.html
Quote:
C came into being in the years 1969-1973, in parallel with the early development of the Unix operating system; the most creative period occurred during 1972. Another spate of changes peaked between 1977 and 1979, when portability of the Unix system was being demonstrated. In the middle of this second period, the first widely available description of the language appeared: The C Programming Language, often called the `white book' or `K&R' [Kernighan 78]


Anyway, I personally find programming in bash very similar to programming in C when doing system level programming involving UNIX/Linux facilities (as I said, pipes, redirection, and so on). Certainly the code syntax/structure/grammar is very different - it's the end functionality/purpose that is closely integrated. At the end of the day, what is required to do that kind of programming is an understanding of Linux structure (system calls - file descriptors - ttys etc) itself and both C and shell language are specially designed to tackle such work.

wiak

_________________
Tiny Linux Blog: http://www.tinylinux.info/
Check Firmware: www.murga-linux.com/puppy/viewtopic.php?p=1022797
tinycore/slitaz: http://www.murga-linux.com/puppy/viewtopic.php?p=990130#990130
Back to top
View user's profile Send private message 
Display posts from previous:   Sort by:   
Page 1 of 1 [8 Posts]  
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » Off-Topic Area » Programming
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.0790s ][ Queries: 11 (0.0243s) ][ GZIP on ]