Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Sun 21 Dec 2014, 01:48
All times are UTC - 4
 Forum index » Off-Topic Area » Programming
[bash] variable persistence in piping
Post new topic   Reply to topic View previous topic :: View next topic
Page 2 of 2 [18 Posts]   Goto page: Previous 1, 2
Author Message
potong

Joined: 06 Mar 2009
Posts: 88

PostPosted: Sun 01 Aug 2010, 04:30    Post subject:  

neurino:

It's looking good!

Perhaps a bit of refactoring with regards to the three calls to sed;
Code:
#!/bin/bash

# set some variables
authority="USER:PASS@"
url="https://${authority}mail.google.com/mail/feed/atom"
db_file=/tmp/db.txt

# set IFS to newline for slurping lines into arrays
OIFS="$IFS" IFS=$'\n'

# get messages
message=($(curl -s $url | tee $db_file| sed -nr '
    #
    # each message should resemble this: title@@link@@name
    #
    # only interested in tags between entries
    /<entry>/,/<\/entry>/{
        # bodge link tag to look like title or name
        s/<link.*href="([^"]*)".*\/>/<link>\1<\/link>/
        # extract details and add place holder
        s/<(title|link|name)>(.*)<\/\1>/\2@@/
        # save lines with a placeholder into HS
        /@@/ H
        # end of an entry: process HS
        /<\/entry>/ {
            # overwrite PS with HS
            g
            # drop newlines and final placeholder
            s/amp;|\n|@@$//gp
            # clear PS
            s/.*//
            # swap empty PS into HS
            x
        }
    }'))

# reset IFS (be kind to the maintainer!)
IFS="$OIFS"
 
# show number of messages
echo "${#message[@]} messages"
# show messages
for (( i=0; i<${#message[@]}; i++)){
    msg=${message[i]} tmp=${msg#*@@}
    author=${msg##*@@} subject=${msg%%@@*} link=${tmp%@@*}
    printf "[%02d] %s | %s\nURL: %s\n\n" $(( i + 1 )) ${author:-"no author"} \
            ${subject:-"no subject"} ${link:-"no link"}
}

After all, the less code you write, hopefully the less bugs you have to fix.

The definitive sed site is here
Check out the lookup table method here

HTH

Potong

p.s. you often see
Code:
echo "$some_variable"| whatever ....
it can usually be replaced by a here-string
Code:
whatever... <<<"$some_variable"

p.p.s PS and HS stand for Pattern Space and Hold Space in sed
Back to top
View user's profile Send private message 
neurino


Joined: 15 Oct 2009
Posts: 360

PostPosted: Sun 01 Aug 2010, 07:47    Post subject:  

Uh-oh a lot to read and understand!
I'm copying your code in geany in order to see how it works.

About your p.p.s.:

Code:
whatever... <<<"$some_variable"


I'll use this now on, I never liked the idea of using echo that way (morover being used to pythonic coding...) but it was way better than stroring strings in files...
Back to top
View user's profile Send private message 
neurino


Joined: 15 Oct 2009
Posts: 360

PostPosted: Sun 01 Aug 2010, 10:48    Post subject:  

Ok, I almost got it.

since the feed reports all email thread contributors the <entry> may look like this:

Code:

...
<entry>
<title>mail title</title>
<summary>mail summary blah blah</summary>
<link rel="alternate" href="http://mail.google.com/mail?account_id..." type="text/html" />
<modified>2010-07-31T20:00:46Z</modified>
<issued>2010-07-31T20:00:46Z</issued>
<id>tag:gmail.google.com,2004:1342813184987251</id>
<author>
<name>author name</name>
<email>author@email.com</email>
</author>
<contributor>
<name>contrib 1 name</name>
<email>contrib1@hotmail.com</email>
</contributor>
<contributor>
<name>contrib 2</name>
<email>contrib 2@libero.it</email>
</contributor>
</entry>
...


I added a rule to rename contributors <name> tag to <cname> to leave the rest of the code unchanged. Otherwise we could end with different lenght arrays

Also added quotes to printf parameters since it broke with multiple words, don't know if there's a better/safer way.
Very Happy

Code:

#!/bin/bash

# set some variables
authority="USER:PASS@"
url="https://${authority}mail.google.com/mail/feed/atom"
db_file=/tmp/db.txt

# set IFS to newline for slurping lines into arrays
OIFS="$IFS" IFS=$'\n'

# get messages
message=($(curl -s $url | tee $db_file| sed -nr '
    #
    # each message should resemble this: title@@link@@name
    #
    # only interested in tags between entries
    /<entry>/,/<\/entry>/{
        #rename (we may need it after) contributor <name> to NOT match author <name>
        /<contributor>/,/<\/contributor>/{
                s/<(\/?)name>/<\1cname>/
        }
        # bodge link tag to look like title or name
        s/<link.*href="([^"]*)".*\/>/<link>\1<\/link>/
        # extract details and add place holder
        s/<(title|link|name)>(.*)<\/\1>/\2@@/
        # save lines with a placeholder into HS
        /@@/ H
        # end of an entry: process HS
        /<\/entry>/ {
            # overwrite PS with HS
            g
            # drop newlines and final placeholder
            s/amp;|\n|@@$//gp
            # clear PS
            s/.*//
            # swap empty PS into HS
            x
        }
    }'))

# reset IFS (be kind to the maintainer!)
IFS="$OIFS"
 
# show number of messages
echo "${#message[@]} messages"
# show messages
for (( i=0; i<${#message[@]}; i++)){
    #~ echo ${message[i]}
    msg=${message[i]} tmp=${msg#*@@}
    author=${msg##*@@} subject=${msg%%@@*} link=${tmp%@@*}
    printf "[%02d] %s | %s\nURL: %s\n\n" $(( i + 1 )) "${author:-"no author"}" \
        "${subject:-"no subject"}" "${link:-"no link"}"
}


P.S.: there's some easier way to do what in python is tuple unpacking?

Something like:

Code:

>>> title, name, summary, link = ('the title', 'the name', 'the summary', 'http://mail.google.com/mail?account_id...')


but for bash

Code:

# title, name, summary, link = ("the title@@the name@@the summary@@http://mail.google.com/mail?account_id...")


Question Question

Thanks for teaching!
Back to top
View user's profile Send private message 
Display posts from previous:   Sort by:   
Page 2 of 2 [18 Posts]   Goto page: Previous 1, 2
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » Off-Topic Area » Programming
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.0680s ][ Queries: 12 (0.0079s) ][ GZIP on ]