Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Sat 25 May 2013, 00:13
All times are UTC - 4

 Forum index » Off-Topic Area » Programming
Argument list too long .. [SOLVED]
Post new topic   Reply to topic View previous topic :: View next topic
Page 1 of 1 [3 Posts]  
Author Message
sc0ttman


Joined: 16 Sep 2009
Posts: 2174
Location: UK
PostPosted: Fri 22 Feb 2013, 15:04    Post subject:  Argument list too long .. [SOLVED]
Subject description: ..damn thing...		  
Background: I wanna import all the IceCast radio stations into an existing list of favourite streams, in VLC-GTK...

I posted previously in this Programming thread about formatting the IceCast xml list.. Well, that's done (thanks guys), and the list is ready to be imported/added to an existing combobox list, in VLC-GTK...

But, cos the IceCast list is about 3MB (10,000 lines), I cant add the new stuff to the 'fave streams' list in VLC-GTK, without getting the 'argument list too long' error...

The list of streams I wanna add to, is the file ${HOME}/.vlc-gtk/vlc-gtk-faves

It contains lines all of the format

NAME|URL

(Anything with a pipe in the middle would do to test the code below... )

Once this list gets to 1,700 lines long, then VLC-GTK starts to bug out, giving 'Argument list too long' all over the place... Under 1,700 and it just takes a while to load, but no errors...

I've tried to condense down the relevant code into an example below:

(the code was originally for a list of about 12 mms:// TV stations... needs updating me thinks)

Code:
FAVES=${HOME}/.vlc-gtk/vlc-gtk-faves #the list of streams ... each line in format NAME|URL

# build the list for a gtkdialog combobox ..  <item>$LINE</item>
while read LINE
do 
   if [ "${LINE%%|*}" != "$FAVE_STREAM" ];then # dont include the last played stream in the list yet, add it top top later
       FAVE_ITEMS="$FAVE_ITEMS
<item>${LINE%%|*}</item>" #get only fave names for combobox list
   fi
done <"$FAVES"

FAVE_ITEMS="$(echo "$FAVE_ITEMS" | sort)" #sort favourite streams

# add last played stream to top of list
if [ "$FAVE_STREAM" != "" ];then
   [ "$(cat $FAVES | grep "${FAVE_STREAM}|")" != "" ] && FAVE_ITEMS="<item>$FAVE_STREAM</item>
${FAVE_ITEMS}" 
fi


and later in the gtkdialog code of the main VLC-GTK GUI:

Code:
<combobox width-request="295">
   <variable>FAVE_STREAM_URL</variable>
   '$FAVE_ITEMS'
</combobox>


..and lastly, here's the (very crappy, inefficient) code that finds the URL later, once the user has chosen from the list.. I've not even got to this bit yet, to test it with over 1,700 lines, but it'll be relevant..

Code:
#FAVE_STREAM_URL is the name of the stream chosen by the user
while read LINE; do
   case $LINE in
      "${FAVE_STREAM_URL}|"*) FAVE_STREAM_URL="${LINE##*|}"; NAME="${LINE%%|*}" # get name and URL #040812 fix case selection
   esac
done<${HOME}/.vlc-gtk/vlc-gtk-faves



How can I rid myself of these 'Argument list too long' errors when the stream list is over 2,000 lines long? And ideally, speed up the code above a little bit at the same time..
_________________
Akita Linux, VLC-GTK, Pup Search, Pup File Search

Last edited by sc0ttman on Sun 24 Feb 2013, 05:25; edited 1 time in total
Back to top
View user's profile Send private message 
SFR


Joined: 26 Oct 2011
Posts: 571
PostPosted: Fri 22 Feb 2013, 17:11    Post subject:  
Hey Sc0ttman

If it comes for large amount of data, It's better to use temp files instead of variables, both for building <item> index as well as Gtkdialog GUI.

I'd do it like this:

First, I created 10,000 items using:
Code:
for i in {1..10000}; do
  echo "Some_Name"$RANDOM"|www.some_url"$i >> $HOME/.vlc-gtk/vlc-gtk-faves
done

and came up with this (though I'm not saying it's the best way; I'm sure there are better):
Code:
#! /bin/bash

# Get name and URL (it's a function called via <button>, just for testing purposes)
user_choice () {
  NAME="`grep "^$FAVE_STREAM_URL|" $HOME/.vlc-gtk/vlc-gtk-faves`"
  xmessage "$NAME"
}
export -f user_choice

# -----------------------------------------------------------------------------

FAVES=${HOME}/.vlc-gtk/vlc-gtk-faves #the list of streams ... each line in format NAME|URL

# Build index for <combobox> but exclude $FAVE_STREAM (if exists)
cat "$FAVES" | awk 'BEGIN {FS="|"}; {print "<item>"$1"</item>"}' | grep -v "^<item>$FAVE_STREAM</item>$" | sort > /tmp/templist

# add last played stream to top of list
if [ "$FAVE_STREAM" != "" ];then
   echo "<item>$FAVE_STREAM</item>" > /tmp/templist_final
else
   echo -n > /tmp/templist_final
fi

cat /tmp/templist >> /tmp/templist_final

# Build GUI as a file, not a variable
echo '<vbox>
  <combobox width-request="295">
    <variable>FAVE_STREAM_URL</variable>
    '"$(cat /tmp/templist_final)"'
  </combobox>
  <button><label>Show chosen item and its URL</label>
    <action>user_choice</action>
  </button>
</vbox>
' > /tmp/temp_GUI

gtkdialog -f /tmp/temp_GUI

It's hard to say if/how this will work ok for you, since I haven't seen the original list, but I hope it'll help you a bit, somehow. Wink

BTW: So many items makes the GUI really unresponsive (at least initially) - I've learned the lesson while writing IconFinder. Laughing

____________

EDIT: Even simpler, much more simpler:
Code:
FAVES=${HOME}/.vlc-gtk/vlc-gtk-faves #the list of streams ... each line in format NAME|URL

# Build GUI as a file, not a variable
echo '<vbox>
  <combobox width-request="295">
    <variable>FAVE_STREAM_URL</variable>
   
'"$([ "$FAVE_STREAM" != "" ] && echo "<item>$FAVE_STREAM</item>")"'
'"$(awk 'BEGIN {FS="|"}; {print "<item>"$1"</item>"}' $FAVES | grep -v "^<item>$FAVE_STREAM</item>$" | sort)"'

  </combobox>
  <button><label>Show chosen item + URL</label>
    <action>xmessage "`grep "^$FAVE_STREAM_URL|" $HOME/.vlc-gtk/vlc-gtk-faves`"</action>
  </button>
</vbox>
' > /tmp/temp_GUI

gtkdialog -f /tmp/temp_GUI


Greetings!
_________________
[O]bdurate [R]ules [D]estroy [E]nthusiastic [R]ebels => [C]reative [H]umans [A]lways [O]pen [S]ource
Omnia mea mecum porto.
Back to top
View user's profile Send private message 
sc0ttman


Joined: 16 Sep 2009
Posts: 2174
Location: UK
PostPosted: Sun 24 Feb 2013, 04:54    Post subject:  
Hi SFR,

I've yet to get round to testing all this in VLC-GTK fully, but your update looks great, I just ran it with 8,000+ lines.. Much simpler than what I came up with! I see the large delay in first opening the combobox, but that's expected..

Thanks for the help, that's great.
_________________
Akita Linux, VLC-GTK, Pup Search, Pup File Search
Back to top
View user's profile Send private message 
Display posts from previous:   Sort by:   
Page 1 of 1 [3 Posts]  
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » Off-Topic Area » Programming
Jump to:  


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group

[ Time: 0.0642s ][ Queries: 11 (0.0146s) ][ GZIP on ]