Question about the use of 'sed' command

For discussions about programming, programming questions/advice, and projects that don't really have anything to do with Puppy.
Post Reply
Message
Author
User avatar
ASRI éducation
Posts: 3197
Joined: Sat 09 May 2009, 12:10
Location: France
Contact:
Contact ASRI éducation

Question about the use of 'sed' command

#1 Post by ASRI éducation »

Hello.
I need help for the command 'sed'.
Thank you in advance for your suggestions.

I prepare a GUI designed for novice users.
The objective is to allow the user to select the categories available in the main menu of Puppy.
For this, I want use a 'sed' on the file /etc/xdg/templates/_root_.jwmrc (to change visibility of category).
Alas, I do not know how...

Tested = does not work!

Code: Select all

#!/bin/sh
#Script to hide category Desktop in main menu of Puppy (with WM jwm)

## Actual category (example = dektop) ==> Visible
GREPDESKTOP=`grep puppy-desktop.menu /etc/xdg/templates/_root_.jwmrc`
OLDVALUE=$GREPDESKTOP

## New category ==> Not visible
HIDE="<!-- PUPPYMENU jwm-xdgmenu /etc/xdg/menus/puppy-desktop.menu -->"
NEWVALUE=$HIDE

## Command sed (to replace OLDVALUE by NEWVALUE)
sed -i 's/$OLDVALUE/$NEWVALUE/' /etc/xdg/templates/_root_.jwmrc

exit
If someone has an idea to improve my sed ...
I am obviously available to provide additional information.

Regards,
Attachments
gui_20150121.png
(17.84 KiB) Downloaded 434 times
Projet ASRI éducation => [url=http://asri-education.org/]Association[/url] | [url=http://forum.asri-education.org/]Forum[/url] | [url=http://dl01.asri-education.org/]Dépôt[/url] | [url=http://kids.asri-education.org/]Espace kids[/url]
Top
step
Posts: 1349
Joined: Fri 04 May 2012, 11:20

Re: Need help for command 'sed'

#2 Post by step »

If I understand what you want to do, you want to change all lines that contain "puppy-desktop.menu" into the same line but with a "<!--" before and a "-->" after (as one line). Is that what you want?
You can try this (untested):

Code: Select all

sed '/puppy-desktop\.menu/s/\(.*\)/<!-- \1 -->/'  /etc/xdg/templates/_root_.jwmrc > /tmp/result
If /tmp/results looks the way you expect then

Code: Select all

mv /tmp/result  /etc/xdg/templates/_root_.jwmrc
or add option -i after sed (as in your script).

If it doesn't work try replacing each backslash into a double backslash and running the command again. If it still doesn't work... sorry, I messed up!
Last edited by step on Thu 22 Jan 2015, 23:07, edited 1 time in total.
[url=http://murga-linux.com/puppy/viewtopic.php?t=117546]Fatdog64-810[/url]|[url=http://goo.gl/hqZtiB]+Packages[/url]|[url=http://goo.gl/6dbEzT]Kodi[/url]|[url=http://goo.gl/JQC4Vz]gtkmenuplus[/url]
Top
User avatar
rcrsn51
Posts: 13096
Joined: Tue 05 Sep 2006, 13:50
Location: Stratford, Ontario

#3 Post by rcrsn51 »

In order to use variables in your sed command, you must use double quotes instead of single quotes.
Top
User avatar
ASRI éducation
Posts: 3197
Joined: Sat 09 May 2009, 12:10
Location: France
Contact:
Contact ASRI éducation

#4 Post by ASRI éducation »

@ step and rcrsn51
Thank you for your answers, I will test today.
Projet ASRI éducation => [url=http://asri-education.org/]Association[/url] | [url=http://forum.asri-education.org/]Forum[/url] | [url=http://dl01.asri-education.org/]Dépôt[/url] | [url=http://kids.asri-education.org/]Espace kids[/url]
Top
User avatar
ASRI éducation
Posts: 3197
Joined: Sat 09 May 2009, 12:10
Location: France
Contact:
Contact ASRI éducation

#5 Post by ASRI éducation »

rcrsn51 wrote:In order to use variables in your sed command, you must use double quotes instead of single quotes.
Hello rcrsn51.
I tried.

Code: Select all

#!/bin/sh
#Script to hide category Desktop in main menu of Puppy (with WM jwm)

## Actual category (example = dektop) ==> Visible
GREPDESKTOP=`grep puppy-desktop.menu /etc/xdg/templates/_root_.jwmrc`
OLDVALUE=$GREPDESKTOP

## New category ==> Not visible
HIDE="<!-- PUPPYMENU jwm-xdgmenu /etc/xdg/menus/puppy-desktop.menu -->"
NEWVALUE=$HIDE

## Command sed (to replace OLDVALUE by NEWVALUE)
sed -i "s/$OLDVALUE/$NEWVALUE/" /etc/xdg/templates/_root_.jwmrc

exit
I get this error message.

Code: Select all

sed: -e expression #1, char 32: unknown option to `s'
Script completed hit RETURN to close window.
Projet ASRI éducation => [url=http://asri-education.org/]Association[/url] | [url=http://forum.asri-education.org/]Forum[/url] | [url=http://dl01.asri-education.org/]Dépôt[/url] | [url=http://kids.asri-education.org/]Espace kids[/url]
Top
User avatar
ASRI éducation
Posts: 3197
Joined: Sat 09 May 2009, 12:10
Location: France
Contact:
Contact ASRI éducation

Re: Need help for command 'sed'

#6 Post by ASRI éducation »

step wrote:

Code: Select all

sed '/puppy-desktop\.menu/s/\(.*\)/<!-- \1 -->/'  /etc/xdg/templates/_root_.jwmrc > /tmp/result
Hello step.
Thank you. Your code works!

Code: Select all

<!--   PUPPYMENU jwm-xdgmenu /etc/xdg/menus/puppy-desktop.menu -->
Still a small question.
I want to associate this function to buttons 'checkbox' (see screenshot in the first post).
So I need to add "<!--" before and a "-->" => now I know how to do.
I also need to remove "<!--" before and a "-->" which were added => I do not know how, but I guess you have the solution.

Thank you in advance.
Regards,
Projet ASRI éducation => [url=http://asri-education.org/]Association[/url] | [url=http://forum.asri-education.org/]Forum[/url] | [url=http://dl01.asri-education.org/]Dépôt[/url] | [url=http://kids.asri-education.org/]Espace kids[/url]
Top
User avatar
Geoffrey
Posts: 2355
Joined: Sun 30 May 2010, 08:42
Location: Queensland

#7 Post by Geoffrey »

ASRI éducation wrote:

Code: Select all

## Command sed (to replace OLDVALUE by NEWVALUE)
sed -i "s/$OLDVALUE/$NEWVALUE/" /etc/xdg/templates/_root_.jwmrc
Shouldn't that be

Code: Select all

sed -i 's/"$OLDVALUE"/"$NEWVALUE"/' /etc/xdg/templates/_root_.jwmrc
[b]Carolina:[/b] [url=http://smokey01.com/carolina/pages/recent-repo.html]Recent Repository Additions[/url]
[img]https://dl.dropboxusercontent.com/s/ahfade8q4def1lq/signbot.gif[/img]
Top
User avatar
ASRI éducation
Posts: 3197
Joined: Sat 09 May 2009, 12:10
Location: France
Contact:
Contact ASRI éducation

#8 Post by ASRI éducation »

Geoffrey wrote:Shouldn't that be

Code: Select all

sed -i 's/"$OLDVALUE"/"$NEWVALUE"/' /etc/xdg/templates/_root_.jwmrc
Hello Geoffrey.
I just tried your proposal.

Code: Select all

#!/bin/sh 
#Script to hide category Desktop in main menu of Puppy (with WM jwm) 

## Actual category (example = dektop) ==> Visible 
GREPDESKTOP=`grep puppy-desktop.menu /etc/xdg/templates/_root_.jwmrc` 
OLDVALUE=$GREPDESKTOP 

## New category ==> Not visible 
HIDE="<!-- PUPPYMENU jwm-xdgmenu /etc/xdg/menus/puppy-desktop.menu -->" 
NEWVALUE=$HIDE 

## Command sed (to replace OLDVALUE by NEWVALUE)
sed -i 's/"$OLDVALUE"/"$NEWVALUE"/' /etc/xdg/templates/_root_.jwmrc

exit
This avoids the error message, but it does not give the desired result: the file _root_.jwmrc is not modified, OLDVALUE is not replaced by NEWVALUE.
Regards
Projet ASRI éducation => [url=http://asri-education.org/]Association[/url] | [url=http://forum.asri-education.org/]Forum[/url] | [url=http://dl01.asri-education.org/]Dépôt[/url] | [url=http://kids.asri-education.org/]Espace kids[/url]
Top
User avatar
L18L
Posts: 3479
Joined: Sat 19 Jun 2010, 18:56
Location: www.eussenheim.de/

Re: Need help for command 'sed'

#9 Post by L18L »

ASRI éducation wrote:I also need to remove "<!--" before and a "-->" which were added => I do not know how, but I guess you have the solution.

Code: Select all

# bla='<!--  hello ASRI -->'
# echo ${bla}
<!-- hello ASRI -->
# 
# new_bla=${bla//\<\!-- }; new_bla=${new_bla// --\>}
# echo $new_bla
hello ASRI
#
(copied from console)

some charactere need escaping: \
hope that helps (c'est sans sed :wink: )


EDIT
http://sed.sourceforge.net/#tools
Last edited by L18L on Sat 24 Jan 2015, 14:37, edited 1 time in total.
Top
User avatar
Geoffrey
Posts: 2355
Joined: Sun 30 May 2010, 08:42
Location: Queensland

#10 Post by Geoffrey »

ASRI éducation wrote:

Code: Select all

## Command sed (to replace OLDVALUE by NEWVALUE)
sed -i 's/"$OLDVALUE"/"$NEWVALUE"/' /etc/xdg/templates/_root_.jwmrc
This avoids the error message, but it does not give the desired result: the file _root_.jwmrc is not modified, OLDVALUE is not replaced by NEWVALUE.
Regards
This works

Code: Select all

sed -i 's/'"$OLDVALUE"'/'"$NEWVALUE"'/'
[b]Carolina:[/b] [url=http://smokey01.com/carolina/pages/recent-repo.html]Recent Repository Additions[/url]
[img]https://dl.dropboxusercontent.com/s/ahfade8q4def1lq/signbot.gif[/img]
Top
User avatar
MochiMoppel
Posts: 2084
Joined: Wed 26 Jan 2011, 09:06
Location: Japan

#11 Post by MochiMoppel »

Hide a menu item:

Code: Select all

ITEM="puppy-desktop.menu"
Q1="<!-- "
Q2=" -->"
sed  -i "s/.*$ITEM.*/$Q1&$Q2/"  /etc/xdg/templates/_root_.jwmrc
Unhide:

Code: Select all

sed -i  "/$ITEM/s/\($Q1\)\(.*\)\($Q2\)/\2/"  /etc/xdg/templates/_root_.jwmrc
EDIT
If you use GNU sed - as all (?) Puppies do - you can make the latter even shorter.

Unhide:

Code: Select all

sed -ri  "/$ITEM/s/$Q1(.*)$Q2/\1/"  /etc/xdg/templates/_root_.jwmrc
_
Last edited by MochiMoppel on Sun 25 Jan 2015, 06:38, edited 1 time in total.
Top
step
Posts: 1349
Joined: Fri 04 May 2012, 11:20

Re: Need help for command 'sed'

#12 Post by step »

ASRI éducation wrote: So I need to add "<!--" before and a "-->" => now I know how to do.
I also need to remove "<!--" before and a "-->" which were added => I do not know how, but I guess you have the solution.
One idea: when you add the <!-- before and --> after: MARK them in a special way. Then you can use the special mark to identify which lines to unhide.
hide

Code: Select all

sed '/puppy-desktop\.menu/s/\(.*\)/<!-- MYMARK \1 -->/'  /etc/xdg/templates/_root_.jwmrc > /tmp/result
unhide (notice option -r)

Code: Select all

sed -r '/ MYMARK /s/<!-- MYMARK (.+) -->/\1/'  /etc/xdg/templates/_root_.jwmrc > /tmp/result
Note that " MYMARK " must not appear anywhere else but in the lines that you want to change. So choose MYMARK judiciously. This answers your question.

When you use shell variables in the sed command (script) you need to be very careful. If your shell variables contain characters that are special to sed or to the regex engine, you may get a hard-to-trace bug.
bad example 1

Code: Select all

F="/hi there/" # notice two slashes
R="/salut il" # notice one slash
sed '/this line/s/'"$F"/"$R"/ /dev/null

sed: -e expression #1, char 24: unknown option to `s'
That happens because the shell expands the sed line into

Code: Select all

sed '/this line/s//hi there///salut il/' /dev/null
and the abundance of slashes trips sed.
bad example 2

Code: Select all

F="***my cool menu***" # notice ***
R="my cool menu"
sed '/this line/s/'"$F"/"$R"/ /dev/null

sed: -e expression #1, char 33: Invalid preceding regular expression
That happens because * has a special meaning to sed, and using *** that way violates sed rules.
bad example 3

Code: Select all

F="File Editor"
R="File &Editor"
echo "this line $F" | sed '/this line/s/'"$F"/"$R"/

this line File File EditorEditor
You probably expected to see "this line File &Editor" as the output of the sed command, but it didn't work that way. That happens because & has a special meaning to sed.

Notice that the list of special characters is larger than just / * &.

So when you use sed with shell variables you expose your script to potential bugs. What can you do about it?
1. Ignore the problem (bad)
2. Ensure that all special characters will never be input (set rules and hope that everyone will follow them, unrealistic)
3. Sanitize all input before passing it to sed as shell variables (more programming).

The last option can be implemented with the tr command, but it comes with some limitations. The idea is to replace all special characters with a non-special character that sed likes.

Code: Select all

special="/*&"
F="/slash/ and  ***star*** issues"
R="&ampersand issue"
F=`echo "$F" | tr "$special" '~'`
R=`echo "$R" | tr "$special" '~'`
echo "this line $F" | sed '/this line/s/'"$F"/"$R"/

this line ~ampersand issue
sed didn't complain and it replaced $F with $R. However all your special characters were turned into tilde. If you can live with that, this is an easy solution. If you can't then consider replacing sed with awk, which has its own syntax and programming rules (more work) or perl, which can quote literals with \Q \E (but not every puppy has perl installed).
[url=http://murga-linux.com/puppy/viewtopic.php?t=117546]Fatdog64-810[/url]|[url=http://goo.gl/hqZtiB]+Packages[/url]|[url=http://goo.gl/6dbEzT]Kodi[/url]|[url=http://goo.gl/JQC4Vz]gtkmenuplus[/url]
Top
User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:
Contact technosaurus

#13 Post by technosaurus »

You should also check out I18n's localized config script in my jwm tools thread. I'm planning a significant update for Joe's recent updates soon.
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].
Top
User avatar
Geoffrey
Posts: 2355
Joined: Sun 30 May 2010, 08:42
Location: Queensland

#14 Post by Geoffrey »

Geoffrey wrote:
ASRI éducation wrote:

Code: Select all

## Command sed (to replace OLDVALUE by NEWVALUE)
sed -i 's/"$OLDVALUE"/"$NEWVALUE"/' /etc/xdg/templates/_root_.jwmrc
This avoids the error message, but it does not give the desired result: the file _root_.jwmrc is not modified, OLDVALUE is not replaced by NEWVALUE.
Regards
This works

Code: Select all

sed -i 's/'"$OLDVALUE"'/'"$NEWVALUE"'/'
I actually tried the script this time, the sed line needs to be

Code: Select all

sed -i "s|$OLDVALUE|$NEWVALUE|" /tmp/_root_.jwmrc
The input file:

Code: Select all

<?xml version="1.0"?>

<JWM>

 <!-- IMPORTANT, ONLY EDIT /etc/xdg/templates/_root_.jwmrc -->
 <!-- jwm menu for puppy Linux -->
 <RootMenu label="Menu" labeled="false" height="24" onroot="3">
 

  PUPPYMENU jwm-xdgmenu /etc/xdg/menus/puppy-utility.menu
  PUPPYMENU jwm-xdgmenu /etc/xdg/menus/puppy-filesystem.menu
  PUPPYMENU jwm-xdgmenu /etc/xdg/menus/puppy-graphic.menu
  PUPPYMENU jwm-xdgmenu /etc/xdg/menus/puppy-office.menu
  PUPPYMENU jwm-xdgmenu /etc/xdg/menus/puppy-network.menu
  PUPPYMENU jwm-xdgmenu /etc/xdg/menus/puppy-internet.menu
  PUPPYMENU jwm-xdgmenu /etc/xdg/menus/puppy-multimedia.menu
  PUPPYMENU jwm-xdgmenu /etc/xdg/menus/puppy-fun.menu
  PUPPYMENU jwm-xdgmenu /etc/xdg/menus/puppy-development.menu
  PUPPYMENU jwm-xdgmenu /etc/xdg/menus/puppy-favorites.menu

  <Separator/>

  <Program label="Help" icon="help24.png">exec defaulthtmlviewer file:///usr/share/doc/index.html</Program>
  <Program label="Control Panel" icon="configuration24.png">exec wizardwizard</Program>
  <Program label="Shutdown" icon="shutdown24.png">exec shutdown-gui</Program>


 </RootMenu>
the result

Code: Select all

<?xml version="1.0"?>

<JWM>

 <!-- IMPORTANT, ONLY EDIT /etc/xdg/templates/_root_.jwmrc -->
 <!-- jwm menu for puppy Linux -->
 <RootMenu label="Menu" labeled="false" height="24" onroot="3">
 
<!-- PUPPYMENU jwm-xdgmenu /etc/xdg/menus/puppy-desktop.menu -->
  PUPPYMENU jwm-xdgmenu /etc/xdg/menus/puppy-system.menu
  PUPPYMENU jwm-xdgmenu /etc/xdg/menus/puppy-setup.menu

  <Separator/>

  PUPPYMENU jwm-xdgmenu /etc/xdg/menus/puppy-utility.menu
  PUPPYMENU jwm-xdgmenu /etc/xdg/menus/puppy-filesystem.menu
  PUPPYMENU jwm-xdgmenu /etc/xdg/menus/puppy-graphic.menu
  PUPPYMENU jwm-xdgmenu /etc/xdg/menus/puppy-document.menu
  PUPPYMENU jwm-xdgmenu /etc/xdg/menus/puppy-calculate.menu
  PUPPYMENU jwm-xdgmenu /etc/xdg/menus/puppy-personal.menu
  PUPPYMENU jwm-xdgmenu /etc/xdg/menus/puppy-network.menu
  PUPPYMENU jwm-xdgmenu /etc/xdg/menus/puppy-internet.menu
  PUPPYMENU jwm-xdgmenu /etc/xdg/menus/puppy-multimedia.menu
  PUPPYMENU jwm-xdgmenu /etc/xdg/menus/puppy-fun.menu

  <Separator/>

  <Program label="Help" icon="help24.png">exec defaulthtmlviewer file:///usr/share/doc/index.html</Program>

  <Menu label="Shutdown" icon="shutdown24.png" height="16">
   <!-- <Exit confirm="false" label="Exit to prompt" icon="prompt16.xpm" /> -->
   <Program label="Exit to prompt" icon="prompt16.xpm">exec wmexit</Program>
   <Program label="Reboot computer" icon="mini-turn.xpm">exec wmreboot</Program>
   <Program label="Power-off computer" icon="mini-stop.xpm">exec wmpoweroff</Program>
   <Program label="Restart X server" icon="mini-x.xpm">restartwm</Program>
   <Restart label="Restart JWM" icon="mini-windows.xpm"/>
  <!-- <Program label="Change Window Manager" icon="mini-windows.xpm">changewm.sh</Program> -->
  </Menu>

 </RootMenu>
Is this the required output ?
[b]Carolina:[/b] [url=http://smokey01.com/carolina/pages/recent-repo.html]Recent Repository Additions[/url]
[img]https://dl.dropboxusercontent.com/s/ahfade8q4def1lq/signbot.gif[/img]
Top
User avatar
dejan555
Posts: 2798
Joined: Sun 30 Nov 2008, 11:57
Location: Montenegro
Contact:
Contact dejan555

#15 Post by dejan555 »

Yes, that should work, because there are pathnames in variable and sed uses slash (/) for separator it errors out, but you can use other separators in sed to avoid errors, your pipe (|) character solves the problem, you can use a lot of different characters for separator this way to avoid escaping. it's only important that variable's values don't have same characters in them, all of these should work:

Code: Select all

sed -i "s:$OLDVALUE:$NEWVALUE:" /etc/xdg/templates/_root_.jwmrc
sed -i "s=$OLDVALUE=$NEWVALUE=" /etc/xdg/templates/_root_.jwmrc
sed -i "s+$OLDVALUE+$NEWVALUE+" /etc/xdg/templates/_root_.jwmrc
See what I mean?
puppy.b0x.me stuff mirrored [url=https://drive.google.com/open?id=0B_Mb589v0iCXNnhSZWRwd3R2UWs]HERE[/url] or [url=http://archive.org/details/Puppy_Linux_puppy.b0x.me_mirror]HERE[/url]
Top
User avatar
ASRI éducation
Posts: 3197
Joined: Sat 09 May 2009, 12:10
Location: France
Contact:
Contact ASRI éducation

#16 Post by ASRI éducation »

@ step, rcrsn51, Geoffrey, L18L, MochiMoppel, technosaurus, dejan555.

It is rare that I use the forum to ask for help about programming. But, I was really stuck by this command 'sed'.
Through your various answers, I have a functional script today !!!
This script will allow me to propose an essential utility that will be integrated into the next version of the puplet ASRI Edu (an edutainment version of Puppy that I will talk to you soon).

In addition to the answer to my problem, you have given me possibilities/ideas about sed, that I will use in future scripts. Together, I think that your answers will be of interest to other users/contributors.

I do not know if you understand my poor English (partially translated with an online application), but I want to say "A big thank you to all".

Regards,
Projet ASRI éducation => [url=http://asri-education.org/]Association[/url] | [url=http://forum.asri-education.org/]Forum[/url] | [url=http://dl01.asri-education.org/]Dépôt[/url] | [url=http://kids.asri-education.org/]Espace kids[/url]
Top
Post Reply

Return to “Programming”