SFS-TCZ_Linker-2.2.pet

Miscellaneous tools
Message
Author
User avatar
jrb
Posts: 1536
Joined: Tue 11 Dec 2007, 19:56
Location: Smithers, BC, Canada

#31 Post by jrb »

Dingo wrote:when sfslinker installs itself, it seems de-associate icons on pinboard items so links are broken
Sorry, I forgot to mention that happening in my last post.

I seem to have found an easy fix however. Just go to /root/.config/rox.sourceforge.net/ROX-Filer/ and delete the globicons file. When you run your mouse over the desktop icons they should come back.

Apparently Puppy301 doesn't expect to find a globicons file there. In Puppy4XX globicons is hard linked between /root/Choices/ROX-Filer/globicons and /root/.config/rox.sourceforge.net/ROX-Filer/globicons

User avatar
jrb
Posts: 1536
Joined: Tue 11 Dec 2007, 19:56
Location: Smithers, BC, Canada

#32 Post by jrb »

BTW - what is that tcz folder for?
Now its my turn to be embarrassed. :oops: I forgot to put a readme file in that folder.

If you make a /mnt/home/tcz folder and keep your .tcz files there then symlinks will automatically be made to the /my_links/tcz folder at bootup, kind of like the /my_links/sfs_mnt_home folder. From there you can copy the links you want to /my_links/sfs_boot_links.

User avatar
WarMocK
Posts: 169
Joined: Thu 05 Jul 2007, 11:53

#33 Post by WarMocK »

Okay, this tool is made of awesome, thanks for creating and sharing it.
However, I found an unpleasant flaw in the design: I had GIMP-2.4.5 installed with a dotpet on my test system and then linked an SFS with the same version of GIMP into it as well. When I unlinked it, GIMP didn't work anymore. Obviously, the symlinks overwrote the installed binaries and got wiped when I unlinked the SFS. Okay, I simply re-installed GIMP afterwards, so nothing of value was lost, but this is a SERIOUS problem nevertheless. Apparently, SFS_linker does not test if there's already a link or binary with the same name available in the system and simply overwrites it. I don't want to imagine what happens if the SFS file contains system critical .so files ...
In my opinion SFS_linker desperately needs to read the content of the SFS, make a list from it, and then check if the system already has some of the files it would create symlinks for. If yes, SFS_linker needs to skip these links and remove them from the uninstall list when unlinking the module again.
In order to prevent a dependency hell when an older SFS is unlinked that already brought some shared libs needed by a newer SFS, it might also be a good idea to let the unlinker re-link the other sfs files to fill the gap. I know it would take some time, but it's better than if you lose half of your system because some critical programs won't start after you made an update.

User avatar
jrb
Posts: 1536
Joined: Tue 11 Dec 2007, 19:56
Location: Smithers, BC, Canada

#34 Post by jrb »

Hi WarMock,
Sorry to hear about your difficulty. You raise a good point but I'm not sure there is a good fix. :?
Obviously, the symlinks overwrote the installed binaries and got wiped when I unlinked the SFS.
Not quite. SFS_Linker will not create symlinks over the top of existing files but it keeps track of what symlinks and files it has created (or tried to create) and copied (or tried to copy) exactly as package manager does. It creates a list of the files in /root/.packages/XXX.files, sadly it does not distinguish between symlinks and files. When you uninstall the SFS it deletes these files and symlinks regardless of whether it created them or not.

For each file both SFS_Linker and package manager check with the original pup-xxx.sfs and if a file with the same name exists there it will reinstate it. (Only from liveCD or frugal installs, full installs suffer the full consequences) However no such mechanism exists for previously installed .pets or .sfs's. Uninstalling and then reinstalling the original .pet is the only fix at present.

If you have added truly system critical files then you might want to think about remastering the pup-xxx.sfs so they will be safe. This is now easily done with Pizzasgood's edit_sfs-2.1.pet.

This is one of the reasons I quit using full installs, too easy to delete critical files. :x

Hope this helps. I will continue to think about the problem, perhaps there is something that can be done.

Cheers, J

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#35 Post by technosaurus »

One way would be to relink all other loaded sfs files after an unlink - if you wanted to get all fancy you could check for duplicate filenames and only do those, but also a lot of trouble so it may be better to just have a "relink all" button or default setting or something of the like.
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

User avatar
WarMocK
Posts: 169
Joined: Thu 05 Jul 2007, 11:53

#36 Post by WarMocK »

@technosaurus: my thoughts exactly. ;-)

@jrb: Thanks for the clarification. At least I know what happened now. And don't worry, it wasn't more than a little inconvenience, relinking the sfs already did the job.
Another thing I noticed when I started working on the Beta 4 version of K-9 Linux: After installing SFS-Linker, I had big troubles with the Terminal: I couldn't close it normally. Instead, XFCE warned me that the program is busy and offered me to terminate the whole process. I don't know if this is an issue with puppy 4.32 (I had no such problems with NOP 4.31) of if I accidently left out a few important scripts or patches when I compiled the barebones system with Woof, but I thought I should let you know.

User avatar
jrb
Posts: 1536
Joined: Tue 11 Dec 2007, 19:56
Location: Smithers, BC, Canada

#37 Post by jrb »

technosaurus wrote:One way would be to relink all other loaded sfs files after an unlink
That is easily done but the real problem is SFS_Linker deleting files installed with .pets. Perhaps it could backup files that already exist and then reinstate them from backups.

User avatar
WarMocK
Posts: 169
Joined: Thu 05 Jul 2007, 11:53

#38 Post by WarMocK »

Or you could try to keep track of what symlinks were actually created and note them down in a separate text file. Upon unlinking, you could use that list instead (after checking if the symlink is still valid OR if it has been overwritten with a real file).

User avatar
jrb
Posts: 1536
Joined: Tue 11 Dec 2007, 19:56
Location: Smithers, BC, Canada

#39 Post by jrb »

WarMock wrote: I don't know if this is an issue with puppy 4.32
Can"t find the post but Barry mentioned that 432 had the "wrong" gtk+ version. I had problems with some of the SFS's and rewoofed it with gtk+-2.14.7-1-p4 as used in puppy431 and have had no problems since.

User avatar
WarMocK
Posts: 169
Joined: Thu 05 Jul 2007, 11:53

#40 Post by WarMocK »

Hmm, not good. Fortunately I still have a core system created from Puppy 4.31 here. :roll:

Oh, and for the XFCE users here:
I've written a little code snipplet that allows you to toggle the sfs_linker to load an unloaded sfs file and vice versa (it works with JWM/ROX as well, of course ;-)):

Code: Select all

#!/bin/sh

if [ `ls $HOME/my_links/sfs_loaded | grep $1` != "" ];then
   sfs_unlinker `ls $(pwd)/$1` 2> /dev/null &
else
   sfs_unlinker `ls $(pwd)/$1` 2> /dev/null &
fi

exit $@

seaside
Posts: 934
Joined: Thu 12 Apr 2007, 00:19

#41 Post by seaside »

jrb wrote: It creates a list of the files in /root/.packages/XXX.files, sadly it does not distinguish between symlinks and files. When you uninstall the SFS it deletes these files and symlinks regardless of whether it created them or not.
It could - by adding code to the unlinking script that would test if the file were a symlink (if -L file). If it wasn't then don't remove. This could save the removal of a critical file.

Copied files would remain - but there are usually only a couple of these and they wouldn't usually cause anything to fail.

Regards,
s

User avatar
WarMocK
Posts: 169
Joined: Thu 05 Jul 2007, 11:53

#42 Post by WarMocK »

Okay, bumping this thread up a bit because I'm trying to find a solution for the uninstall problem.
I wrote a tiny script that compares the index file of an installed SFS with all the other package index files, sorting out what is overwritten by an sfs or dotpet installed after the sfs you want to remove.
Technically, it moves the *.files file of the sfs from .packages to $HOME, dumps the content of each *.files file into a temporary file under /tmp, and then compares that file with the index file of the sfs, leaving only what would be added from the sfs index file into the temporary dump list.
I think that pretty much sorts out any files which might have been overwritten by either dotpets or other sfs files, so your unlinker would only have to remove the symlinks listed in the difference file generated by my script.
The script works damn fast, so it wouldn't add too much to the uninstallation time.
So, if you might want to give it a try:

Code: Select all

#!/bin/sh

mv $HOME/.packages/$1 $HOME
cat $HOME/.packages/*.files > /tmp/installedfiles
diff $HOME/$1 /tmp/installedfiles | grep / | grep -v ">" | cut -b 3- > /tmp/difference

I still need to figure out how to implement it into your unlinker script, among with the test for if the file to be removed is actually a symlink or not (as a safety measure). Do you have any documentation about how your script works in detail? That would help me a lot. ;-)

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#43 Post by technosaurus »

if you use ln -s instead of ln -s -f it will not overwrite existing files - if you set up a variable called
OVERWRITE="-f" #set to "" to avoid overwriting
It could even use an Xdialog to do on original linking
.... then something like
ln -s $OVERWRITE $SFSMNTPATH$ONEFILE $ONEFILE
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

User avatar
jrb
Posts: 1536
Joined: Tue 11 Dec 2007, 19:56
Location: Smithers, BC, Canada

#44 Post by jrb »

WarMock, I believe you've done it! :D I have incorporated your technique into /usr/local/bin/sfs_linker and done a few preliminary tests and it works!

I placed:

Code: Select all

cat /root/.packages/*.files > /tmp/installedpackagefiles #suggested by WarMock 22mar10
at the beginning (line 6) to generate a list of previously installed files.

I modified the .file generating code (line 96) to read:

Code: Select all

find ./$SFSNAME -mount -mindepth 1 | sed -e "$APATTERN" | sed -e 's/\/root0\//\/root\//g' > /tmp/$SFSNAME.files
and (line 98]

Code: Select all

find ./$SFSNAME -type l -mount -mindepth 1 | sed -e "$APATTERN" | sed -e 's/\/root0\//\/root\//g' >> /tmp/$SFSNAME.files
then used (line 100):

Code: Select all

diff /tmp/$SFSNAME.files /tmp/installedpackagefiles | grep / | grep -v ">" | cut -b 3- > /tmp/$SFSNAME.difference
to list only those new files installed by this package, and finally(line 101)

Code: Select all

cp /tmp/$SFSNAME.difference /root/.packages/$SFSNAME.files
to create the SFSNAME.files file in /root/.packages.

I created a .pet of the icons from one of my SFS's and installed it. Then used SFS_Linker to install the SFS. The icons were not listed in /root/.packages/SFS.files, although all the other files were, and were not uninstalled along with the SFS, perfect!

Using this technique /usr/local/bin/sfs_unlinker does not need to be changed and files copied from the SFS package can be removed safely at uninstall as opposed to just removing links.

I am attaching the modified /usr/local/bin/sfs_linker for you to test. Will repackage the .pet and post after further testing.

Thanks, J
Attachments
sfs_linker.gz
test file. Extract, set executable and place in /usr/local/bin
(1.77 KiB) Downloaded 690 times

User avatar
jrb
Posts: 1536
Joined: Tue 11 Dec 2007, 19:56
Location: Smithers, BC, Canada

#45 Post by jrb »

technosaurus,
This is the code that I used to create the symlinks from /mnt/SFSpackage to /

Code: Select all

cp -rs /mnt/"$SFSNAME"/* /
It doesn't overwrite other existing files or symlinks.

User avatar
WarMocK
Posts: 169
Joined: Thu 05 Jul 2007, 11:53

#46 Post by WarMocK »

You're welcome, man. ;-)
This means that now I can fully implement your script into K-9 Linux and embed my SFS management framework into the ISO.
Oh, and I just realized there's still one little issue with the unlinking process: if there's a petget running while you try to unlink a module, it might occur that the unlinker takes a snapshot of the required libs BEFORE petget managed to add the installing dotpet's *.files to the package directory. Needless to say, that would be pretty bad if this dotpet would contain files that overwrite some of the symlinks.
A simple solution would be an Xdialog window telling the user that they need to make sure there are no other installations running while the SFS is unlinked. Add a yes-no option to the mix ("yes" starts unlinking, "no" cancels the process), and the prob would be solved as well. Of course, you could also use ps to check the system for running installation processes and either pause your own script or tell linux to pause the petgets and block any new instances of petget right away.

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#47 Post by technosaurus »

jrb wrote:technosaurus,
This is the code that I used to create the symlinks from /mnt/SFSpackage to /

Code: Select all

cp -rs /mnt/"$SFSNAME"/* /
It doesn't overwrite other existing files or symlinks.
Wouldn't that generate an error message for files that exist that you could capture?
Something like

Code: Select all

cp -rs /mnt/"$SFSNAME"/* 2>>/tmp/DONOTUNLINK
then add a grep of /tmp/DONOTUNLINK in the unlink script (you may also need to use cut or something to remove the rest of the error message)

If the other way works though - that's great, just that sometimes find can be really slow
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

User avatar
WarMocK
Posts: 169
Joined: Thu 05 Jul 2007, 11:53

#48 Post by WarMocK »

Damn, I knew there was something fishy about the whole thing with the modification.
@jrb: I think that diff is not the best way to check out what files are already there. I made a new attempt using "sort" and "uniq", which removes duplicate lines from the filedump before comparing it to a second dump with the sfs list included. These two dumps are then compared with sort AND uniq, sending only the unique files of the SFS to the standard output. Several checks with my .packages folder were successful, and it worked DAMN fast!:D

The code is:

Code: Select all

cat $HOME/.packages/*.files | sort -u > /tmp/dump0
mv $HOME/.packages/$SFSFILE $HOME
cat $HOME/.packages/*.files | sort -u > /tmp/dump1
sort /tmp/dump0 /tmp/dump1 | uniq -u > /tmp/difference
mv $HOME/$SFSFILE $HOME/.packages
Add this one to both the linker AND the unlinker, and use the generated difference list for adding symlinks or removing them (after you verified that it is a symlink at all). Please keep in mind that this check only works with the FULL list of the SFS content, otherwise you might accidently skip files that were added or removed after the sfs file was linked.

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#49 Post by technosaurus »

@Warmock - sort |uniq is a nice useful combo especially with the latest busybox (use #! /bin/ash vs. #! /bin/sh)

this should give you the duplicates list in case you want an option for user override

Code: Select all

mv $HOME/.packages/$SFSFILE $HOME/$SFSFILE
cat $HOME/.packages/*.files | sort -u > /tmp/dump0
mv $HOME/$SFSFILE $HOME/.packages/$SFSFILE
cat $HOME/.packages/$SFSFILE /tmp/dump0 |sort |uniq -d > /tmp/duplicates #don't unlink automatically
#next line is optional depending on how you implement unlinking
cat $HOME/.packages/$SFSFILE /tmp/duplicates |sort |uniq -u > /tmp/difference #safe to unlink
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

seaside
Posts: 934
Joined: Thu 12 Apr 2007, 00:19

#50 Post by seaside »

I have alot of files that are not registered and probably others do as well.

This method of technosaurus's -

Code: Select all

cp -rs /mnt/"$SFSNAME"/* / 2>>DONOTUNLINK
would protect against removing files that should be kept and are not registered.

Another approach afterwards, might be something like this -

Copy SFSPACKAGE file to /tmp/SAFE-TO-UNLINK and then:

Code: Select all

for NAME in $(cat DONOTUNLINK)
do
  sed -ie "\|^$NAME\$|d" /tmp/SAFE-TO-UNLINK
done
This would match the linking errors (those files that couldn't be linked because they already existed) to the presumed installed SFSPACKAGE list and output a SAFE-TO-UNLINK list.

(Might even be faster - not sure, though.... :D :D )

s

Post Reply