How to convert a savefile to sfs?

Using applications, configuring, problems
Post Reply
Message
Author
maans
Posts: 55
Joined: Thu 07 Jan 2010, 18:56
Location: Denmark

How to convert a savefile to sfs?

#1 Post by maans »

Hi Puppians (and PuppyWizards)

I've been remastering Puppy derivatives for a couple of years now - and I LOVE IT :D

Now I'd like to be able to assemble collections of apps, settings etc. into SFS (or pet) files - instead of doing a full remaster for every variant...

I have everything set for remastering. In different folders I have the pristine system.sfs file and a personal storage file for each variant.

Is there a simple way to convert a save/storage file into a SFS or pet file directly - or do I have to go through the whole installation/configuring process once more (for every and each of my storefiles), to be able to use tools like new2dir and dir2pet?

Cheers Måns Mårtensson

User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

#2 Post by greengeek »

I discussed something a little bit similar with technosaurus and he mentions risks:
http://www.murga-linux.com/puppy/viewto ... &start=155
He might still have the remaster code he used for this.

User avatar
jrb
Posts: 1536
Joined: Tue 11 Dec 2007, 19:56
Location: Smithers, BC, Canada

#3 Post by jrb »

I too got tired of remastering every time a new Puppy came out so I came up with an alternative which I discuss in Empowering the Zdrv
.
This amounts to creating one SFS with all your favourite .pets and then adapting it to each new Puppy. With the aid of Edit-SFS 2.1 (squash filesystem editor) its quite easy to do. :D

maans
Posts: 55
Joined: Thu 07 Jan 2010, 18:56
Location: Denmark

Re:SFS as an alternative to remastering

#4 Post by maans »

Hi
greengeek wrote:I discussed something a little bit similar with technosaurus and he mentions risks:
http://www.murga-linux.com/puppy/viewto ... &start=155
@greengeek
Thanks! I've read the post - and I understand technosaurus' concerns.
I just made a pet from a mounted save-file with "dir2pet" and it works well, because it copies everything - including symlinks etc...
When I want to remove things from the pet - doing "pet2tgz" and extract it to a folder - symlinks (and stuff?) are automatically removed and when I compile a new pet from the extracted folder I get a pet which is MUCH bigger - even if I removed a lot of things....
Then it's easier to stick with the first pet - made directly from the save-file, even if it's full of unneccesary stuff...
I'd love to know how to do it the "right" way though...

jrb wrote:... creating one SFS with all your favourite .pets and then adapting it to each new Puppy. With the aid of Edit-SFS 2.1 (squash filesystem editor) its quite easy to do. :D
@jrb
Thanks!
If I understand the procedure correctly, I should/could use any SFS file, empty it - and put in any file - directory structure (ie contents from a pet or my save file) - and it will (re)write the SFS to have my files in it?!
The idea of "renaming puppy_xxx.sfs to zdrv_puppy_xxx.sfs and rename the generic SFS to puppy_xxx.sfs" - is compelling and just might be something to strive for, however I still haven't made my first SFS file yet - and I'm struggeling to get a "head and tails" of a possible simple workflow for creating pet-packages out of already successfully installed applications. I guess I will get there at some point.

Where I'm at - with my save-file, which I want to collect into a SFS file:
I installed the Node.js package**, Chromium, and tws (a Node.js server made for running TiddlyWiki* files=single file html/javascript apps) through port 1337. I made an AppStarter for TiddlyWiki ressources. I also installed a Chromium extension for the tws/server and made the system start up with the tws-server and the AppStarter. There are also a couple of desktop files - one for the AppStarter and another one for opening a TiddlyWiki as a Chromium app,

I guess that I should be able to convert my pet file (made from a mounted save-file) into a SFS file. (?)
Then I should be able to use Edit-SFS 2.1 (EDIT: Hmm - Edit-SFS 2.1 doesn't seem to be compatible with Precise 5.4.3. It won't install... ) to edit the contents of the SFS file, without breaking symlinks etc... (I simply want to remove most of the unneccesary files copied from the original save-file)....

Please tell, if you know a better way to do it.

Thanks for helping me out here - so far it seems to be possible to do what I want to do, however I'm not sure if I should change my work flow and try to install Node.js, Chromium - create my .sh and .desktop files while I'm running a new2dir command.... haven't tried that yet - as I don't quite understand how I get changes I make (icons, my-applications etc) get into the equation?

Cheers Måns

*http://tiddlywiki.com
**http://nodejs.org/

Edit: Here's the pet I've compiled from my save-file: (tws4precise.pet (63.7 MB)) DOWNLOAD
Now I've made a SFS file like this:

Code: Select all

mksquashfs <path_to_directory> <path_to_directory>.sfs -noappend
where my mounted save-file is the first directory and the second directory is on my usb stick.
The sfs file is a little smaller than the pet file (59 MB) and I haven't tested it yet... DOWNLOAD

maans
Posts: 55
Joined: Thu 07 Jan 2010, 18:56
Location: Denmark

Re:SFS as an alternative to remastering

#5 Post by maans »

Am I on the right track with this workflow?:
1) Install individual packages to individual directories by:
1.1) Downloading all neccesary files/packages
1.2) Installing to individual directories with:

Code: Select all

new2dir make install
2) Convert each dir/installation to a pet with: dir2pet
3) Copy all pets to one dir and run

Code: Select all

mksquashfs <path_to_directory> <path_to_directory>MyCollection.sfs -noappend
OR

Should I skip 2 (creating pets) and merge all directories into one folder and create a SFS directly from it?

Any advice is most welcome :-)

Cheers Måns

maans
Posts: 55
Joined: Thu 07 Jan 2010, 18:56
Location: Denmark

Re:SFS as an alternative to remastering

#6 Post by maans »

Hmm
maans wrote: 1) Install individual packages to individual directories by:
1.1) Downloading all neccesary files/packages
1.2) Installing to individual directories with:

Code: Select all

new2dir make install
I don't have success with new2dir make install - not with debs nor with unpacked tar.gz - files... All I get is empty -486 directories... No error messages...

Do I need to load devx.sfs to make it happen?

I don't know how to exploit the fact that all information is available in /root/.packages/user-installed-packages...

To me it seems that it should be possible to create pets from these installed packages - getting everything copied into one directory and do the magic dir2pet command??

Any advice is most welcome :-)

Cheers Måns

Ps: I'm working with Puppy Precise 5.4.3

User avatar
Flash
Official Dog Handler
Posts: 13071
Joined: Wed 04 May 2005, 16:04
Location: Arizona USA

#7 Post by Flash »

Since this thread is basically about remastering, I'd be remiss not to bring up multisession Puppy. :lol:
Unless you intend to install your remastered Puppy or run it from a USB flash drive, creating a multisession Puppy DVD is the easiest and simplest way to customize Puppy Linux. Running Puppy from a multisession DVD or CD has the added benefit that a record of all changes to the basic Puppy is preserved on the DVD for forensic purposes.
Last edited by Flash on Wed 27 Feb 2013, 02:00, edited 1 time in total.

maans
Posts: 55
Joined: Thu 07 Jan 2010, 18:56
Location: Denmark

#8 Post by maans »

Hi Flash
Flash wrote:Since this thread is basically about remastering, I'd be remiss not to bring up multisession Puppy. :lol:
Thanks :P
I am mostly working from an usb pfix=ram - or a folder with no savefile.
I use multiboot from same usb, where I have different puplets (mostly my own remasters)... I use Grub4dos to create a menu.lst file - and delete the uuid lines - to make it bootable on any pc...

I wonder - Where is the temporary storage to be found on the running live-system?

I'd like to be able to install packages on a running livesystem (no save-file), find the user-installed packages/folders and copy the structures to a folder which I can run dir2pet from...

Cheers Måns

User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

#9 Post by greengeek »

Just wild guessing here - would it be one of the pup_ro folders you can see in /initrd? Does the addition of a new package change the contents of one of those folders? Does each folder represent a layer of the filesystem, and the pup_rw represents the merged filesystem?

(you can tell I'm stabbing in the dark...)

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#10 Post by sunburnt »

Hi maans; You don`t say what kind of pkgs. you want to use, SFS would work as is.

It seems to me that the easiest way to do what you want is with a temp. Save file.
Install pkgs. to an empty Save file and then run dir2pet on the Save file mount dir.
When done clean the Save file for the next install. Reboot with no Save file.


greengeek; Good guess, the *_ro dirs. are SFS file mounts, and the *_rw is the Save files.
And /initrd/mnt/dev_save is the mounted partition with the Puppy files.

maans
Posts: 55
Joined: Thu 07 Jan 2010, 18:56
Location: Denmark

Creating a pet from pup_rw (in a livesession)

#11 Post by maans »

Hi sunburnt & greengeek
sunburnt wrote: It seems to me that the easiest way to do what you want is with a temp. Save file.
Install pkgs. to an empty Save file and then run dir2pet on the Save file mount dir.
----
... the *_rw is the Save files.
I did this to create pets in a livesession:
1) Installed applications, I wanted, made changes to the system.
2) Navigated to /initrd/
3) Opened a consol and wrote:

Code: Select all

dir2pet pup_rw
.
4) I got a pet file called "pup_rw" (it's called pup_rw no matter what you decide in the "buildprocess").
5) Wrote

Code: Select all

pet2tgz pup_rw.pet
i consol, to get a .gz fil.
6) Opened pup_rw.gz with XArchiver (Do NOT extract it to a folder - it will delete all symlinks!!!)
7) Deleted all files and folders I didn't need from the archive.
8) Executed

Code: Select all

tgz2pet pup_rw.gz
9) (Remember to save the pet-fil to usb or hdd)
10) Booted into a new live-session to test if everything works as intended...

I've made a pet-file, which installs:
1) Node.js + tws
2) Chromium + tws extension
3) Makes Chromium the standardbrowser
4) Puts programlaunchers in /root/Startup and /usr/share/applications

If you find one of them (and click on it) you don't have to restart x to start the tws-server. (It's possible to do this with a "postinstallscript" (postinstall.init??) - however I haven't found out how to do that yet :oops: )
The Pet-file is available from here: http://twsos.tiddlyspace.gir.dk/#Pets
+ a couple of videos/screencasts made with ffconvert: twsos - youtube channel

Thank you very much for your kind support :D

Cheers Måns

maans
Posts: 55
Joined: Thu 07 Jan 2010, 18:56
Location: Denmark

Re: Creating a pet from pup_rw (in a livesession)

#12 Post by maans »

maans wrote: If you find one of them (and click on it) you don't have to restart x to start the tws-server. (It's possible to do this with a "postinstallscript" (postinstall.init??) - however I haven't found out how to do that yet :oops: )
Now I made a pinstall.sh script, which I want to put into the pet file:

Code: Select all

#!/bin/sh
#post-install script.
#creatuppy: current directory is rootfs-complete, which has the final filesystem.
#pupget: current directory is /.
#Startup tws server ...
./root/Web-Server/TWS-NewTabs/starttws.sh
I tried converting the pet file to a tar.gz file - and added the pinstall.sh file - then I converted it back to a pet again.
It doesn't seem to work.

The pinstall script is put "above" the "pup_rw" folder level - and it doesn't work....
The difficult part seems to be figuring out how to extract the tar.gz file to a folder without loosing all symlinks...

Anyone knows how to to that - or how to put the script into the right folder in Xarchiver?

Cheers Måns
Attachments
capture31266.png
(78.83 KiB) Downloaded 1362 times

User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

#13 Post by greengeek »

Hey, well done. Sounds like you are almost there. Unfortunately I don't have enough experience to know the answer to your question. I have struggled for a few months to avoid losing symlinks when extracting/remastering and the only answer I came up with was that I had to increase the size of my savefile and do everything within /root.

It seemed like I had problems with system permissions and using /root as the workspace for everything was the only way I could preserve the permissions for symlinks during copying and extracting.

I recall technosaurus advising me to do a "cp -arp" but I don't think I was using it correctly as I didn't get the results I wanted with that syntax.

I have noticed a lot of the experienced guys referring to "chroot" and I wonder if that has something to do with preserving permissions/symlinks etc? Hope someone can help.

maans
Posts: 55
Joined: Thu 07 Jan 2010, 18:56
Location: Denmark

Made the pinstall.sh work

#14 Post by maans »

Hi greengeek
greengeek wrote:... answer I came up with was that I had to increase the size of my savefile and do everything within /root.

It seemed like I had problems with system permissions and using /root as the workspace for everything was the only way I could preserve the permissions for symlinks during copying and extracting.
Great :D
Thanks!! I will try your suggestion for my next project.

I was a little impatient - so I went through the whole process of installing the package to a live system, copied the pinstall.sh script into the pup_rw folder and did the (really time consuming) manual work of removing unwanted folders and files from the tar.gz file...
NOW IT WORKS WITH THE POST INSTALL SCRIPT !! :D
download tws-node.js-chromium2.pet

Cheers Måns

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#15 Post by sunburnt »

The Save file /initrd/pup_rw should be clean I think ( but maybe not ).
So you should be able to make Pets or SFS pkgs. from it without deleting.

However, if it is polluted, then modify the Puppy boot code and add another layer to it.
The new layer could better yet be a dir. instead of a Save file.

Puppy use to have problems with SFS files, but now that it`s Ubuntu based
Puppy now has a good tool chain if all Ubuntu files are used.
SFS files are far better than Pet files, much like no-install apps.

User avatar
tallboy
Posts: 1760
Joined: Tue 21 Sep 2010, 21:56
Location: Drøbak, Norway

#16 Post by tallboy »

greengeek wrote:I recall technosaurus advising me to do a "cp -arp" but I don't think I was using it correctly as I didn't get the results I wanted with that syntax.
You could check out 'cpio' as a possible alternative, especially it's 'Copy-pass mode'.

tallboy
True freedom is a live Puppy on a multisession CD/DVD.

nooby
Posts: 10369
Joined: Sun 29 Jun 2008, 19:05
Location: SwedenEurope

#17 Post by nooby »

sunburnt you wrote Puppy use to have problems with SFS files, but now that it`s Ubuntu based
so what about Puppy Slacko using Slackware files?
I use Google Search on Puppy Forum
not an ideal solution though

Post Reply