Page 1 of 3

Yet another basic remaster script for frugal install

Posted: Mon 13 Mar 2017, 16:02
by nic007

Posted: Tue 28 Mar 2017, 04:50
by musher0
HI nic007.

On your copy line, wouldn't

Code: Select all

cp -a /initrd/pup_ro?/* /initrd/pup_rw/*  /mnt/home/puppyfilesystem
be ok?

Or does the pup_ro1 need to be copied after the others?

But very nice. Simpler remastering than this, you die! ;)

BFN.

Posted: Tue 28 Mar 2017, 06:54
by nic007
musher0 wrote:HI nic007.

On your copy line, wouldn't

Code: Select all

cp -a /initrd/pup_ro?/* /initrd/pup_rw/*  /mnt/home/puppyfilesystem
be ok?

Or does the pup_ro1 need to be copied after the others?

But very nice. Simpler remastering than this, you die! ;)

BFN.
pup_ro1 needs to be copied after the others as it holds the contents of the savefile when booting from usb flashdrive. pup_rw records the latest changes during a session so needs to be copied last. Where pup_ro1 is not in use your suggestion should work.

Re: Yet another basic remaster script for frugal install

Posted: Mon 29 May 2017, 17:23
by belham2
nic007 wrote:I use this quick and very basic script to remaster the base sfs sometimes so thought I'd just share it anyway. This should capture the contents of your savefile/folder (ie.programs installed, etc.) as well as the contents of extra sfs files loaded at the time. The working directory must be in a linux filesystem environment (a linux partition works best). Run the script immediately at desktop after booting with savefile/folder and loading any extra sfs files you want to include with your remaster before doing anything else. Note: this is an example script and you may need to change the paths in the script according to your setup. After the new sfs has been created, rename it to the correct name of your distribution's base sfs and move it to where your puppy files reside. Rename the old base sfs and the savefile to something like 1xxxxx (as backup should things not work out as expected). Reboot (without your savefile) and see if your remaster has been successful (afterwhich you can delete the old base sfs and savefile). I tested this with Racy. Puppy's newer than that will probably have the file .XLOADED in /root instead of /etc so you need to change that in the script if so. Hope this is useful to some.

Code: Select all

#!/bin/sh
mkdir /mnt/home/puppyfilesystem
cp -a /initrd/pup_ro2/* /initrd/pup_ro3/* /initrd/pup_ro4/* /initrd/pup_ro5/* /initrd/pup_ro6/* /initrd/pup_ro7/* /initrd/pup_ro8/* /initrd/pup_ro9/* /initrd/pup_rw/*  /mnt/home/puppyfilesystem
wait
cd /mnt/home/puppyfilesystem
rm -r ./initrd ./sys ./etc/.XLOADED
Xdialog -center -msgbox "Puppyfilesystem ready. Check and edit if needed. Press OK when ready" 0 0
mksquashfs /mnt/home/puppyfilesystem /mnt/home/new.sfs -noappend
wait 
rm -r /mnt/home/puppyfilesystem
Xdialog -center -title "SUCCESS!!!"  --no-buttons -infobox "Operation completed, exiting now" 0 0 3000 
   
exit
Note
The above code is for frugal install to harddrive. If you are booting and running puppy from flashdrive, you need to replace /initrd/pup_rw/* with /initrd/pup_ro1/* in the script (in the same order).
Hi nic007,

I just wanted to post and say 'thank you' for this script. I use it on everything, from old pups up to the very newest (just changing that part you noted in the script for .XLOADED). For me, since I do everything via "frugal" installs, your simple script allowing me to remaster the 'main-puppy.sfs' only is much better than trying to fully remaster a puppy. It s quicker, cleaner and gives me simple flexibility when setting other frugal installs/folders up.

Thank again. :wink:

Posted: Mon 29 May 2017, 20:32
by nic007
Glad you find it useful belham2. BTW - you can actually just add ./root/.XLOADED to the rm line of the script. The script will delete the applicable file depending on the Puppy you are running and the non-applicable one will not hinder the running of the script.

Re: Yet another basic remaster script for frugal install

Posted: Tue 30 May 2017, 06:10
by greengeek
nic007 wrote:

Code: Select all

#!/bin/sh
mkdir /mnt/home/puppyfilesystem....
I always run my puppy without any savefile and I don't have any /mnt/home - everything is done in ram.

Can you suggest any modification of this script to cover my situation? I would like to be able to grab any sfs files that are loaded.

Thus far I have been using an earlier version of one of your remaster scripts and it grabs pets that are loaded but not sfs files.

cheers!

EDIT : I just had a go at making some changes and came up with the following:

Code: Select all

#!/bin/sh 
mkdir /tmp/puppyfilesystem 
cp -a /initrd/pup_ro2/* /initrd/pup_ro3/* /initrd/pup_ro4/* /initrd/pup_ro5/* /initrd/pup_ro6/* /initrd/pup_ro7/* /initrd/pup_ro8/* /initrd/pup_ro9/* /initrd/pup_rw/*  /tmp/puppyfilesystem 
wait 
cd /tmp/puppyfilesystem 
rm -r ./initrd ./sys ./etc/.XLOADED 
Xdialog -center -msgbox "Puppyfilesystem ready. Check and edit if needed. Then please wait for 'success'. message. Press OK when ready." 0 0 
mksquashfs /tmp/puppyfilesystem /tmp/new.sfs -noappend 
wait 
rm -r /tmp/puppyfilesystem 
Xdialog -center -title "SUCCESS!!!"  --no-buttons -infobox "Operation completed, exiting now" 0 0 3000 
exit
Seems to work - I hope to test it in more depth.

Posted: Tue 30 May 2017, 06:51
by bigpup
Grabbing loaded SFS packages.

To me that seems to negate the purpose of SFS packages you load.
The idea of SFS packages, of programs, was to be able to use the program and not have it as part of the save or Puppy main SFS package.

If the original SFS package is present.
After you do the remaster, just load it when you want to use it.

The idea of a remaster is to get the contents of the save added as part of the main Puppy SFS.
The custom setup, added installed programs, etc.....

You can get the main Puppy SFS package too big, where it will take a lot of memory to load into memory.

Posted: Tue 30 May 2017, 06:56
by greengeek
bigpup wrote:To me that seems to negate the purpose of SFS packages you load.The idea of SFS packages, of programs, was to be able to use the program and not have it as part of the save or Puppy main SFS package.
.
Possibly - but what if you wanted to load (and grab) the devx and remaster it into your new puppy?

There are times where grafting an sfs might be useful.

Posted: Tue 30 May 2017, 07:47
by bigpup
Sure, anything is what you may want to do. :lol:

However, in your example. The devx is only needed when compiling.
The stuff in it is not needed, normally, to use Puppy and run programs.
Python, may be the only thing in it, you may need to run some programs.
In that situation, install Python.

Posted: Tue 30 May 2017, 09:18
by mfb
Hi greengeek,

The screenshot below shows the code I use to remaster from tahr-64-6.0.5 running entirely in ram including two loaded sfs files (one of which is its 100+MB 32 bit compatibility sfs), but I only load the devx.sfs (from the normally unmounted sda2) when I want to use it.

I boot from a CD (with a copy of the main sfs on my hard drive partition sda2 - to bypass the main sfs file on the CD and reduce elapsed boot time from some 200+ seconds to about 65 seconds).

I like gzip level 1 compression and with 3GB total ram (1GB used) it takes 32 seconds to complete the remastered 640MB main sfs (there are many installed pets as well as the loaded sfs files).

The actual script is also available for download. After download, rename it deleting the final three (dot gz) characters and make it executable.

importantly, before running the script you may wish to edit:

* the name of any current main sfs you want to keep (if it might otherwise be overwritten by the remastered version)

* /mnt/sda2/puppy_tahr64_6.0.5.sfs - to reflect your personal choice of main sfs location and/or the exact name as preset for your Pup distribution's main sfs

* gzip1 compression to xz or perhaps lz4 if available (Packit from SFR, though not used to remaster, will show your options if you view its "sfs" drop-down-list).

Extra notes:

* the script can be run from anywhere, but may be best placed on the same partition where the remastered sfs will be located - so the partition is mounted before clicking to run the script.

* The script just runs, without any further action, so just wait for the beep and the centre screen message showing the time taken - then mouse over the message to exit - do not save before rebooting to test.

Re: Yet another basic remaster script for frugal install

Posted: Tue 30 May 2017, 10:10
by nic007
greengeek wrote:
nic007 wrote:

Code: Select all

#!/bin/sh
mkdir /mnt/home/puppyfilesystem....
I always run my puppy without any savefile and I don't have any /mnt/home - everything is done in ram.

Can you suggest any modification of this script to cover my situation? I would like to be able to grab any sfs files that are loaded.

Thus far I have been using an earlier version of one of your remaster scripts and it grabs pets that are loaded but not sfs files.

cheers!

EDIT : I just had a go at making some changes and came up with the following:

Code: Select all

#!/bin/sh 
mkdir /tmp/puppyfilesystem 
cp -a /initrd/pup_ro2/* /initrd/pup_ro3/* /initrd/pup_ro4/* /initrd/pup_ro5/* /initrd/pup_ro6/* /initrd/pup_ro7/* /initrd/pup_ro8/* /initrd/pup_ro9/* /initrd/pup_rw/*  /tmp/puppyfilesystem 
wait 
cd /tmp/puppyfilesystem 
rm -r ./initrd ./sys ./etc/.XLOADED 
Xdialog -center -msgbox "Puppyfilesystem ready. Check and edit if needed. Then please wait for 'success'. message. Press OK when ready." 0 0 
mksquashfs /tmp/puppyfilesystem /tmp/new.sfs -noappend 
wait 
rm -r /tmp/puppyfilesystem 
Xdialog -center -title "SUCCESS!!!"  --no-buttons -infobox "Operation completed, exiting now" 0 0 3000 
exit
Seems to work - I hope to test it in more depth.
That's doing it in RAM, should work if you have enough RAM. The following thread may be of interest to you: http://www.murga-linux.com/puppy/viewtopic.php?t=109745
I keep the original base sfs intact, load sfs's for my applications and record any personal settings also in an sfs file.

Unload SFSes before Remaster

Posted: Tue 30 May 2017, 15:44
by mikeslr
[quote="bigpup"]Grabbing loaded SFS packages.

To me that seems to negate the purpose of SFS packages you load.
The idea of SFS packages, of programs, was to be able to use the program and not have it as part of the save or Puppy main SFS package./quote]

Seems to me that this objection could be overcome if, before running remaster, unwanted SFSes are unloaded. Or am I missing something? Coding isn't my strong suit. But, as I read it, this line

cp -a /initrd/pup_ro2/* /initrd/pup_ro3/* /initrd/pup_ro4/* /initrd/pup_ro5/* /initrd/pup_ro6/* /initrd/pup_ro7/* /initrd/pup_ro8/* /initrd/pup_ro9/* /initrd/pup_rw/*

copies loaded SFSes into the remaster. If any of the "ro"s are not in use, noting from that "ro" gets copied.

Since I always run portable-wine (a 32-bit application) from Tahrpup64 and Xenialpup64, part of my initial setup routine is to load the 32-bit compatibility SFS.* On the other hand, I prefer to use the most current web-browsers and LibreOffice SFSes. If I understand the script correctly, a remaster would accomplish my goal if, before remastering, I unloaded Libreoffice and webbrowsers but left the 32-bit compatibility SFS loaded.

mikesLr

* Actually, I've long since converted the compatibility SFS to an installable pet. Consider the statement for the purpose of illustration. Perhaps core Qt4 and Qt5 SFSes would have been better. Both are relatively small, and some applications I use need one, and some the other.

Re: Unload SFSes before Remaster

Posted: Tue 30 May 2017, 17:05
by nic007
mikeslr wrote:
bigpup wrote:Grabbing loaded SFS packages.

To me that seems to negate the purpose of SFS packages you load.
The idea of SFS packages, of programs, was to be able to use the program and not have it as part of the save or Puppy main SFS package./quote]

Seems to me that this objection could be overcome if, before running remaster, unwanted SFSes are unloaded. Or am I missing something? Coding isn't my strong suit. But, as I read it, this line

cp -a /initrd/pup_ro2/* /initrd/pup_ro3/* /initrd/pup_ro4/* /initrd/pup_ro5/* /initrd/pup_ro6/* /initrd/pup_ro7/* /initrd/pup_ro8/* /initrd/pup_ro9/* /initrd/pup_rw/*

copies loaded SFSes into the remaster. If any of the "ro"s are not in use, noting from that "ro" gets copied.

Since I always run portable-wine (a 32-bit application) from Tahrpup64 and Xenialpup64, part of my initial setup routine is to load the 32-bit compatibility SFS.* On the other hand, I prefer to use the most current web-browsers and LibreOffice SFSes. If I understand the script correctly, a remaster would accomplish my goal if, before remastering, I unloaded Libreoffice and webbrowsers but left the 32-bit compatibility SFS loaded.

mikesLr

* Actually, I've long since converted the compatibility SFS to an installable pet. Consider the statement for the purpose of illustration. Perhaps core Qt4 and Qt5 SFSes would have been better. Both are relatively small, and some applications I use need one, and some the other.
Correct. The script will only include the contents of those extra sfs's which are loaded at the time of the remaster. The installed pets, loaded sfs's and configuration settings will be included in a new base sfs. Some people prefer to have everything included in their base sfs and it is probably the way to go if you want to make your own puplet for sharing.

Posted: Tue 30 May 2017, 23:15
by Moat
Plus, it seems some programs are not readily available as .pet packages, but are available as an SFS - and testing a program by running it for a while as an SFS before deciding to include it in the base/main SFS sounds to me like a really great option. Kudos to nic007 for this! 8)

I admit to being one who prefers everything in the base SFS (within reason) - sure, can take longer to boot, but it's always there at the ready when needed, and no fiddling with loading/unloading and interrupting the workflow.

Bob

Posted: Tue 30 May 2017, 23:28
by ndujoe1
Question: so if I remove all the default puppy applications I don't wish to use, the will remaster everything I do use, personal files and sfs's?

Posted: Wed 31 May 2017, 02:43
by nic007
ndujoe1 wrote:Question: so if I remove all the default puppy applications I don't wish to use, the will remaster everything I do use, personal files and sfs's?
It will include all loaded sfs's, pets you have installed and personal files that are stored in the savefile/folder (not personal files that are stored elsewhere on your drive). As far as removal of default built-in applications goes: - this will depend how you remove it. If you use some application that only white-out (mark them that they can not be seen by the operating system) files, I don't think this basic script will actually delete them from the base sfs (but it should be masked so that there won't be a menu entry for them, etc.). If you physically remove them by editing the base sfs, it will definitely be gone.

Posted: Thu 01 Jun 2017, 15:35
by mikeslr
Repeating my previous caution "coding is not my strong suit", as I understand nic007's command line:

cp -a /initrd/pup_ro2/* /initrd/pup_ro3/* /initrd/pup_ro4/* /initrd/pup_ro5/* /initrd/pup_ro6/* /initrd/pup_ro7/* /initrd/pup_ro8/* /initrd/pup_ro9/* /initrd/pup_rw/* /mnt/home/puppyfilesystem

whatever you have in /root will be copied. /root is where all application settings are stored.

The builtin remaster application will not include in your custom remastered version the applications you removed using Menu>Setup>Remove builtin packages. The way this works --when the builtin remaster works-- is that remove builtin doesn't actually remove the package, only "white's out" the link to it. So the builtin remaster, doesn't have a link to follow and, consequently, doesn't include the application. nic007's remaster simply copies what's there including both the applications and the "white outs".

Frankly, removing applications IMHO is a waste of time (unless, of course, you're involved in some grand project, like taking a Puppy which uses JWM as its Window Manager and substituting XFCE as Window Manager). Storage space is cheap. Recently I purchased a package of 3 8Gbs USB-Keys for $12. Hard-drives can be found for as little as $.04 per Gb*. Most Devs creating Puppies don't include large applications --such as LibreOffice-- in their creation; and a Frugally installed Puppy doesn't load an entire builtin** application into RAM, but rather creates links to it which are followed as needed. The only downside of builtin applications is that they appear on the Menu delaying your selection of those applications you actually want to use. Newer Puppies have builtin Menu>Setup>Menu Manager --edit the Menu, which can remove an application's appearance from the Menu. In older Puppies, you can edit their desktop files, /usr/share/applications/xxx.desktop, so that the category argument is one which will not produce a display, e.g Categories=Calculator to Categories=ZCalculator. Both techniques would, I believe, be carried over into your remaster using nic007's remaster script.

mikesLr

* If it takes me an hour to remove 25 Mbs of applications, conserving the space worth 4 cents per Gb, the value of my time is .1 cent per hour. Would you work for that?

** When you install a new version of an application or load an SFS, say Palemoon, into a Puppy where the Dev has builtin a version of Palemoon, both versions continue to exist. But the links Puppy uses will now point to the new version rather than the old.

nic007 can explain it better. I've used the term "links" which I, and I think most of us, understand. But nic007 can probably explain how inodes function.

Posted: Fri 02 Jun 2017, 22:07
by smokey01
I always have the devx loaded as I do a lot of compiling. This method would allow me to include the devx in the main base so I could compile in a virtual environment like Vbox and Qemu. Nice.

Posted: Sat 03 Jun 2017, 00:21
by nic007
mikeslr wrote:Repeating my previous caution "coding is not my strong suit", as I understand nic007's command line:

cp -a /initrd/pup_ro2/* /initrd/pup_ro3/* /initrd/pup_ro4/* /initrd/pup_ro5/* /initrd/pup_ro6/* /initrd/pup_ro7/* /initrd/pup_ro8/* /initrd/pup_ro9/* /initrd/pup_rw/* /mnt/home/puppyfilesystem

whatever you have in /root will be copied. /root is where all application settings are stored.

The builtin remaster application will not include in your custom remastered version the applications you removed using Menu>Setup>Remove builtin packages. The way this works --when the builtin remaster works-- is that remove builtin doesn't actually remove the package, only "white's out" the link to it. So the builtin remaster, doesn't have a link to follow and, consequently, doesn't include the application. nic007's remaster simply copies what's there including both the applications and the "white outs".

Frankly, removing applications IMHO is a waste of time (unless, of course, you're involved in some grand project, like taking a Puppy which uses JWM as its Window Manager and substituting XFCE as Window Manager). Storage space is cheap. Recently I purchased a package of 3 8Gbs USB-Keys for $12. Hard-drives can be found for as little as $.04 per Gb*. Most Devs creating Puppies don't include large applications --such as LibreOffice-- in their creation; and a Frugally installed Puppy doesn't load an entire builtin** application into RAM, but rather creates links to it which are followed as needed. The only downside of builtin applications is that they appear on the Menu delaying your selection of those applications you actually want to use. Newer Puppies have builtin Menu>Setup>Menu Manager --edit the Menu, which can remove an application's appearance from the Menu. In older Puppies, you can edit their desktop files, /usr/share/applications/xxx.desktop, so that the category argument is one which will not produce a display, e.g Categories=Calculator to Categories=ZCalculator. Both techniques would, I believe, be carried over into your remaster using nic007's remaster script.

mikesLr

* If it takes me an hour to remove 25 Mbs of applications, conserving the space worth 4 cents per Gb, the value of my time is .1 cent per hour. Would you work for that?

** When you install a new version of an application or load an SFS, say Palemoon, into a Puppy where the Dev has builtin a version of Palemoon, both versions continue to exist. But the links Puppy uses will now point to the new version rather than the old.

nic007 can explain it better. I've used the term "links" which I, and I think most of us, understand. But nic007 can probably explain how inodes function.
Unfortunately this does NOT seem to be the case. I "removed" Seamonkey using "Remove builtin Packages" and then ran the builtin remaster script. Only the menu entries were deleted, Seamonkey was still there in the remastered sfs. My script also only removes the menu entries. I remember pup412 had an application called pet-be-gone which worked but Remove builtin packages doesn't seem to.

Posted: Sat 03 Jun 2017, 15:36
by mikeslr
Another reason not to actually remove "builtins" unless you really know what you are doing. With the "remove builtin" application so limited, it would seem that in order to actually remove an application and all of its files, you'd have to open its listing in /root/.packages in a text viewer, then manually delete the folders, and in some cases, the specific files shown on that list. But to safely do so, you have to be certain that every file on that list isn't also used by some application you still want to be functional.*

Perhaps the coding requirement of a program that would keep track of all files within each system and which applications make use of each file explains why "pet-be-gone" is no longer supported, and "remove builtins" has limited value.

mikesLr

* and if you possess that knowledge, its probably faster to woof your own Puppy version than to strip someone else's version of what you don't want.