Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Fri 25 Jul 2014, 03:01
All times are UTC - 4
 Forum index » House Training » Users ( For the regulars )
Unattended remaster script
Moderators: Flash, Ian, JohnMurga
Post_new_topic   Reply_to_topic View_previous_topic :: View_next_topic
Page 1 of 3 Posts_count   Goto page: 1, 2, 3 Next
Author Message
nic007

Joined: 13 Nov 2011
Posts: 589

PostPosted: Thu 10 Jul 2014, 10:55    Post_subject:  Unattended remaster script  

This is my attempt creating a basic, customised, remastered SFS "unattended" (the cd not needed). Try it if you think it useful.

1) For remaster including contents of save file only (installed pets, etc.)

Code:
mkdir /puppyfilesystem /puppyfilesystem/proc /puppyfilesystem/tmp
cp -a /initrd/pup_ro2/usr /bin /lib /sbin /opt /var /dev /archive  /puppyfilesystem
cp -a /root /etc  /puppyfilesystem
rm /puppyfilesystem/etc/.XLOADED
cp -a /initrd/pup_rw/usr /var /root /lib /bin /sbin /opt /archive  /puppyfilesystem
cp -a /initrd/pup_ro1/usr /var /root /lib /bin /sbin /opt /archive  /puppyfilesystem
mkdir /mnt/home/puppylivediscbuild
cd /mnt/home
mksquashfs /puppyfilesystem puppylivediscbuild/puppy-remastered.sfs -noappend
rm /puppyfilesystem -r


Method: Just copy the text in full to terminal (select all, copy, middle-click in terminal to paste). Rename the remastered SFS. Note that the copying and deleting of files/directories are done in a Puppy Linux filesystem. In this example I enlarged my save file to accommodate the working directory. Further note that your terminal may show a few copy "errors". Ignore this as the code is relatively comprehensive and your particular setup may/will not require all of the copying over. It won't affect the end result. Also, make sure any changes you make are saved to the save file before doing the remaster. Best to do the remaster after a reboot.

If you want to make an iso, you will need to copy the necessary files to /mnt/home/puppylivediscbuild from the live cd and run the following commands in terminal:
Code:
cd /mnt/home
mkisofs -b isolinux.bin -c boot.cat -D -l -R -v -no-emul-boot -boot-load-size 4 -boot-info-table -o "puppy_remaster.iso" puppylivediscbuild
.

2) For remaster including save file, mounted add-on sfs's booted via bootmanager, mounted sfs's booted on the fly and pup_z:

Code:
mkdir /puppyfilesystem /puppyfilesystem/proc /puppyfilesystem/tmp
cp -a /initrd/pup_ro2/var /dev  /puppyfilesystem
cp -a /root /etc /usr /lib /bin /sbin /opt /archive  /puppyfilesystem
rm /puppyfilesystem/etc/.XLOADED
cp -a /initrd/pup_rw/usr /var /root /lib /bin /sbin /opt /archive  /puppyfilesystem
cp -a /initrd/pup_ro1/usr /var /root /lib /bin /sbin /opt /archive  /puppyfilesystem
cp -a /initrd/pup_ro3/* /puppyfilesystem
cp -a /initrd/pup_ro4/* /puppyfilesystem
cp -a /initrd/pup_ro5/* /puppyfilesystem
cp -a /initrd/pup_ro6/* /puppyfilesystem
cp -a /initrd/pup_ro7/* /puppyfilesystem
cp -a /initrd/pup_ro8/* /puppyfilesystem
cp -a /initrd/pup_ro9/* /puppyfilesystem
cp -a /initrd/pup_z/* /puppyfilesystem
mkdir /mnt/home/puppylivediscbuild
cd /mnt/home
mksquashfs /puppyfilesystem puppylivediscbuild/puppy-remastered.sfs -noappend
rm /puppyfilesystem -r


Tested on official releases of Puppy412, Puppy431 and Wary Puppy. May or may not work on other puppies/puplets (the second code is more likely to succeed with other puppies, just unmount sfs's, etc. you don't want to include in the remaster).

Edited_times_total
Back to top
View user's profile Send_private_message 
mikeb


Joined: 23 Nov 2006
Posts: 8028

PostPosted: Thu 10 Jul 2014, 12:27    Post_subject:  

one proviso for this method...it will use several hundred megs in the filesystem/save file or ram if used that way....so a space check would be a good idea since it may remaster leaving out many files without warning as can happen even with the standard way.

The idea of squashing directly and adding to it is to minimise space usage.

Mike
Back to top
View user's profile Send_private_message 
nic007

Joined: 13 Nov 2011
Posts: 589

PostPosted: Thu 10 Jul 2014, 12:39    Post_subject:  

Yes otherwise you have to do it in a puppy linux filesystem outside your save file unless you have a massive amount of RAM. I don't have puppy linux filesystem partitions on my harddrive so use the save file. Generally space of working directory should at least be 3 times the size of your base sfs plus the occupied space on your save file.
Back to top
View user's profile Send_private_message 
Sylvander

Joined: 15 Dec 2008
Posts: 3400
Location: West Lothian, Scotland, UK

PostPosted: Thu 10 Jul 2014, 12:59    Post_subject:  

1. On my Slacko-5.7.0-pae...
Used your commands, one line at a time.
Produced a much bigger ISO file than normal.
When I booted the remastered CD-RW, I used the puppy pfix=ram command to get it to ignore the existing slackosave file.
It appeared to work OK, BUT...

2. When it got to the desktop, there were none of my customizations, and it was asking for config settings for language, locale, etc.

3. I had wanted a remaster that included EVERYTHING, so I decided to go back to using the old setup...
I rebooted my old CD-RW, BUT...
My customizations were still GONE!
Any idea why? Confused

4. To get them back, here's what I did:
a. Deleted the SFS file and the slackosave file from their folder on the internal HDD partition.
b. Replaced the SFS using the original from the CD-RW, and replaced the slackosave using a backup copy made in early June..
c. Deleted all the files used to make the new ISO, and deleted the new ISO.
No idea which of the above produced the result I desired and got.
Back to top
View user's profile Send_private_message 
nic007

Joined: 13 Nov 2011
Posts: 589

PostPosted: Thu 10 Jul 2014, 13:49    Post_subject:  

Added some more folders of pup_rw to merge if available. This should pretty much cover all installed pets to the save file and also help with the customization.
Back to top
View user's profile Send_private_message 
nic007

Joined: 13 Nov 2011
Posts: 589

PostPosted: Thu 10 Jul 2014, 14:00    Post_subject:  

Sylvander wrote:
1. On my Slacko-5.7.0-pae...
Used your commands, one line at a time.
Produced a much bigger ISO file than normal.
When I booted the remastered CD-RW, I used the puppy pfix=ram command to get it to ignore the existing slackosave file.
It appeared to work OK, BUT...

2. When it got to the desktop, there were none of my customizations, and it was asking for config settings for language, locale, etc.

3. I had wanted a remaster that included EVERYTHING, so I decided to go back to using the old setup...
I rebooted my old CD-RW, BUT...
My customizations were still GONE!
Any idea why? Confused



4. To get them back, here's what I did:
a. Deleted the SFS file and the slackosave file from their folder on the internal HDD partition.
b. Replaced the SFS using the original from the CD-RW, and replaced the slackosave using a backup copy made in early June..
c. Deleted all the files used to make the new ISO, and deleted the new ISO.


No idea which of the above produced the result I desired and got.

I only did the remaster of the base sfs and didn't go all the way to building the iso (just added that bit for the sake of completeness). I've added a few more folders of pup_rw (if they are present) which should help with customization. I've tested this on puppy 412 and wary puppy, don't know about the rest. BTW the idea is to copy the full text to terminal one shot and let the system run through everything unattended. I delete my save file after the process.
Back to top
View user's profile Send_private_message 
mikeb


Joined: 23 Nov 2006
Posts: 8028

PostPosted: Thu 10 Jul 2014, 14:37    Post_subject:  

mkdir /puppyfilesystem
ok well that is in the puppy filesystem... though doing it outside would be better considering the size and must be posix.

squashing the layered filesystem minus a few custom folders to be added straight after is less of a headache... less likely you miss out stuff.

mike
Back to top
View user's profile Send_private_message 
nic007

Joined: 13 Nov 2011
Posts: 589

PostPosted: Thu 10 Jul 2014, 15:19    Post_subject:  

The working system directory must be in a linux file system otherwise the copying won't work. So you need to use either the save file, an ext partition/folder filesystem or RAM. Whatever you use, make sure it's big enough
Back to top
View user's profile Send_private_message 
gcmartin

Joined: 14 Oct 2005
Posts: 4088
Location: Earth

PostPosted: Thu 10 Jul 2014, 15:47    Post_subject:  

This may be an odd question, but I'll claim "ignorance" (which in my case is true).

I ask
If my system operates everything from its filesystem view; namely "/" (also called the system's root)
  • Can one just cp/rsync everything from "/" directory for a TOTAL system view into /puppyfilesystem?
  • Why does it have to be done individually and why from ...pup_rw?

  • If I do it from "/" would it not include base, pets, sfs, everything?
  • Or is, somehow, the kernel an element which negates this approach for use in building a remaster?
Curious why this is not ever discussed or used.

_________________
Get ACTIVE Create Circles; Do those good things which benefit people's needs!
We are all related ... Its time to show that we know this!
3 Different Puppy Search Engine or use DogPile
Back to top
View user's profile Send_private_message 
nic007

Joined: 13 Nov 2011
Posts: 589

PostPosted: Fri 11 Jul 2014, 01:57    Post_subject:  

gcmartin wrote:
This may be an odd question, but I'll claim "ignorance" (which in my case is true).

I ask
If my system operates everything from its filesystem view; namely "/" (also called the system's root)
  • Can one just cp/rsync everything from "/" directory for a TOTAL system view into /puppyfilesystem?
  • Why does it have to be done individually and why from ...pup_rw?

  • If I do it from "/" would it not include base, pets, sfs, everything?
  • Or is, somehow, the kernel an element which negates this approach for use in building a remaster?
Curious why this is not ever discussed or used.

Well, it's true that everything is under / but that doesn't mean you can just copy everything. The thing is that some directories/files can not just be copied over as this will result in a "broken" remaster which will in all probability have booting and other problems (an example may be copying the sys folder). And of course the contents of everything that is mounted will be included and that could be all the data on your hard drives/usb drives as well as mounted SFS's, all of which you don't necessarily want to include. Note also that only certain directories of pup_rw are included. Including everything under pup_rw (the save file) will lead to a broken remaster. Same is true should you want to include some mounted sfs's which may be mounted under another pup_ro. All the directories can not just be copied over. Pup_ro2 contains the pristine/clean base SFS, so starting there and adding on is the way to go. BTW - this is an unattended method as far as producing the remastered SFS is concerned. Just copy the full text to terminal, press enter and the system will do the rest (there's no copying of files/folders to be done manually unless you want to make the iso too).
Back to top
View user's profile Send_private_message 
gcmartin

Joined: 14 Oct 2005
Posts: 4088
Location: Earth

PostPosted: Fri 11 Jul 2014, 04:43    Post_subject:  

Thanks for your explanations on remaster. Had forgotten about potentially "mounted" partitions. While I did remember "loop" mounts of SFSs.

Your explanations are particularly clear of your steps you offered earlier. Thx as well for that.

One other method, I remember was introduced by @Flash here. He did NOT call it a remaster, but, similarly it accomplishes the same thing. His posting indicates when using a DVD with save-sessions to be captured into a new presentation.

For taking one's running system with user personalizations, customizations, and tailoring, anything that can be done without the use of DVD or ISO, if possible is something some of us have sort that allows capture without the need to "practice capturing over and over and over" to get the process right. There is a great need if it can be clear and simple.

I find it unattractive to EXPECT any user who is not knowledgeable enough to know what wasn't copied and what was where remaster utilities dumped them into a Rox screen with instructions that is only clear to the developer. Utility developers know that aspects of the users systems are contained in system folders: so why stop and ask? Could a description at the beginning of the utility as clear questions to the user so that he can answer in the dialogue's form yes/no to questions at the front end without having to have waited for half-hour to get it wrong and have to start-over with no prior audit trail to potentially get it wrong again.

Its not that the developers don't know what they are doing...they do. But as energetic as they are, in this utility's case, a front ended approach of the outcome's intention would probably be a much better option for the user understanding, than has been used in the past.

Again, thanks @NIC007 for drawing attention to capture of the running operating system into a new bootable entity. ... namely "Remaster"

_________________
Get ACTIVE Create Circles; Do those good things which benefit people's needs!
We are all related ... Its time to show that we know this!
3 Different Puppy Search Engine or use DogPile
Back to top
View user's profile Send_private_message 
stemsee


Joined: 27 Jun 2013
Posts: 265
Location: London

PostPosted: Fri 11 Jul 2014, 06:16    Post_subject:  

How about 'read' 'path' and maximum compression and remove .XLOADED for /etc/.XLOADED or /root/.XLOADED file, added pup_a and pup_y. Also added timeout function for user input to not create iso, or enter y or at timeout assume no to NOT create iso.
EDIT: updated
Code:
#!/bin/sh
#original by nic007 modified by StemSee

echo "Enter partition name of ext 2/3/4 partition for remaster folder eg sda2
or press enter for '/' in ram."
read drive

mount /dev/$drive /mnt/$drive
mkdir -p /mnt/$drive/puppyfilesystem/proc /mnt/$drive/puppyfilesystem/tmp /mnt/$drive/puppylivediscbuild

rm -f /var/cache/apt/archives/*.deb     #for EmSeeV2+ apt/synaptic downloads delete.

cp -a /initrd/pup_ro2/var /dev  /mnt/$drive/puppyfilesystem/
cp -a /root /etc /usr /lib /bin /sbin /opt /archive /include /srv /selinux /obexd-0.38 /mnt/$drive/puppyfilesystem/
cp -a /initrd/pup_ro1/usr /var /root /lib /bin /sbin /opt /archive  /mnt/$drive/puppyfilesystem/
cp -a /initrd/pup_ro3/* /mnt/$drive/puppyfilesystem/
cp -a /initrd/pup_ro4/* /mnt/$drive/puppyfilesystem/
cp -a /initrd/pup_ro5/* /mnt/$drive/puppyfilesystem/
cp -a /initrd/pup_ro6/* /mnt/$drive/puppyfilesystem/
cp -a /initrd/pup_ro7/* /mnt/$drive/puppyfilesystem/
cp -a /initrd/pup_ro8/* /mnt/$drive/puppyfilesystem/
cp -a /initrd/pup_ro9/* /mnt/$drive/puppyfilesystem/
cp -a /initrd/pup_z/* /mnt/$drive/puppyfilesystem/
cp -a /initrd/pup_a/* /mnt/$drive/puppyfilesystem/
cp -a /initrd/pup_y/* /mnt/$drive/puppyfilesystem/
cp -a /initrd/pup_rw/usr /var /root /lib /bin /sbin /opt /archive  /mnt/$drive/puppyfilesystem/                  #copied last so not overwritten

if [ -e /mnt/$drive/puppyfilesystem/etc/.XLOADED ]; then
   rm -f /mnt/$drive/puppyfilesystem/etc/.XLOADED
fi
if [ -e /mnt/$drive/puppyfilesystem/root/.XLOADED ]; then
   rm -f /mnt/$drive/puppyfilesystem/root/.XLOADED
fi

rm -f /mnt/$drive/puppyfilesystem/root/.bash_history .history .Trash/. .cache/chrome/default/cache/. .cache/mozilla/firefox/.         #clean bash history browser caches.

cd /mnt/$drive/puppyfilesystem
find . -type f -name '.wh.*' -print -exec rm -rf {} \;                        #remove .wh. files
cd ..
mksquashfs puppyfilesystem puppylivediscbuild/puppy-remastered.sfs -noappend -comp xz -b 1024k -Xbcj x86
#rm /mnt/$drive/puppyfilesystem -r    # left in existence for fixing simple errors and re-squash.

timelimit=60
echo "If you want to make an iso file then copy boot files (isolinux.bin,
isolinux.cfg, initrd.gz, vmlinuz, grldr, boot.msg, help.msg, help2.msg, logo.16,
README.HTM) from cd or directory to /mnt/"$drive"/puppylivediscbuild."
echo -e "You have $timelimit seconds\n Type 'y' to create iso or 'n' to NOT create iso then press enter: \c"
answer=""
read -t $timelimit answer
if [ ! -z "$answer" ]; then
echo -e "\n You entered '$answer'"
else
echo -e "\n TIME OUT\n Assume no"
answer=n                                                       #because no boot files copied!!
fi

case $answer in
   y) mkisofs -b isolinux.bin -c boot.cat -D -l -R -v -no-emul-boot -boot-load-size 4 -boot-info-table -o "puppy_remaster.iso" puppylivediscbuild
   echo "'puppy_remaster.iso' is on drive /mnt/"$drive".";;

   n) echo "'puppyfilesystem' and 'puppylivediscbuild' directories left in existence.
Delete manually if no longer needed. End of script!";;

   *) echo "Sorry, unexpected input.'puppyfilesystem' and 'puppylivediscbuild'
   directories left in existence.  Exiting script!  Have a nice one!";;

esac
exit 0



cheers @nic007

Edited_times_total
Back to top
View user's profile Send_private_message MSNM 
Sylvander

Joined: 15 Dec 2008
Posts: 3400
Location: West Lothian, Scotland, UK

PostPosted: Fri 11 Jul 2014, 06:19    Post_subject:  

@nic007

1. Had a 2nd go with your new code.

2. Using the 1st group of commands [pasted and executed in a single block] there was made:
/initrd/mnt/dev_save/puppylivediscbuild/puppy-remastered.sfs = 312.1MB in size.

3. PROBLEM when I used the 2nd block of code, as follows:
Code:
# cd /mnt/home
# mkisofs -b isolinux.bin -c boot.cat -D -l -R -v -no-emul-boot -boot-load-size 4 -boot-info-table -o "puppy_remaster.iso" puppylivediscbuild
Setting input-charset to 'UTF-8' from locale.
3.01a08 (i686-pc-linux-gnu)
Scanning puppylivediscbuild
Writing:   Initial Padblock                        Start Block 0
Done with: Initial Padblock                        Block(s)    16
Writing:   Primary Volume Descriptor               Start Block 16
Done with: Primary Volume Descriptor               Block(s)    1
Writing:   Eltorito Volume Descriptor              Start Block 17
mkisofs: Uh oh, I cant find the boot image 'isolinux.bin' inside the target tree.
#

My desktop PC has 8GB of RAM, which is plenty methinks.
Found /initrd/mnt/dev_save/puppy_remaster.iso = 34kB ! Rolling Eyes
Something wrong here! Too small.

Any idea what's going wrong?

4. When I'm finished working in this session...
I would normally close my Slacko without saving session changes...
So if I do this as per normal, all of these changes/files will be lost unless I move [or copy] them to some location outside the Slacko filesystem.
I'll try to delay doing that for as long as possible in the hope of getting a reply whilst still running this session.
Back to top
View user's profile Send_private_message 
nic007

Joined: 13 Nov 2011
Posts: 589

PostPosted: Fri 11 Jul 2014, 06:43    Post_subject:  

The biggest benefit of using a remaster process like this is that it gives one freedom of choice what you want to include in your remaster or not. This specific example takes care of the installed programs in the save file but does not include mounted sfs's and other "underdogs". The later can easily be included by adding a few lines of code. Add-on sfs's for example can be loaded by the bootmanager or on the fly. If loaded by the bootmanager the contents thereof can be found in one of the mounted pup_ro's which can then just be copied. If on the fly, the files will normally be loaded in /usr so that folder can just be merged to include the files.
Back to top
View user's profile Send_private_message 
nic007

Joined: 13 Nov 2011
Posts: 589

PostPosted: Fri 11 Jul 2014, 06:58    Post_subject:  

Sylvander wrote:
@nic007

1. Had a 2nd go with your new code.

2. Using the 1st group of commands [pasted and executed in a single block] there was made:
/initrd/mnt/dev_save/puppylivediscbuild/puppy-remastered.sfs = 312.1MB in size.

3. PROBLEM when I used the 2nd block of code, as follows:
Code:
# cd /mnt/home
# mkisofs -b isolinux.bin -c boot.cat -D -l -R -v -no-emul-boot -boot-load-size 4 -boot-info-table -o "puppy_remaster.iso" puppylivediscbuild
Setting input-charset to 'UTF-8' from locale.
3.01a08 (i686-pc-linux-gnu)
Scanning puppylivediscbuild
Writing:   Initial Padblock                        Start Block 0
Done with: Initial Padblock                        Block(s)    16
Writing:   Primary Volume Descriptor               Start Block 16
Done with: Primary Volume Descriptor               Block(s)    1
Writing:   Eltorito Volume Descriptor              Start Block 17
mkisofs: Uh oh, I cant find the boot image 'isolinux.bin' inside the target tree.
#

My desktop PC has 8GB of RAM, which is plenty methinks.
Found /initrd/mnt/dev_save/puppy_remaster.iso = 34kB ! Rolling Eyes
Something wrong here! Too small.

Any idea what's going wrong?

4. When I'm finished working in this session...
I would normally close my Slacko without saving session changes...
So if I do this as per normal, all of these changes/files will be lost unless I move [or copy] them to some location outside the Slacko filesystem.
I'll try to delay doing that for as long as possible in the hope of getting a reply whilst still running this session.


Sylvander, I don't run Puppy off CD and my knowledge of making iso's is limited. The second code for making the iso was copied straight from Puppy Wiki. Did you copy the necessary files from the cd (excluding the sfs of course) to the location of the new remastered sfs and then ran the command for making of the iso? My concern is the remastered SFS. Did that come out right and have you been able to boot it frugally?
Back to top
View user's profile Send_private_message 
Display_posts:   Sort by:   
Page 1 of 3 Posts_count   Goto page: 1, 2, 3 Next
Post_new_topic   Reply_to_topic View_previous_topic :: View_next_topic
 Forum index » House Training » Users ( For the regulars )
Jump to:  

Rules_post_cannot
Rules_reply_cannot
Rules_edit_cannot
Rules_delete_cannot
Rules_vote_cannot
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.1264s ][ Queries: 11 (0.0081s) ][ GZIP on ]