Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Wed 22 Oct 2014, 02:44
All times are UTC - 4
 Forum index » Advanced Topics » Puppy Projects
Remastering remastering
Moderators: Flash, JohnMurga
Post_new_topic   Reply_to_topic View_previous_topic :: View_next_topic
Page 1 of 2 Posts_count   Goto page: 1, 2 Next
Author Message
pa_mcclamrock


Joined: 03 Jun 2005
Posts: 657
Location: Fort Wayne, Indiana, USA

PostPosted: Wed 04 Feb 2009, 18:33    Post_subject:  Remastering remastering
Sub_title: Questions on new GTK2-look script for making custom live-CDs
 

I'm working on an app tentatively entitled "Remaster Express" to make the Puppy custom live-CD remastering process more versatile and more user-friendly, with a GUI that persists throughout the process, incorporates more options than the existing remaster script (/usr/sbin/remasterpup2), and takes on the selected GTK+ 2.0 theme (by way of Gnocl). Here are links to some screenshots of what I've got so far. [EDIT 2009 Feb 14: Don't look at these any more, look at the completed set of screenshots with the links in the message I just posted below.)

Initial "Welcome" window: http://www.geocities.com/pa_mcclamrock/remaster01.gif

Remaster Slimmer window, for removing built-in packages before remastering: http://www.geocities.com/pa_mcclamrock/remasterslim.gif

Step Two, estimate working space needed: http://www.geocities.com/pa_mcclamrock/remaster02.3.gif

Show estimated working space and select working partition: http://www.geocities.com/pa_mcclamrock/remaster02.6.gif

Step Three, show information about remastering while making heap big squashfile: http://www.geocities.com/pa_mcclamrock/remaster03.gif

Step Four, verify that contents of /tmp/root include everything desired: http://www.geocities.com/pa_mcclamrock/remaster04.gif

Step Five, do the same with /tmp/etc and specify hardware configuration if desired: http://www.geocities.com/pa_mcclamrock/remaster05.gif

Step Six, edit isolinux.cfg if desired: http://www.geocities.com/pa_mcclamrock/remaster06.1.gif

Step Seven, rename pup_xxx.sfs file if desired:
http://www.geocities.com/pa_mcclamrock/remaster07.gif

I don't have the rest done yet, but I do have several questions that I hope somebody here can answer.

* How do you best estimate the size of the pup_xxx.sfs file compared to the uncompressed size of the directories that go into it? The "remasterpup2" script just seems to guess that maybe it will be about 1/3 as big, and maybe you'll need, oh, 150 to 200 MB of additional free space if you're going to save an ISO. Is it possible to do better than this? As I said above, I'd like to have a progressbar to show what percentage of stuff has been squashed into the squashfile and what percentage remains to be squashed, but this would seem to require a pretty good estimate of how big the whole thing will be when done.

* I need to make sure I know what to do with the various "packages" files in /root/.packages. I observe that "packages.txt" has the complete list of official packages from the current Puppy Unleashed; "livepackages.txt" is the same as the "Official Puppy 4" list you see in PETget; "livepackages2.txt" and "livepackages3.txt" are the same as Official Puppy 2 and Official Puppy 3 respectively; and "alienpackages.txt" lists the unofficial packages you have installed. To this I have added "slimpacks-xxx.txt," a modified list of built-in packages the user might wish to uninstall with "Remaster Slimmer" before remastering.

What I want to do, I think, is to (1) copy "packages.txt" intact to the /tmp/root/.packages directory for inclusion in the new CD; (2) copy the various "livepackages" files and "slimpacks-xxx.txt" to /tmp/root/.packages too, but then (3) move all the "on" lines out of the "livepackages" copies and merge them into /tmp/root/.packages/slimpacks-xxx.txt, since the packages named in these lines will become built-in packages in the new CD and will be uninstallable with Remaster Slimmer but not with PETget; and (4) omit "alienpackages," since all the packages from there will become built-in packages in the new CD. Does that sound right?

* Following line 196, "#now copy the files... ," "remasterpup2" omits "boot.cat" from the list of files to be copied from the existing live-CD, and includes something called "goofy.16" instead. What's this, and why?

* Are all these "syncs" really necessary? Almost everywhere you do anything with files in "remasterpup2," it seems, there's a "sync" before and after. Would there be any significant chance of disaster if you didn't have quite so many "syncs"?

* All references to "zdrv" can be omitted for Puppy 4.1 and later, can't they?

Thanks in advance for any help!

David McClamrock

_________________
It's stupid to use inferior software for ideological reasons.
--Linus Torvalds

Visit Pa Penguin's Icebox: http://www.pa-mcclamrock.com/icebox.html

Edited_times_total
Back to top
View user's profile Send_private_message Visit_website 
Lobster
Official Crustacean


Joined: 04 May 2005
Posts: 15117
Location: Paradox Realm

PostPosted: Wed 04 Feb 2009, 23:28    Post_subject:  

The last easy remaster was from Dougal and worked very well.
It was integrated into the main Puppy but was easier in Dougals version, so it might be worth contacting him directly.

I look forward to this and it would be a welcome feature to the next Puppy after Deep Thought or if ready for Deep Thought.

Cool

_________________
Puppy WIKI
Back to top
View user's profile Send_private_message Visit_website 
pa_mcclamrock


Joined: 03 Jun 2005
Posts: 657
Location: Fort Wayne, Indiana, USA

PostPosted: Thu 05 Feb 2009, 08:03    Post_subject:  

Lobster wrote:
I look forward to this and it would be a welcome feature to the next Puppy after Deep Thought or if ready for Deep Thought.


Thanks! I think I'm fairly far along with it, and it may even be ready for 4.2 Alpha4, depending on (1) how soon Alpha4 comes out and (2) how soon I can get specific answers to my questions. Smile

David McClamrock

_________________
It's stupid to use inferior software for ideological reasons.
--Linus Torvalds

Visit Pa Penguin's Icebox: http://www.pa-mcclamrock.com/icebox.html
Back to top
View user's profile Send_private_message Visit_website 
technosaurus


Joined: 18 May 2008
Posts: 4353

PostPosted: Thu 05 Feb 2009, 10:03    Post_subject:  

I had a look at the remasterpup2 script a while back and used it to make a mkaddonsfs script ... started toward what you are working on - I can upload my scripts tonight when I get home, but the original script is already posted in the forum - I'll edit my post with a link when I find it.... there is also a dir2iso script floating around somewhere - I have it set up to where you manually add the other files (except pup_XXX.sfs) from the cd

Edit: http://www.murga-linux.com/puppy/viewtopic.php?t=37856

& dir2iso http://www.murga-linux.com/puppy/viewtopic.php?search_id=2139761034&t=36573

_________________
Web Programming - Pet Packaging 100 & 101

Edited_times_total
Back to top
View user's profile Send_private_message 
tasmod


Joined: 04 Dec 2008
Posts: 1461
Location: North Lincolnshire. UK

PostPosted: Thu 05 Feb 2009, 11:10    Post_subject:  

David,
I'm not a programmer or anything but I was just looking at the pwidgets and noticed that Puppy Space seems to reflect the puppy space plus amount used. Is that any help?

Rob Lane
Back to top
View user's profile Send_private_message Visit_website 
technosaurus


Joined: 18 May 2008
Posts: 4353

PostPosted: Thu 05 Feb 2009, 18:10    Post_subject:  

I wouldn't remove the zdrv because it can still be used and I do frequently as a way to test how an sfs will work with default versions and puplets (if you name an addon to zdrv you don't need a pupsave file to load it - its automatic)

for /root/.packages/*.files - I left them in to allow people to more easily remove programs from puplets (petbegone only works for default apps). Either way is fine - depends on how you want it to be used I guess

syncs - I think? it is to minimize disk usage... otherwise you need the space of the full expanded file system + the size of your sfs (I got rid of them in my script, but only because I wanted the user to have more control)

Is boot.cat automatically generated when making the iso bootable? & goofy.16 the boot picture (usually tux)?

_________________
Web Programming - Pet Packaging 100 & 101
Back to top
View user's profile Send_private_message 
pa_mcclamrock


Joined: 03 Jun 2005
Posts: 657
Location: Fort Wayne, Indiana, USA

PostPosted: Fri 06 Feb 2009, 08:26    Post_subject:  

tasmod wrote:
I'm not a programmer or anything but I was just looking at the pwidgets and noticed that Puppy Space seems to reflect the puppy space plus amount used. Is that any help?


Um, don't know offhand, but I figured out a way to avoid having to predict the exact size of the pup_xxx.sfs file. (Never do things the hard way, I say, if you can legitimately get away with doing them the easy way--but thanks for trying to help me do this the hard way!)

(1) For the purpose of estimating the amount of working space needed, I'll just compare the calculated size of the uncompressed filesystems (plus a bit extra for the kernel and other files to be copied from the existing live-CD) with the size of the ISO I get when this program actually works, and then overestimate somewhat to allow for the possibility that other people's systems may include more already compressed files than mine does.

(2) For the purpose of the progressbar, I'll just go by a count of the total number of files to be copied. The progress may be a bit uneven, what with big and little files, but I'm pretty sure nobody will complain too much. There will be a little "program output" window at the bottom, in which the user will be able to see filenames whizzing past at top speed (if the files are small) or not (if they're big) during this step of the process.

_________________
It's stupid to use inferior software for ideological reasons.
--Linus Torvalds

Visit Pa Penguin's Icebox: http://www.pa-mcclamrock.com/icebox.html
Back to top
View user's profile Send_private_message Visit_website 
pa_mcclamrock


Joined: 03 Jun 2005
Posts: 657
Location: Fort Wayne, Indiana, USA

PostPosted: Fri 06 Feb 2009, 08:33    Post_subject:  

technosaurus wrote:
I wouldn't remove the zdrv because it can still be used and I do frequently as a way to test how an sfs will work with default versions and puplets (if you name an addon to zdrv you don't need a pupsave file to load it - its automatic)

for /root/.packages/*.files - I left them in to allow people to more easily remove programs from puplets (petbegone only works for default apps). Either way is fine - depends on how you want it to be used I guess

syncs - I think? it is to minimize disk usage... otherwise you need the space of the full expanded file system + the size of your sfs (I got rid of them in my script, but only because I wanted the user to have more control)

Is boot.cat automatically generated when making the iso bootable? & goofy.16 the boot picture (usually tux)?


I looked at the "mkisofs" man page and, sure enough, boot.cat is automatically generated. Don't know what goofy.16 is, but I can't copy it since I don't have it on my live-CD.

Thanks--I think I've got enough answers to proceed--hope I'll get done pretty soon!

_________________
It's stupid to use inferior software for ideological reasons.
--Linus Torvalds

Visit Pa Penguin's Icebox: http://www.pa-mcclamrock.com/icebox.html
Back to top
View user's profile Send_private_message Visit_website 
seeker

Joined: 26 Nov 2008
Posts: 89
Location: Indiana, USA

PostPosted: Sun 08 Feb 2009, 19:40    Post_subject:  

Quote:
Step Two, estimate working space needed:

This is nit picking, but not everyone has the desktop drive icons enabled. Maybe a reference to Pmount is in order?

By the way, I really like what you are doing with gnocl and tcl/tk.

Seeker
Back to top
View user's profile Send_private_message 
pa_mcclamrock


Joined: 03 Jun 2005
Posts: 657
Location: Fort Wayne, Indiana, USA

PostPosted: Sun 08 Feb 2009, 21:01    Post_subject:  

seeker wrote:
This is nit picking, but not everyone has the desktop drive icons enabled. Maybe a reference to Pmount is in order?


Evidently so. A quick change from "click the desktop icon" to "click the desktop icon or use Pmount" should do the job.

Quote:
By the way, I really like what you are doing with gnocl and tcl/tk.


Thanks! I think I'm almost ready to try the complete remastering process with Remaster Express. The Gnocl progressbar seems to be easier to get working right than a Tk "ttk::progressbar"; I've done test runs of the "squashfile-making" part of the process with a little squashfile (incorporating the contents of /bin only) instead of a big one, and it works. For making an ISO and burning a CD, the procedures I already use in WISH CD-Writer (http://www.geocities.com/pa_mcclamrock/wishcd.html) should work with little or no modification, and I'm pretty sure I've figured out how to extract the relevant numbers from the "mkisofs" and "cdrecord" output to make the progressbar work. If I give it a try tonight, though, I'll run a serious risk of staying up too late squashing bugs, and I've got to get to work tomorrow morning. Rolling Eyes Soon, very soon!

David McClamrock

_________________
It's stupid to use inferior software for ideological reasons.
--Linus Torvalds

Visit Pa Penguin's Icebox: http://www.pa-mcclamrock.com/icebox.html
Back to top
View user's profile Send_private_message Visit_website 
Pizzasgood


Joined: 04 May 2005
Posts: 6270
Location: Knoxville, TN, USA

PostPosted: Mon 09 Feb 2009, 02:31    Post_subject:  

goofy.16 is the name of a splash image that isolinux can display during the pause before it starts booting. It's an arbitrary name. Way back in the early 1.0.x days I made a splash screen but couldn't decide on which font was best, so I uploaded several versions named after the fonts. Barry chose the one using the goofy font, and for some reason left it named goofy.16 rather than renaming it to something like boot.16 or image.16. Then he continued using that name in future releases using completely different images.

We haven't used a splash image since we switched to Puppy 2.x.

Until 4.2 that is. Now we have one again. This time I've revised the system somewhat, so that the USB installs can also have it, and can have the pause so they can add things like 'puppy pfix=ram' or whatever.

So you need to know how the new system works:
There is now a directory at /usr/lib/syslinux/files/ which contains a number of files. They should all appear in the ISO. They're also copied onto a USB drive when installed to one. This is done with an *, not by hardcoding filenames.

There is a catch: the syslinux.cfg file in that location is not quite complete, because the final line depends on how Puppy is being installed (or whether being placed on a CD). It's the "append" line that looks like this for a CD version:
Code:
append initrd=initrd.gz pmedia=cd

In addition to appending the append line, the file also needs to be renamed to isolinux.cfg for a CD to use it. (USB installs use it as syslinux.cfg, except when they have an ext2/3 filesystem, in which case they want extlinux.conf - this naming convention is part of the syslinux suit, not my doing...)

This way we don't have hardcoded files, so if somebody wants to add a second screen to the pause (like the F2 screen in 4.2) they don't have to go and update all the remaster scripts. And they are also used in USB installs, because I updated the install script to look in /usr/lib/syslinux/files too.

_________________
Between depriving a man of one hour from his life and depriving him of his life there exists only a difference of degree. --Muad'Dib

Back to top
View user's profile Send_private_message Visit_website 
pa_mcclamrock


Joined: 03 Jun 2005
Posts: 657
Location: Fort Wayne, Indiana, USA

PostPosted: Sat 14 Feb 2009, 07:25    Post_subject:  

Here are links to the completed set of screenshots.

Initial "Welcome" window: http://www.geocities.com/pa_mcclamrock/remaster01.1.gif

Remaster Slimmer window, for removing built-in packages before remastering: http://www.geocities.com/pa_mcclamrock/remasterslim.gif

Step Two, estimate working space needed: http://www.geocities.com/pa_mcclamrock/remaster02.5.gif

Show estimated working space and select working partition: http://www.geocities.com/pa_mcclamrock/remaster02.7.gif

Step Three, show information about remastering while making heap big squashfile: http://www.geocities.com/pa_mcclamrock/remaster03.gif

Step Four, verify that contents of /tmp/root include everything desired: http://www.geocities.com/pa_mcclamrock/remaster04.gif

Step Five, do the same with /tmp/etc and specify hardware configuration if desired: http://www.geocities.com/pa_mcclamrock/remaster05.gif

Step Six, edit isolinux.cfg if desired: http://www.geocities.com/pa_mcclamrock/remaster06.1.gif

Step Seven, rename pup_xxx.sfs file if desired: http://www.geocities.com/pa_mcclamrock/remaster07.gif

Step Eight, select "Burn Only," "Save ISO Only," or "Save ISO & Burn": http://www.geocities.com/pa_mcclamrock/remaster08.gif

Step Nine, finish up by displaying more reading matter while live-CD gets burned, or at least ISO gets created: http://www.geocities.com/pa_mcclamrock/remaster09.gif

_________________
It's stupid to use inferior software for ideological reasons.
--Linus Torvalds

Visit Pa Penguin's Icebox: http://www.pa-mcclamrock.com/icebox.html
Back to top
View user's profile Send_private_message Visit_website 
pa_mcclamrock


Joined: 03 Jun 2005
Posts: 657
Location: Fort Wayne, Indiana, USA

PostPosted: Sat 14 Feb 2009, 07:56    Post_subject:  

Pizzasgood wrote:
We haven't used a splash image since we switched to Puppy 2.x.

Until 4.2 that is. Now we have one again. This time I've revised the system somewhat, so that the USB installs can also have it, and can have the pause so they can add things like 'puppy pfix=ram' or whatever.

So you need to know how the new system works:
There is now a directory at /usr/lib/syslinux/files/ which contains a number of files [details follow].


Um, always looking for the easy way if feasible, I ask: If I just (1) list all the files I see on a Puppy 4.2 Alpha4 live-CD (except pup_420.sfs and boot.cat) for copying to the new ISO and/or CD, and (2) include copies of syslinux.cfg under the names isolinux.cfg and extlinux.conf, will that do the job for the 4.2 version of Remaster Express?

_________________
It's stupid to use inferior software for ideological reasons.
--Linus Torvalds

Visit Pa Penguin's Icebox: http://www.pa-mcclamrock.com/icebox.html
Back to top
View user's profile Send_private_message Visit_website 
Pizzasgood


Joined: 04 May 2005
Posts: 6270
Location: Knoxville, TN, USA

PostPosted: Sat 14 Feb 2009, 14:41    Post_subject:  

The key thing is to hard code as few names as possible, so that we can modify the files included in the ISO later without having to edit half a million scripts to handle the new names.

Now that I think about it, I guess the new method doesn't really affect a remaster so much as it affects installations and such. Unless you're allowing a remaster without an existing CD/ISO, in which case you would need to go into that directory to get the files, and append this line to the new isolinux.cfg file:
Code:
append initrd=initrd.gz pmedia=cd


But when it comes to remastering from an existing CD / ISO, you could get away with just doing something like this:
Code:
mount -o loop old_puppy.iso old_iso_dir
rsync -r --exclude=boot.cat --exclude puppy_[0-9][0-9][0-9].sfs old_iso_dir/ new_iso_dir/
umount old_iso_dir

That would automatically copy everything, directories included, from the old iso into the new one, with the exception that it will exclude pup_xxx.sfs and boot.cat. (I used rsync because cp doesn't have an exclude option).

Then just create the pup_xxx.sfs file.

_________________
Between depriving a man of one hour from his life and depriving him of his life there exists only a difference of degree. --Muad'Dib

Back to top
View user's profile Send_private_message Visit_website 
pa_mcclamrock


Joined: 03 Jun 2005
Posts: 657
Location: Fort Wayne, Indiana, USA

PostPosted: Sat 14 Feb 2009, 19:48    Post_subject:  

Pizzasgood wrote:
The key thing is to hard code as few names as possible, so that we can modify the files included in the ISO later without having to edit half a million scripts to handle the new names.


Fair enough.

Quote:
Now that I think about it, I guess the new method doesn't really affect a remaster so much as it affects installations and such. Unless you're allowing a remaster without an existing CD/ISO [ . . . ]


I'm not!

Quote:
But when it comes to remastering from an existing CD / ISO, you could get away with just doing something like this:
Code:
mount -o loop old_puppy.iso old_iso_dir
rsync -r --exclude=boot.cat --exclude puppy_[0-9][0-9][0-9].sfs old_iso_dir/ new_iso_dir/
umount old_iso_dir


Let me see, in Tcl (if $LIVEDRIVE stands for the CD, which is already mounted at this point in my code, and $PUPBUILD is the /puppylivecdbuild directory) that would be:

Code:
set cd_files [glob -directory $LIVEDRIVE *]
foreach f $cd_files {
   if {[regexp "boot.cat|pup_\[0-9\]+?.+?\.sfs" $f] == 0} {
      file copy -force $f $PUPBUILD
   }
}


Thanks--that's the last finishing touch! I'm now running Puppy 4.1.1 from a live-CD I burned with Remaster Express, and the total size is less than official Puppy 4.1.1 because of uninstallation of built-in packages. I'll package up the 4.1.1 version and release it at the first opportunity. Then I'll see if I can get the 4.2 Beta packages.txt and 0pkgs_db.pet, and produce a 4.2 version--which, if all goes well, may have the honor of being the first new package for Puppy 4.2 that missed the beta deadline! Rolling Eyes

_________________
It's stupid to use inferior software for ideological reasons.
--Linus Torvalds

Visit Pa Penguin's Icebox: http://www.pa-mcclamrock.com/icebox.html
Back to top
View user's profile Send_private_message Visit_website 
Display_posts:   Sort by:   
Page 1 of 2 Posts_count   Goto page: 1, 2 Next
Post_new_topic   Reply_to_topic View_previous_topic :: View_next_topic
 Forum index » Advanced Topics » Puppy Projects
Jump to:  

Rules_post_cannot
Rules_reply_cannot
Rules_edit_cannot
Rules_delete_cannot
Rules_vote_cannot
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.1253s ][ Queries: 13 (0.0059s) ][ GZIP on ]