The time now is Thu 23 May 2013, 15:47
All times are UTC - 4 |
|
Page 1 of 2 [21 Posts] |
Goto page: 1, 2 Next |
| Author |
Message |
pa_mcclamrock

Joined: 03 Jun 2005 Posts: 657 Location: Fort Wayne, Indiana, USA
|
Posted: Wed 04 Feb 2009, 18:33 Post subject:
Remastering remastering Subject description: Questions on new GTK2-look script for making custom live-CDs |
|
I'm working on an app tentatively entitled "Remaster Express" to make the Puppy custom live-CD remastering process more versatile and more user-friendly, with a GUI that persists throughout the process, incorporates more options than the existing remaster script (/usr/sbin/remasterpup2), and takes on the selected GTK+ 2.0 theme (by way of Gnocl). Here are links to some screenshots of what I've got so far. [EDIT 2009 Feb 14: Don't look at these any more, look at the completed set of screenshots with the links in the message I just posted below.)
Initial "Welcome" window: http://www.geocities.com/pa_mcclamrock/remaster01.gif
Remaster Slimmer window, for removing built-in packages before remastering: http://www.geocities.com/pa_mcclamrock/remasterslim.gif
Step Two, estimate working space needed: http://www.geocities.com/pa_mcclamrock/remaster02.3.gif
Show estimated working space and select working partition: http://www.geocities.com/pa_mcclamrock/remaster02.6.gif
Step Three, show information about remastering while making heap big squashfile: http://www.geocities.com/pa_mcclamrock/remaster03.gif
Step Four, verify that contents of /tmp/root include everything desired: http://www.geocities.com/pa_mcclamrock/remaster04.gif
Step Five, do the same with /tmp/etc and specify hardware configuration if desired: http://www.geocities.com/pa_mcclamrock/remaster05.gif
Step Six, edit isolinux.cfg if desired: http://www.geocities.com/pa_mcclamrock/remaster06.1.gif
Step Seven, rename pup_xxx.sfs file if desired:
http://www.geocities.com/pa_mcclamrock/remaster07.gif
I don't have the rest done yet, but I do have several questions that I hope somebody here can answer.
* How do you best estimate the size of the pup_xxx.sfs file compared to the uncompressed size of the directories that go into it? The "remasterpup2" script just seems to guess that maybe it will be about 1/3 as big, and maybe you'll need, oh, 150 to 200 MB of additional free space if you're going to save an ISO. Is it possible to do better than this? As I said above, I'd like to have a progressbar to show what percentage of stuff has been squashed into the squashfile and what percentage remains to be squashed, but this would seem to require a pretty good estimate of how big the whole thing will be when done.
* I need to make sure I know what to do with the various "packages" files in /root/.packages. I observe that "packages.txt" has the complete list of official packages from the current Puppy Unleashed; "livepackages.txt" is the same as the "Official Puppy 4" list you see in PETget; "livepackages2.txt" and "livepackages3.txt" are the same as Official Puppy 2 and Official Puppy 3 respectively; and "alienpackages.txt" lists the unofficial packages you have installed. To this I have added "slimpacks-xxx.txt," a modified list of built-in packages the user might wish to uninstall with "Remaster Slimmer" before remastering.
What I want to do, I think, is to (1) copy "packages.txt" intact to the /tmp/root/.packages directory for inclusion in the new CD; (2) copy the various "livepackages" files and "slimpacks-xxx.txt" to /tmp/root/.packages too, but then (3) move all the "on" lines out of the "livepackages" copies and merge them into /tmp/root/.packages/slimpacks-xxx.txt, since the packages named in these lines will become built-in packages in the new CD and will be uninstallable with Remaster Slimmer but not with PETget; and (4) omit "alienpackages," since all the packages from there will become built-in packages in the new CD. Does that sound right?
* Following line 196, "#now copy the files... ," "remasterpup2" omits "boot.cat" from the list of files to be copied from the existing live-CD, and includes something called "goofy.16" instead. What's this, and why?
* Are all these "syncs" really necessary? Almost everywhere you do anything with files in "remasterpup2," it seems, there's a "sync" before and after. Would there be any significant chance of disaster if you didn't have quite so many "syncs"?
* All references to "zdrv" can be omitted for Puppy 4.1 and later, can't they?
Thanks in advance for any help!
David McClamrock
_________________ It's stupid to use inferior software for ideological reasons.
--Linus Torvalds
Visit Pa Penguin's Icebox: http://www.pa-mcclamrock.com/icebox.html
Last edited by pa_mcclamrock on Sat 14 Feb 2009, 07:14; edited 7 times in total
|
|
Back to top
|
|
 |
Lobster
Official Crustacean

Joined: 04 May 2005 Posts: 15109 Location: Paradox Realm
|
Posted: Wed 04 Feb 2009, 23:28 Post subject:
|
|
The last easy remaster was from Dougal and worked very well.
It was integrated into the main Puppy but was easier in Dougals version, so it might be worth contacting him directly.
I look forward to this and it would be a welcome feature to the next Puppy after Deep Thought or if ready for Deep Thought.
_________________ Puppy WIKI
|
|
Back to top
|
|
 |
pa_mcclamrock

Joined: 03 Jun 2005 Posts: 657 Location: Fort Wayne, Indiana, USA
|
Posted: Thu 05 Feb 2009, 08:03 Post subject:
|
|
| Lobster wrote: | | I look forward to this and it would be a welcome feature to the next Puppy after Deep Thought or if ready for Deep Thought. |
Thanks! I think I'm fairly far along with it, and it may even be ready for 4.2 Alpha4, depending on (1) how soon Alpha4 comes out and (2) how soon I can get specific answers to my questions.
David McClamrock
_________________ It's stupid to use inferior software for ideological reasons.
--Linus Torvalds
Visit Pa Penguin's Icebox: http://www.pa-mcclamrock.com/icebox.html
|
|
Back to top
|
|
 |
technosaurus

Joined: 18 May 2008 Posts: 3843
|
Posted: Thu 05 Feb 2009, 10:03 Post subject:
|
|
I had a look at the remasterpup2 script a while back and used it to make a mkaddonsfs script ... started toward what you are working on - I can upload my scripts tonight when I get home, but the original script is already posted in the forum - I'll edit my post with a link when I find it.... there is also a dir2iso script floating around somewhere - I have it set up to where you manually add the other files (except pup_XXX.sfs) from the cd
Edit: http://www.murga-linux.com/puppy/viewtopic.php?t=37856
& dir2iso http://www.murga-linux.com/puppy/viewtopic.php?search_id=2139761034&t=36573
_________________ Puppy Web Desktop Now with pet packages - Pet Packaging 100 & 101
Last edited by technosaurus on Thu 05 Feb 2009, 12:47; edited 2 times in total
|
|
Back to top
|
|
 |
tasmod

Joined: 04 Dec 2008 Posts: 1459 Location: North Lincolnshire. UK
|
Posted: Thu 05 Feb 2009, 11:10 Post subject:
|
|
David,
I'm not a programmer or anything but I was just looking at the pwidgets and noticed that Puppy Space seems to reflect the puppy space plus amount used. Is that any help?
Rob Lane
|
|
Back to top
|
|
 |
technosaurus

Joined: 18 May 2008 Posts: 3843
|
Posted: Thu 05 Feb 2009, 18:10 Post subject:
|
|
I wouldn't remove the zdrv because it can still be used and I do frequently as a way to test how an sfs will work with default versions and puplets (if you name an addon to zdrv you don't need a pupsave file to load it - its automatic)
for /root/.packages/*.files - I left them in to allow people to more easily remove programs from puplets (petbegone only works for default apps). Either way is fine - depends on how you want it to be used I guess
syncs - I think? it is to minimize disk usage... otherwise you need the space of the full expanded file system + the size of your sfs (I got rid of them in my script, but only because I wanted the user to have more control)
Is boot.cat automatically generated when making the iso bootable? & goofy.16 the boot picture (usually tux)?
_________________ Puppy Web Desktop Now with pet packages - Pet Packaging 100 & 101
|
|
Back to top
|
|
 |
pa_mcclamrock

Joined: 03 Jun 2005 Posts: 657 Location: Fort Wayne, Indiana, USA
|
Posted: Fri 06 Feb 2009, 08:26 Post subject:
|
|
| tasmod wrote: | | I'm not a programmer or anything but I was just looking at the pwidgets and noticed that Puppy Space seems to reflect the puppy space plus amount used. Is that any help? |
Um, don't know offhand, but I figured out a way to avoid having to predict the exact size of the pup_xxx.sfs file. (Never do things the hard way, I say, if you can legitimately get away with doing them the easy way--but thanks for trying to help me do this the hard way!)
(1) For the purpose of estimating the amount of working space needed, I'll just compare the calculated size of the uncompressed filesystems (plus a bit extra for the kernel and other files to be copied from the existing live-CD) with the size of the ISO I get when this program actually works, and then overestimate somewhat to allow for the possibility that other people's systems may include more already compressed files than mine does.
(2) For the purpose of the progressbar, I'll just go by a count of the total number of files to be copied. The progress may be a bit uneven, what with big and little files, but I'm pretty sure nobody will complain too much. There will be a little "program output" window at the bottom, in which the user will be able to see filenames whizzing past at top speed (if the files are small) or not (if they're big) during this step of the process.
_________________ It's stupid to use inferior software for ideological reasons.
--Linus Torvalds
Visit Pa Penguin's Icebox: http://www.pa-mcclamrock.com/icebox.html
|
|
Back to top
|
|
 |
pa_mcclamrock

Joined: 03 Jun 2005 Posts: 657 Location: Fort Wayne, Indiana, USA
|
Posted: Fri 06 Feb 2009, 08:33 Post subject:
|
|
| technosaurus wrote: | I wouldn't remove the zdrv because it can still be used and I do frequently as a way to test how an sfs will work with default versions and puplets (if you name an addon to zdrv you don't need a pupsave file to load it - its automatic)
for /root/.packages/*.files - I left them in to allow people to more easily remove programs from puplets (petbegone only works for default apps). Either way is fine - depends on how you want it to be used I guess
syncs - I think? it is to minimize disk usage... otherwise you need the space of the full expanded file system + the size of your sfs (I got rid of them in my script, but only because I wanted the user to have more control)
Is boot.cat automatically generated when making the iso bootable? & goofy.16 the boot picture (usually tux)? |
I looked at the "mkisofs" man page and, sure enough, boot.cat is automatically generated. Don't know what goofy.16 is, but I can't copy it since I don't have it on my live-CD.
Thanks--I think I've got enough answers to proceed--hope I'll get done pretty soon!
_________________ It's stupid to use inferior software for ideological reasons.
--Linus Torvalds
Visit Pa Penguin's Icebox: http://www.pa-mcclamrock.com/icebox.html
|
|
Back to top
|
|
 |
seeker
Joined: 26 Nov 2008 Posts: 89 Location: Indiana, USA
|
Posted: Sun 08 Feb 2009, 19:40 Post subject:
|
|
| Quote: | | Step Two, estimate working space needed: |
This is nit picking, but not everyone has the desktop drive icons enabled. Maybe a reference to Pmount is in order?
By the way, I really like what you are doing with gnocl and tcl/tk.
Seeker
|
|
Back to top
|
|
 |
pa_mcclamrock

Joined: 03 Jun 2005 Posts: 657 Location: Fort Wayne, Indiana, USA
|
Posted: Sun 08 Feb 2009, 21:01 Post subject:
|
|
| seeker wrote: | | This is nit picking, but not everyone has the desktop drive icons enabled. Maybe a reference to Pmount is in order? |
Evidently so. A quick change from "click the desktop icon" to "click the desktop icon or use Pmount" should do the job.
| Quote: | | By the way, I really like what you are doing with gnocl and tcl/tk. |
Thanks! I think I'm almost ready to try the complete remastering process with Remaster Express. The Gnocl progressbar seems to be easier to get working right than a Tk "ttk::progressbar"; I've done test runs of the "squashfile-making" part of the process with a little squashfile (incorporating the contents of /bin only) instead of a big one, and it works. For making an ISO and burning a CD, the procedures I already use in WISH CD-Writer (http://www.geocities.com/pa_mcclamrock/wishcd.html) should work with little or no modification, and I'm pretty sure I've figured out how to extract the relevant numbers from the "mkisofs" and "cdrecord" output to make the progressbar work. If I give it a try tonight, though, I'll run a serious risk of staying up too late squashing bugs, and I've got to get to work tomorrow morning. Soon, very soon!
David McClamrock
_________________ It's stupid to use inferior software for ideological reasons.
--Linus Torvalds
Visit Pa Penguin's Icebox: http://www.pa-mcclamrock.com/icebox.html
|
|
Back to top
|
|
 |
Pizzasgood

Joined: 04 May 2005 Posts: 6270 Location: Knoxville, TN, USA
|
Posted: Mon 09 Feb 2009, 02:31 Post subject:
|
|
goofy.16 is the name of a splash image that isolinux can display during the pause before it starts booting. It's an arbitrary name. Way back in the early 1.0.x days I made a splash screen but couldn't decide on which font was best, so I uploaded several versions named after the fonts. Barry chose the one using the goofy font, and for some reason left it named goofy.16 rather than renaming it to something like boot.16 or image.16. Then he continued using that name in future releases using completely different images.
We haven't used a splash image since we switched to Puppy 2.x.
Until 4.2 that is. Now we have one again. This time I've revised the system somewhat, so that the USB installs can also have it, and can have the pause so they can add things like 'puppy pfix=ram' or whatever.
So you need to know how the new system works:
There is now a directory at /usr/lib/syslinux/files/ which contains a number of files. They should all appear in the ISO. They're also copied onto a USB drive when installed to one. This is done with an *, not by hardcoding filenames.
There is a catch: the syslinux.cfg file in that location is not quite complete, because the final line depends on how Puppy is being installed (or whether being placed on a CD). It's the "append" line that looks like this for a CD version:
| Code: | | append initrd=initrd.gz pmedia=cd |
In addition to appending the append line, the file also needs to be renamed to isolinux.cfg for a CD to use it. (USB installs use it as syslinux.cfg, except when they have an ext2/3 filesystem, in which case they want extlinux.conf - this naming convention is part of the syslinux suit, not my doing...)
This way we don't have hardcoded files, so if somebody wants to add a second screen to the pause (like the F2 screen in 4.2) they don't have to go and update all the remaster scripts. And they are also used in USB installs, because I updated the install script to look in /usr/lib/syslinux/files too.
_________________ Between depriving a man of one hour from his life and depriving him of his life there exists only a difference of degree. --Muad'Dib

|
|
Back to top
|
|
 |
pa_mcclamrock

Joined: 03 Jun 2005 Posts: 657 Location: Fort Wayne, Indiana, USA
|
Posted: Sat 14 Feb 2009, 07:25 Post subject:
|
|
Here are links to the completed set of screenshots.
Initial "Welcome" window: http://www.geocities.com/pa_mcclamrock/remaster01.1.gif
Remaster Slimmer window, for removing built-in packages before remastering: http://www.geocities.com/pa_mcclamrock/remasterslim.gif
Step Two, estimate working space needed: http://www.geocities.com/pa_mcclamrock/remaster02.5.gif
Show estimated working space and select working partition: http://www.geocities.com/pa_mcclamrock/remaster02.7.gif
Step Three, show information about remastering while making heap big squashfile: http://www.geocities.com/pa_mcclamrock/remaster03.gif
Step Four, verify that contents of /tmp/root include everything desired: http://www.geocities.com/pa_mcclamrock/remaster04.gif
Step Five, do the same with /tmp/etc and specify hardware configuration if desired: http://www.geocities.com/pa_mcclamrock/remaster05.gif
Step Six, edit isolinux.cfg if desired: http://www.geocities.com/pa_mcclamrock/remaster06.1.gif
Step Seven, rename pup_xxx.sfs file if desired: http://www.geocities.com/pa_mcclamrock/remaster07.gif
Step Eight, select "Burn Only," "Save ISO Only," or "Save ISO & Burn": http://www.geocities.com/pa_mcclamrock/remaster08.gif
Step Nine, finish up by displaying more reading matter while live-CD gets burned, or at least ISO gets created: http://www.geocities.com/pa_mcclamrock/remaster09.gif
_________________ It's stupid to use inferior software for ideological reasons.
--Linus Torvalds
Visit Pa Penguin's Icebox: http://www.pa-mcclamrock.com/icebox.html
|
|
Back to top
|
|
 |
pa_mcclamrock

Joined: 03 Jun 2005 Posts: 657 Location: Fort Wayne, Indiana, USA
|
Posted: Sat 14 Feb 2009, 07:56 Post subject:
|
|
| Pizzasgood wrote: | We haven't used a splash image since we switched to Puppy 2.x.
Until 4.2 that is. Now we have one again. This time I've revised the system somewhat, so that the USB installs can also have it, and can have the pause so they can add things like 'puppy pfix=ram' or whatever.
So you need to know how the new system works:
There is now a directory at /usr/lib/syslinux/files/ which contains a number of files [details follow]. |
Um, always looking for the easy way if feasible, I ask: If I just (1) list all the files I see on a Puppy 4.2 Alpha4 live-CD (except pup_420.sfs and boot.cat) for copying to the new ISO and/or CD, and (2) include copies of syslinux.cfg under the names isolinux.cfg and extlinux.conf, will that do the job for the 4.2 version of Remaster Express?
_________________ It's stupid to use inferior software for ideological reasons.
--Linus Torvalds
Visit Pa Penguin's Icebox: http://www.pa-mcclamrock.com/icebox.html
|
|
Back to top
|
|
 |
Pizzasgood

Joined: 04 May 2005 Posts: 6270 Location: Knoxville, TN, USA
|
Posted: Sat 14 Feb 2009, 14:41 Post subject:
|
|
The key thing is to hard code as few names as possible, so that we can modify the files included in the ISO later without having to edit half a million scripts to handle the new names.
Now that I think about it, I guess the new method doesn't really affect a remaster so much as it affects installations and such. Unless you're allowing a remaster without an existing CD/ISO, in which case you would need to go into that directory to get the files, and append this line to the new isolinux.cfg file:
| Code: | | append initrd=initrd.gz pmedia=cd |
But when it comes to remastering from an existing CD / ISO, you could get away with just doing something like this:
| Code: | mount -o loop old_puppy.iso old_iso_dir
rsync -r --exclude=boot.cat --exclude puppy_[0-9][0-9][0-9].sfs old_iso_dir/ new_iso_dir/
umount old_iso_dir |
That would automatically copy everything, directories included, from the old iso into the new one, with the exception that it will exclude pup_xxx.sfs and boot.cat. (I used rsync because cp doesn't have an exclude option).
Then just create the pup_xxx.sfs file.
_________________ Between depriving a man of one hour from his life and depriving him of his life there exists only a difference of degree. --Muad'Dib

|
|
Back to top
|
|
 |
pa_mcclamrock

Joined: 03 Jun 2005 Posts: 657 Location: Fort Wayne, Indiana, USA
|
Posted: Sat 14 Feb 2009, 19:48 Post subject:
|
|
| Pizzasgood wrote: | | The key thing is to hard code as few names as possible, so that we can modify the files included in the ISO later without having to edit half a million scripts to handle the new names. |
Fair enough.
| Quote: | | Now that I think about it, I guess the new method doesn't really affect a remaster so much as it affects installations and such. Unless you're allowing a remaster without an existing CD/ISO [ . . . ] |
I'm not!
| Quote: | But when it comes to remastering from an existing CD / ISO, you could get away with just doing something like this:
| Code: | mount -o loop old_puppy.iso old_iso_dir
rsync -r --exclude=boot.cat --exclude puppy_[0-9][0-9][0-9].sfs old_iso_dir/ new_iso_dir/
umount old_iso_dir |
|
Let me see, in Tcl (if $LIVEDRIVE stands for the CD, which is already mounted at this point in my code, and $PUPBUILD is the /puppylivecdbuild directory) that would be:
| Code: | set cd_files [glob -directory $LIVEDRIVE *]
foreach f $cd_files {
if {[regexp "boot.cat|pup_\[0-9\]+?.+?\.sfs" $f] == 0} {
file copy -force $f $PUPBUILD
}
} |
Thanks--that's the last finishing touch! I'm now running Puppy 4.1.1 from a live-CD I burned with Remaster Express, and the total size is less than official Puppy 4.1.1 because of uninstallation of built-in packages. I'll package up the 4.1.1 version and release it at the first opportunity. Then I'll see if I can get the 4.2 Beta packages.txt and 0pkgs_db.pet, and produce a 4.2 version--which, if all goes well, may have the honor of being the first new package for Puppy 4.2 that missed the beta deadline!
_________________ It's stupid to use inferior software for ideological reasons.
--Linus Torvalds
Visit Pa Penguin's Icebox: http://www.pa-mcclamrock.com/icebox.html
|
|
Back to top
|
|
 |
|
|
Page 1 of 2 [21 Posts] |
Goto page: 1, 2 Next |
|
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum You cannot attach files in this forum You can download files in this forum
|
Powered by phpBB © 2001, 2005 phpBB Group
|