@tclhost
there are three scripts
1) sfsdirect-remaster is the newest and so far it just creates an sfs which should replace your main sfs on a frugal install. Maybe just rename the old one first until you are sure the new one boots.
2) unattendedremaster is the most versatile with special functions, squash an existing remaster directory or make iso from previous remaster, or make new remaster and new iso without interaction, suitable for all puppies.
3) unresync, sync your live system with an existing remaster directory for updates, or create a new remaster directory only.
New Remaster script Working!!
-
stemsee
I just realised something .... namely the script assumes 'huge' kernel is used i.e. the modules and firmware are not included in the remaster. I will add code to differentiate! That is why Jasper's sfs is smaller then expected. oops simply remove /lib/modules /lib/firmware from mksquashfs command for now.
The script is now working with Studio1337. As the problem was with modules & firmware, also in /root/.cache/ there is a menu configuration file which is essential.
There is a need for each distribution to have a specific exclusion file (a txt file containing line by line list of folders and files to be excluded in the mksquashfs command). Currently the script uses mksquashfs source-1 source-2 destination.sfs -e 'excluded files and folders', but using -'f' instead of '-e' uses the distro specific txt file of exclusions. These would best be created by each distro's creator/maintainer (or anyone with know how and time to submit such a file to me for inclusion) with provision for likely-to-be-installed apps. Then the script would automatically select the appropriate exclusion file by matching the DISTRO_SPECS variables!
The script creates an sfs from a frugal install, but I see no reason why it would not also work from a full install also (simply have not tested!). But I used to mksquashfs my full drive install and that worked without any exclusions or finesse.
The script is now working with Studio1337. As the problem was with modules & firmware, also in /root/.cache/ there is a menu configuration file which is essential.
There is a need for each distribution to have a specific exclusion file (a txt file containing line by line list of folders and files to be excluded in the mksquashfs command). Currently the script uses mksquashfs source-1 source-2 destination.sfs -e 'excluded files and folders', but using -'f' instead of '-e' uses the distro specific txt file of exclusions. These would best be created by each distro's creator/maintainer (or anyone with know how and time to submit such a file to me for inclusion) with provision for likely-to-be-installed apps. Then the script would automatically select the appropriate exclusion file by matching the DISTRO_SPECS variables!
The script creates an sfs from a frugal install, but I see no reason why it would not also work from a full install also (simply have not tested!). But I used to mksquashfs my full drive install and that worked without any exclusions or finesse.
Hi stemsee,
I promised I would look into this but didn't due to lack of time and various reason, sorry
Anyway I'm glad that you manage to make it work.
cheers!
I promised I would look into this but didn't due to lack of time and various reason, sorry
Anyway I'm glad that you manage to make it work.cheers!
Fatdog64 forum links: [url=http://murga-linux.com/puppy/viewtopic.php?t=117546]Latest version[/url] | [url=https://cutt.ly/ke8sn5H]Contributed packages[/url] | [url=https://cutt.ly/se8scrb]ISO builder[/url]
-
stemsee
Hi Jamesbond
No problem. But I got the scripts working for fatdog 700, studio1337, EmSeeV2.2 and DebianDog. For fatdog 700 when i remastered and got the sfs to 800mb and packed into the initrd, it wouldn't boot. So I wondered what is the limit of the initrd size and why?
Also where do i find the code to auto edit from remaster script the initrd and replace the old sfs with the new one and repack, for FatDog64 700 b1...
Cheers
No problem. But I got the scripts working for fatdog 700, studio1337, EmSeeV2.2 and DebianDog. For fatdog 700 when i remastered and got the sfs to 800mb and packed into the initrd, it wouldn't boot. So I wondered what is the limit of the initrd size and why?
Also where do i find the code to auto edit from remaster script the initrd and replace the old sfs with the new one and repack, for FatDog64 700 b1...
Cheers
The normal limit of initrd is about 30% to 50% of the RAM size by default, and increased to 80% if you use "rootfstype=ramfs" on the bootloader. I haven't tested an SFS of that size for a long time, but I remember I could get it to boot with 2GB of RAM in a qemu. I may be wrong, though, it was a long time ago.stemsee wrote:No problem. But I got the scripts working for fatdog 700, studio1337, EmSeeV2.2 and DebianDog. For fatdog 700 when i remastered and got the sfs to 800mb and packed into the initrd, it wouldn't boot. So I wondered what is the limit of the initrd size and why?
Look in /usr/bin/filemnt.Also where do i find the code to auto edit from remaster script the initrd and replace the old sfs with the new one and repack, for FatDog64 700 b1...
Of if you can't be bothered, this will expand the initrd:
Code: Select all
mkdir /tmp/xxx; cd /tmp/xxx
cpio -i < /path/to/your/initrdCode: Select all
cd /tmp/xxx
find . | cpio -o -H newc > /path/to/your/new/initrd
Fatdog64 forum links: [url=http://murga-linux.com/puppy/viewtopic.php?t=117546]Latest version[/url] | [url=https://cutt.ly/ke8sn5H]Contributed packages[/url] | [url=https://cutt.ly/se8scrb]ISO builder[/url]
-
stemsee
@Jamesbond
Thanks for the code. It works great! I added the original FatDog and DebianDog boot files for iso and a generic set for 'usual' pups.
The sfs is 5MB.
To do = fine tune the exclude files for the perfectionists! Hopefully a user of each distro will submit an exclude file (-'f' - a line by line list of files and folders to exclude from the remaster for a pristine sfs/iso). But that depends on the take up/interest ... as clearly there is no 'need' for these new scripts, but for newbies to have one package that can remaster every pup/live-distro would be pretty cool.
Thanks for the code. It works great! I added the original FatDog and DebianDog boot files for iso and a generic set for 'usual' pups.
The sfs is 5MB.
To do = fine tune the exclude files for the perfectionists! Hopefully a user of each distro will submit an exclude file (-'f' - a line by line list of files and folders to exclude from the remaster for a pristine sfs/iso). But that depends on the take up/interest ... as clearly there is no 'need' for these new scripts, but for newbies to have one package that can remaster every pup/live-distro would be pretty cool.
Last edited by stemsee on Fri 26 Sep 2014, 20:51, edited 1 time in total.
-
stemsee
-
Jasper
Hi stemsee,
I only use Precise 5.6 in multi-session-live-CD/DVD mode and I'm pleased to report that your SFSDirect is working well.
I think I might need at least 1 GB of ram to take advantage of the super-fast ram option - so I chose not to make my remastered sfs in ram and I entered my chosen hard drive partition - some 5 minutes later my remastered sfs was on that partition and all I had to do was to reboot to use it.
The remaster does include all of your new 3-option-5MB-sfs most of which I don't have any need for.
My regards and thanks
If anyone else wants to make a multi-session-live-CD/DVD they can save to - they must include one dated-save-file, but that's not necessary for those who want their remaster set in stone.
I only use Precise 5.6 in multi-session-live-CD/DVD mode and I'm pleased to report that your SFSDirect is working well.
I think I might need at least 1 GB of ram to take advantage of the super-fast ram option - so I chose not to make my remastered sfs in ram and I entered my chosen hard drive partition - some 5 minutes later my remastered sfs was on that partition and all I had to do was to reboot to use it.
The remaster does include all of your new 3-option-5MB-sfs most of which I don't have any need for.
My regards and thanks
If anyone else wants to make a multi-session-live-CD/DVD they can save to - they must include one dated-save-file, but that's not necessary for those who want their remaster set in stone.
-
stemsee
Hi Jasper
So sfsdirect is slower or faster for you? You can edit the sfs and remove /boot/fd/efiboot.img that weighs 10mb by itself!!!
Dated savefile? please provide details so I can include it or the possibility to create it in the special functions section of the script.! I am also thinking about including kernel/modules/firmware upgrades from my personal repositories.
So sfsdirect is slower or faster for you? You can edit the sfs and remove /boot/fd/efiboot.img that weighs 10mb by itself!!!
Dated savefile? please provide details so I can include it or the possibility to create it in the special functions section of the script.! I am also thinking about including kernel/modules/firmware upgrades from my personal repositories.
-
stemsee
Hi Stemsee.
I was unclear which option to choose for my puppy so it ended up using the automatic default. It started doing its remaster when I noticed my memory was running dangerously low. I noticed the script was using root for all the remaster files. I don't recall if there was a directory option for workspace but if not, you might want to consider adding an option to choose a workspace other than root.
Best,
Slavvo67
I was unclear which option to choose for my puppy so it ended up using the automatic default. It started doing its remaster when I noticed my memory was running dangerously low. I noticed the script was using root for all the remaster files. I don't recall if there was a directory option for workspace but if not, you might want to consider adding an option to choose a workspace other than root.
Best,
Slavvo67
-
stemsee
Slavvo67
Download the latest uploaded 26th evening,. There is more time to read the options. Default is now on /sda2. The version you have has an error which occurred after I added burn to cd/dvd and write to usb functions.
sfsdirect is fastest and gives options for ram (/tmp) or on device (sda/sdb etc)
Download the latest uploaded 26th evening,. There is more time to read the options. Default is now on /sda2. The version you have has an error which occurred after I added burn to cd/dvd and write to usb functions.
sfsdirect is fastest and gives options for ram (/tmp) or on device (sda/sdb etc)
Hi Stemsee:
I just wanted to report back that though backup was fast, it didn't seem to work quite right. The first time I booted, I had garbled lettering under the menu and icons. The next time, the lettering appeared ok, however my wireless usb card wasn't being read so no wlan0. I'll will do more testing but so far, it looks like the remaster needs some tweaking.
Best,
Slavvo67
P.S. - Using OV Precise Retro 5.8
I just wanted to report back that though backup was fast, it didn't seem to work quite right. The first time I booted, I had garbled lettering under the menu and icons. The next time, the lettering appeared ok, however my wireless usb card wasn't being read so no wlan0. I'll will do more testing but so far, it looks like the remaster needs some tweaking.
Best,
Slavvo67
P.S. - Using OV Precise Retro 5.8
-
stemsee
slavvo67
thanks for feedback!
sfs-direct-remaster works from the running system if you happen to use the system while it is remastering that could cause errors. Also
after remaster and before reboot did you remove the savefile? or change the grub kernel boot argument to pfix=ram (or copy, but not fsck)?? As the contents of the savefile should then be included in the remastered .sfs. Also before installing the latest package did you properly remove the previous package?
However, I have again improved the scripts and removed the third script UnReSync which is already a function of the other two scripts.
Now I am testing, it should be ready soon.
thanks for feedback!
sfs-direct-remaster works from the running system if you happen to use the system while it is remastering that could cause errors. Also
after remaster and before reboot did you remove the savefile? or change the grub kernel boot argument to pfix=ram (or copy, but not fsck)?? As the contents of the savefile should then be included in the remastered .sfs. Also before installing the latest package did you properly remove the previous package?
However, I have again improved the scripts and removed the third script UnReSync which is already a function of the other two scripts.
Now I am testing, it should be ready soon.
-
Jasper
Hi stemsee,
From the last item on page 2 of this thread, two days ago, you wrote:
I have now finalised my own thoughts on a dated-save-folder and I will post the details here, but first:
* re your opening words above - slower or faster than what - some 5 minutes?
* re your earlier words -
* I unload ALL my sfs files before I remaster (else they may as well be pets) - tinkering with your sfs (except to extract the sfsdirect and/or unattendedremaster files for separate use) is assuredly not the answer.
My regards
From the last item on page 2 of this thread, two days ago, you wrote:
Hi Jasper
So sfsdirect is slower or faster for you? You can edit the sfs and remove /boot/fd/efiboot.img that weighs 10mb by itself!!!
Dated savefile? please provide details so I can include it or the possibility to create it in the special functions section of the script.! I am also thinking about including kernel/modules/firmware upgrades from my personal repositories.
I have now finalised my own thoughts on a dated-save-folder and I will post the details here, but first:
* re your opening words above - slower or faster than what - some 5 minutes?
* re your earlier words -
If you want to know what I might want to install tomorrow - ask me, not someone else - but then I have no idea. That's the way I read my emboldening of your words. However, I suspect you could remaster any FULL installation with infallible accuracy. - but to what advantage?These would best be created by each distro's creator/maintainer (or anyone with know how and time to submit such a file to me for inclusion) with provision for likely-to-be-installed apps. Then the script would automatically select the appropriate exclusion file by matching the DISTRO_SPECS variables!
* I unload ALL my sfs files before I remaster (else they may as well be pets) - tinkering with your sfs (except to extract the sfsdirect and/or unattendedremaster files for separate use) is assuredly not the answer.
My regards
-
stemsee
Hi Jasper
Do not be taken by my rhetorical goading .... of course I know how fast it is! Relishing the fact!
" with provision" simply means consider where such apps store personal data and caches for the purpose of cleaning/adding folders/files to exclude list during remaster!
I have added cd-rw dvd-rw utilities, multisession cd/dvd creation! But I can't test it because I broke my usb dvd writer!
Still to do .... integrate kernel upgrades with modules as built-in or sfs.
Do not be taken by my rhetorical goading .... of course I know how fast it is! Relishing the fact!
" with provision" simply means consider where such apps store personal data and caches for the purpose of cleaning/adding folders/files to exclude list during remaster!
I have added cd-rw dvd-rw utilities, multisession cd/dvd creation! But I can't test it because I broke my usb dvd writer!
Still to do .... integrate kernel upgrades with modules as built-in or sfs.
-
Jasper
Hi stemsee,
Your reply had one helpful explanation, but I would appreciate your views on any possible advantage of remastering a full installation and my views on the deficiencies of your sfs.
Also, whilst I have no personal knowledge, I suspect the CD-R is currently significantly cheaper and more widely used than the CD-RW, though either type is easily burned.
Your multi-session-live-CD/DVD method works superbly, though not quite perfectly, for me as I had to make one adjustment (which I'll soon explain in detail).
I no longer have any frugal installations - so my concern is for others - as your sfs has, it seems, needed to be so large and so specific to deal with very few distros and perhaps particular versions thereof.
My regards
PS I favour, and use, the DVD+RW because I thought it faster than the DVD-RW and it requires no formatting prior to first use. On second thoughts, perhaps you had already catered for CD-R and DVD+RW.
Your reply had one helpful explanation, but I would appreciate your views on any possible advantage of remastering a full installation and my views on the deficiencies of your sfs.
Also, whilst I have no personal knowledge, I suspect the CD-R is currently significantly cheaper and more widely used than the CD-RW, though either type is easily burned.
Your multi-session-live-CD/DVD method works superbly, though not quite perfectly, for me as I had to make one adjustment (which I'll soon explain in detail).
I no longer have any frugal installations - so my concern is for others - as your sfs has, it seems, needed to be so large and so specific to deal with very few distros and perhaps particular versions thereof.
My regards
PS I favour, and use, the DVD+RW because I thought it faster than the DVD-RW and it requires no formatting prior to first use. On second thoughts, perhaps you had already catered for CD-R and DVD+RW.
-
stemsee
@Jasper I do not fully understand the reason for your question! The advantage of remastering a full installation is threefold.
1) It creates a compressed backup which can be transferred to some other device.
2) It creates a frugal installable sfs from a full install. (reverse of f2fs install)
3) It satisfies my curiosity.
Of course if you have never experienced the sheer speed of a full puppy install, and the convenience of removing packages permanently with zero whiteouts then you will miss this point of view.
EmSeeV2 was developed entirely as a full install and converted to an sfs, because when you delete things they can't come back and bite you on the bum (so to speak), as with frugal installs and savefiles.
1) It creates a compressed backup which can be transferred to some other device.
2) It creates a frugal installable sfs from a full install. (reverse of f2fs install)
3) It satisfies my curiosity.
Of course if you have never experienced the sheer speed of a full puppy install, and the convenience of removing packages permanently with zero whiteouts then you will miss this point of view.
EmSeeV2 was developed entirely as a full install and converted to an sfs, because when you delete things they can't come back and bite you on the bum (so to speak), as with frugal installs and savefiles.
Gentlemen:
Perhaps another opinion as to why remastering is important. I currently have 5 PC's in my house, though 4 are somewhat older. The one I type on now had the DVD writer burn out and there is no operating system that boots properly from the hard drive (I suppose I can change that). So, I basically have a PC with USB ports to boot from. One of my other computers has the opposite, a working DVD writer but an old Bios that does not allow booting from USB. The third, newer one, is solely for work and I need to keep Windows there for certain compatibility purposes. This one, as it's my work only computer, has everything working properly. The other two are for other family members, so I leave them alone.
So, for me, once I add all the packages that I want and the dependencies to make sure they work properly, the patches such as the recent BASH and Heartbleed fixes and my own bash script packages, I want to keep it and I want to use it on any computer in the house, regardless of the restrictions on each of them (boot from CD on one, USB from the other).
So a Remaster / Backup program that works somewhat quickly (so I'm not keeping the computer on half the night to get it done) with many options is worth its weight in gold. This may not be a program for everyone but for me, Stemsee gets my full applause! Keep on a rolling!
Cheers!
Slavvo67
Perhaps another opinion as to why remastering is important. I currently have 5 PC's in my house, though 4 are somewhat older. The one I type on now had the DVD writer burn out and there is no operating system that boots properly from the hard drive (I suppose I can change that). So, I basically have a PC with USB ports to boot from. One of my other computers has the opposite, a working DVD writer but an old Bios that does not allow booting from USB. The third, newer one, is solely for work and I need to keep Windows there for certain compatibility purposes. This one, as it's my work only computer, has everything working properly. The other two are for other family members, so I leave them alone.
So, for me, once I add all the packages that I want and the dependencies to make sure they work properly, the patches such as the recent BASH and Heartbleed fixes and my own bash script packages, I want to keep it and I want to use it on any computer in the house, regardless of the restrictions on each of them (boot from CD on one, USB from the other).
So a Remaster / Backup program that works somewhat quickly (so I'm not keeping the computer on half the night to get it done) with many options is worth its weight in gold. This may not be a program for everyone but for me, Stemsee gets my full applause! Keep on a rolling!
Cheers!
Slavvo67