SFS file maker that tracks changes to the SFS files.

Under development: PCMCIA, wireless, etc.
Post Reply
Message
Author
User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

SFS file maker that tracks changes to the SFS files.

#1 Post by sunburnt »

I'm thinking that instead of installing app. packages to the SAVE file, the app.
installs should be done to the SFS file instead, much like remastering does.

This would make the SAVE file more of a settings file & it'd only have to be
8MB to 16MB in size to hold all the settings, & config. & boot scripts.
This would allow the SAVE/settings file to be easily loaded into ram.

App. packages (DotPup, PupGet, .tar.gz, etc.) would be installed to the SFS file,
thus making tracking of the installed files critical to being able to uninstall them.
So for each install & uninstall a new SFS file would be made.

In my app.: sfsInstaller for Full-HD Puppy installs only, I did this with a list.
It also backs up duplicate files before the install & restores them upon uninstall.

So to install an app. a new SFS file's made, this means ONLY 1 SFS file,
so then unionfs has only 2 branches to handle (Settings file & SFS file).
This means that the file system will work faster, especially on older PCs.
This also precludes the need to swap SFS files as we'd hoped could be done.
And lots of different SFS files with different apps. in them isn't needed,
any app. install package compatible with Puppy could be used.

This sounds like an improvement to the current "standard install process".
To merge my apps.: mksfs & sfsinstaller into a new one wouldn't be hard.
If I get some support I'll gladly do the needed work to produce the app.

If there's interest in this idea... Speak now... (woof woof) Terry

User avatar
WhoDo
Posts: 4428
Joined: Wed 12 Jul 2006, 01:58
Location: Lake Macquarie NSW Australia

Re: SFS file maker that tracks changes to the SFS files.

#2 Post by WhoDo »

sunburnt wrote:So to install an app. a new SFS file's made, this means ONLY 1 SFS file, so then unionfs has only 2 branches to handle (Settings file & SFS file). This means that the file system will work faster, especially on older PCs. This also precludes the need to swap SFS files as we'd hoped could be done. And lots of different SFS files with different apps. in them isn't needed, any app. install package compatible with Puppy could be used.

This sounds like an improvement to the current "standard install process". To merge my apps.: mksfs & sfsinstaller into a new one wouldn't be hard. If I get some support I'll gladly do the needed work to produce the app.
I'm not sure if I understand what you are suggesting, so bear with me while I reiterate in my own words to see if I have.

1. You are advocating only 3 x sfs files for Puppy - pup_2xx.sfs, zdrv_2xx.sfs and apps_2xx.sfs?

2. pup_2xx.sfs would contain the core operating environment with base applications set?

3. zdrv_2xx.sfs could contain all drivers, including CUPS?

4. apps_2xx.sfs would contain all additional applications to user taste - .pup, .pet, .tar.gz, etc.

5. pup_save.2fs would then contain settings for the base operating environment only - xorg.conf, alsa, etc.

If I have that correct, I think it could work well and still leave room for one or two extra sfs files for HUGE applications like Open Office. 8)

I understand Barry is looking to build some sfs, like zdrv_2xx.sfs, inside the initrd.gz file, which would be humungous (by his own description), but not impact on what loads into RAM and leave more room for sfs unions too.

My only concern would be in backing up such a huge apps_2xx.sfs file as some people will end up with. If you put all of your eggs in one basket, you better have a strong basket! :P
[i]Actions speak louder than words ... and they usually work when words don't![/i]
SIP:whodo@proxy01.sipphone.com; whodo@realsip.com

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#3 Post by sunburnt »

No... Only 1 SFS file & 1 SAVE-Settings file, so only 2 union branches.

ALL other SFS files, installs, & driver modules would become part of the SFS file.

The SFS file maker utility would add/remove apps. & etc. to/from the SFS file,
same as apps. are added/removed from the SAVE file or a Full-HD install now.

And yes... The SFS file would become very large if lots of installs were made.
But then it'd be exactly what the person wanted... Right?
Basically it's just a new way to remaster Puppy with install packages.

The SFS file would hold ALL the apps. & etc., & so backing up is a good idea.
Also backing up any media or document files you have is always a good idea.
The kernel & initrd.gz files would remain the same, no changes needed.

The original SFS file that came with the Puppy release would have to have
an uninstall file for all the apps. in it, IF they're to be able to be uninstalled.
That's why if the original release was small in size (few apps.) it'd be alot
easier to add to it, than to have to remove anything from it.

The zdrv SFS file is all driver modules I beleave, too much bloat to add it.
A auto. module-library downloader would make FAR more sense.
Same goes for codecs & filters, lots of space for something maybe used.
For this setup, if a module's needed from the zdrv SFS file, it can be mounted
but NOT unioned, & the needed module added to the main SFS file.

Extra SFS files can still be unioned at boot same as always, this setup reduces
the number of SFS files needed, & over comes having to swap them.
Several SFS files could be made & Puppy prompts for which to use at boot.

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#4 Post by sunburnt »

Addendum: A further thought about simplifing the file system.

I just realized what I did with LanPuppy...
I copied the Settings file into / (the ramdisk), so it doesn't need to be unioned.
The Settings file'd be mounted at boot & copied to ram, & then unmounted.
Small file changes would be done in ram & have to be written to the Settings file
at shutdown, but at least it'd be very fast because of the file's small size.

This makes for ONLY 1 file to union, the main SFS file to be unioned on /
The fewer unions that there are, the faster the file system access is.
As with LanPuppy, this setup would allow up to 4 extra SFS files to be added.


Hummm... I wonder if AUFS can union mount a branch "under the mount point"?
Then the ramdisk "/" would be on top of the SFS file & any changes made
to the file system in the ramdisk would overshadow the SFS file.

User avatar
Pizzasgood
Posts: 6183
Joined: Wed 04 May 2005, 20:28
Location: Knoxville, TN, USA

#5 Post by Pizzasgood »

This would be a cool idea, except that creating SFS files is horribly slow, even on my 3GHz machine (though far faster than it was on the 450MHz one). As far as I know, you can't simply edit them, you'd have to re-build it each time.

There is an option to append I think.

Another problem is that it somewhat negates the invulnerability of Puppy. Currently, the pup_xxx.sfs file is read-only, thus the actual data can only be masked, not destroyed. So, to "re-install" you just wipe out the pup_save.2fs file and make a new one. But if the pup_xxx.sfs file were writable, you'd have to replace it too.

It is interesting though. If the user is willing to accept a longer shutdown time, it would be cool. As for the new issue of corrupted pup_xxx.sfs, in my case it's just a little hassle rather than a big problem, because I archive every Puppy iso I download. So I'd have to mount an iso, or just keep an extra copy of the .sfs and replace the corrupted one. No big issue, they're "small".

But I do too much rebooting, so I'd find it horribly annoying. How it is now, I can simulate it manually when I need it (either with Edit-SFS or by rolling my own "alpha" Pizzapup) and the rest of the time just use a save-file.


I'm more interested in the idea of shunting the pup_save.2fs file into ram, then running similarly to pupmode 13. Though the slight slowdown every 30 minutes as things are synced can be annoying. I'm sure there's plenty of room for improvement in the whole thing though, especially if you keep track of which files are already synced, and decrease the interval to 5 minutes (except when using flash drives).

Exploring that is on my long-term to-do list, if I ever get really bored with whatever project I'm working on.
[size=75]Between depriving a man of one hour from his life and depriving him of his life there exists only a difference of degree. --Muad'Dib[/size]
[img]http://www.browserloadofcoolness.com/sig.png[/img]

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#6 Post by sunburnt »

Pizzasgood; I'm not sure what booting has to do with this setup, except for
choosing which SFS file to use.
A new SFS file's only made when a package is installed to it.

The save at shutdown would go very quickly as the Settings are only 4MB.

Yes there is append to an SFS file, but I've never got it to work.
But if we got it working, then installs would go fairly quickly.

Yes making SFS files IS slow, as folks start out they'd make a few until
things settled into place for them, & then they'd make very few after that.
It allows folks to remaster in small chunks, an app. at a time.
As the kernel & initrd.gz file are the same, folks could exchange SFS files!

The security issue is slight as only the SFS file GUI would make SFS files.
The GUI's uninstall would remove anything that was put into the SFS file.

Both this setup & being able to swap SFS files each have advantages.
Swapping needs AUFS & PCs with +512MB of ram... Thanks to NathanF.

User avatar
mysticmarks
Posts: 159
Joined: Tue 27 Feb 2007, 01:56
Location: California
Contact:

Down side

#7 Post by mysticmarks »

Although i agree that it would be a great way to boost a run after it is settled down. I do agree with pizzasgood's point. build is slow, and from a cd point of view, you would have to burn a new disc within a very short time.
Moreover, having one sfs file would mean that updating puppy would no longer be as simple as copying in the new vmlinuz, pup_xxx.sfs, etc. it would mean that to upgrade you would have to start the whole cylle over and rebuild it again.

I've done lots of squashfs reading since i discovered puppy, and i love the application of them.

Now taking pizzasgoods app, and adding your build from scratch stuff to it would be a great permanent addition to puppy.;)(but you've both heard that before. :D )

I made a suggestion in that forum section, semi related to this. Make sure and pick your brain with that one too.

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#8 Post by sunburnt »

mysticmarks; True about using the same Save file in a new Puppy version.

But why can't you use the same SFS file also? It's all the same files anyway.
The ONLY distinction is whether the files are in the Save file or in the SFS file.

Plus all that's needed to muss things up is a minor change or upgrade & the
old stuff isn't going to work anyway... Such is the upgrade blues.
A change in the way a config. file is used, or an upgrade to ROX & it's GONE!

So... Just build a new SFS file with the app. packages you want, 1 shot...
Just like remastering is now, except this is a bit more convenient to do.

User avatar
mysticmarks
Posts: 159
Joined: Tue 27 Feb 2007, 01:56
Location: California
Contact:

#9 Post by mysticmarks »

im all about trying new ways of doing things. ;)

User avatar
Pizzasgood
Posts: 6183
Joined: Wed 04 May 2005, 20:28
Location: Knoxville, TN, USA

#10 Post by Pizzasgood »

Ah, I see. I thought you meant to update the sfs file on each shutdown. Yes, it wouldn't be quite so bad if the delay is thrown in with the installing-delay.
[size=75]Between depriving a man of one hour from his life and depriving him of his life there exists only a difference of degree. --Muad'Dib[/size]
[img]http://www.browserloadofcoolness.com/sig.png[/img]

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#11 Post by sunburnt »

That idea was the addendum which offers the idea of mounting & copying the
Save/Settings file to the ramdisk at bootup then unmounting it, & then at
shutdown mount the Save/Settings file again & save the settings back to it.

This removes the Save/Settings file from the union, leaving just the SFS file.
This speeds up file access, or allows up to 4 extra SFS files to be unioned.

The idea relates to the SFS package installer, but isn't really a part of it.


A thought... Why have the unleashed packages & then have to make more
packages from them (DotPup, PupGet, etc.), why not just use unleashed?
So this package installer for SFS files would use unleashed & alien packages.

This goes along with the idea of simplifying & reducing the effort needed to
produce & maintain Puppy & especially the packages that make it up.

Post Reply