Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Mon 28 Jul 2014, 16:32
All times are UTC - 4
 Forum index » House Training » Users ( For the regulars )
Designing Puppy to be a RAM operating system.
Moderators: Flash, Ian, JohnMurga
Post new topic   Reply to topic View previous topic :: View next topic
Page 5 of 6 [89 Posts]   Goto page: Previous 1, 2, 3, 4, 5, 6 Next
Author Message
sunburnt


Joined: 08 Jun 2005
Posts: 5010
Location: Arizona, U.S.A.

PostPosted: Mon 23 Jun 2008, 04:41    Post subject:  

As said in the other forum post... UnionFS & AUFS both take a fair amount of cpu power & lots of ram to work well.
That said, it should be obvious that if there's another way to do it... GOOD!
Back to top
View user's profile Send private message 
HairyWill


Joined: 26 May 2006
Posts: 2949
Location: Southampton, UK

PostPosted: Mon 23 Jun 2008, 05:13    Post subject:  

I just got puppy to boot using multi-session style directories. The only change needed to initrd.gz was to add in a sync in init at about line 945 before
Code:
. $OLDFILESMNTPT/etc/rc.d/BOOTCONFIG
Before I added the sync I was getting a kernel panic.
I haven't tried modifying rc.shutdown yet, but I have written a trivial script that creates a new session directory and saves /initrd/pup_rw into it.
Code:
#!/bin/sh
mkdir -p /mnt/hda2
mount /dev/hda2 /mnt/data
NEWDIR=/mnt/data/$(date '+%Y-%m-%d-%H-%M')
echo $NEWDIR
mkdir -p $NEWDIR
cp -r /initrd/pup_rw/root $NEWDIR
cp -r /initrd/pup_rw/etc $NEWDIR
cp -r /initrd/pup_rw/usr $NEWDIR
cp -r /initrd/pup_rw/bin $NEWDIR
cp -r /initrd/pup_rw/sbin $NEWDIR
cp -r /initrd/pup_rw/opt $NEWDIR
cp -r /initrd/pup_rw/lib $NEWDIR
cp -r /initrd/pup_rw/mnt $NEWDIR
cp -r /initrd/pup_rw/var $NEWDIR
cp -r /initrd/pup_rw/archive $NEWDIR
rm  $NEWDIR/etc/.XLOADED
umount /mnt/data
It will need some logic in rc.shutdown to work out whether the session should be burnt to CD or saved to disc.
Here is what my PUPSTATE looks like
Code:
PUPMODE=77
PDEV1=''
DEV1FS=''
PUPSFS='hda2,ext2,/pup_400.sfs'
PUPSAVE='hda2,ext2,/2008-06-20-01-01'
PMEDIA=''
SATADRIVES=''
#these directories are unionfs layers in /initrd...
SAVE_LAYER='/pup_ro1'
PUP_LAYER='/pup_ro2'
#The partition that has the pup_save file is mounted here...
PUP_HOME=''
#(in /initrd) ...note, /mnt/home is a link to it.
#this file has extra kernel drivers and firmware...
ZDRV='hda2,ext2,/puppy-4.00/zdrv_400.sfs'
PSWAPFILE=''
PSAVEMARK=''
If I had realised it was this easy I would have done this ages ago.
Minor downside. Once you create 2008-06-23-09-01 style directories in a partition you cannot boot any frugal install on that partition because the boot process automatically uses the multisession stuff

_________________
Will
contribute: community website, screenshots, puplets, wiki, rss
Back to top
View user's profile Send private message 
sunburnt


Joined: 08 Jun 2005
Posts: 5010
Location: Arizona, U.S.A.

PostPosted: Mon 23 Jun 2008, 05:36    Post subject:  

Hi HarryWill; How's it goin'?
An image file could be used on the partition to separate the partition from it of course.

Have you looked at the other forum discussion I pointed out?
It uses initramfs for the / file system & links to SFS & image files, & also partitions.
So the real files can be anywhere on an any type of device with any file system.
It splits up the big Puppy SFS file into smaller parts so apps. can be quickly added to them.
If there's space the SFS & image files can be loaded into ram individually with a Boot Manager type GUI.
And as suggested here, the settings are saved by themselves (~ 8 MB),
probably in a image file but could be directly on a partition (no backup & restore).
Have a look & tell me what you think...

http://www.murga-linux.com/puppy/viewtopic.php?p=208709#208709
Back to top
View user's profile Send private message 
HairyWill


Joined: 26 May 2006
Posts: 2949
Location: Southampton, UK

PostPosted: Mon 23 Jun 2008, 13:14    Post subject:  

hi sunburnt, I'm still digesting your post.

I found the cause of the kernel panic. In init there is
Code:
#older pup_save.2fs <v2.16 will not have this file...
[ ! -f $OLDFILESMNTPT/etc/rc.d/BOOTCONFIG ] && mkdir -p $OLDFILESMNTPT/etc/rc.d && touch $OLDFILESMNTPT/etc/rc.d/BOOTCONFIG
#note, OLDFILESMNTPT can also be "" so BOOTCONFIG needs to exist in initrd also.
. $OLDFILESMNTPT/etc/rc.d/BOOTCONFIG #can have EXTRASFSLIST variable.

the touch fails if /etc/rc.d doesn't exist, so I changed it to
Code:
[ ! -f $OLDFILESMNTPT/etc/rc.d/BOOTCONFIG ] && mkdir -p $OLDFILESMNTPT/etc/rc.d && touch $OLDFILESMNTPT/etc/rc.d/BOOTCONFIG
alternatively if you don't want to modify init you can boot multisession style by creating a directory structure similar to this /2000-00-00-00-00/etc/rc.d/ inside the partition you want to boot.

This stuff only works on a linux filesystem, ie not fat or ntfs (no symlinks)

Some rather tentative use cases:
forensic analysis of which files change in which sessions
running entirely in ram
multisession on usb storage allowing the usb storage to be disconnected or used for something else after boot (as with multisession CD)
choice of whether to save session or not

I still need to work on rc.shutdown.

_________________
Will
contribute: community website, screenshots, puplets, wiki, rss
Back to top
View user's profile Send private message 
erikson


Joined: 27 Feb 2008
Posts: 735
Location: Ghent, Belgium

PostPosted: Mon 23 Jun 2008, 13:59    Post subject:  

I'm hacking Wink

I notice that PUPMODE is set in the init script (from initrd.gz) and is used in the rc.shutdown script.

Are these the only scripts in Puppy wherein PUPMODE is used, or are there still other ones?

Re. unionfs and aufs, is there "recommended reading" explaining basic concepts and implementation? I mean in general, not restricted to its usage in Puppy?

_________________
If it ain't broke, don't fix it. --- erikson
hp/compaq nx9030 (1.6GHz/480MB/37.2GB), ADSL, Linksys wireless router
http://www.desonville.net/
Puppy page: http://www.desonville.net/en/joere.puppy.htm

Back to top
View user's profile Send private message Visit poster's website 
Bruce B


Joined: 18 May 2005
Posts: 11080
Location: The Peoples Republic of California

PostPosted: Mon 23 Jun 2008, 14:14    Post subject:  

Timothyli,

timothyli wrote:


{cut}

How can we reduce the size and contents of pup_save to its absolute minimum so that it loads very fast in RAM during startup.?

{snip}



I really appreciate fast startups. But, hopefully we spend much more time computing than we do starting.

Reducing the size of the pup_save and size contents are two considerations. Mind if I address them that way.

contents ) the user determines the majority of the contents in the pup_save file. value considerations of the speed tradeoffs are up to the user

the user can also at his discretion have contents which never get loaded to RAM. this would be things like mounted partitions and etcetera.

pup_save size ) if after building the pup_save file, you discover it has more free space than wanted, adjust its size


--------------------------

From another timothyli post and with more cutting and snipping on my part.

timothyli wrote:
To achieve our goals, It is difficult to avoid a save-back to r/w media, or to a server somewhere on session changes.


True.

Just an note from my experiences, once you have Puppy setup as you want, you'd be surprised how seldom you want to save sessions.

In fact I think one of the strong points of this is the NOT saving sessions, although save back method(s) needs to be developed to make it worthwhile.

-------------------------

All of us are running Puppy sessions. Things are changing during these sessions. Maybe the user at poweroff time can ask himself: Is there anything from this session I want to keep?

The answer will vary, sometimes yes and sometimes no.

In any event, the answers will give the user fair statistic regarding how often he even wants changes made to his Puppy.


Bruce
Back to top
View user's profile Send private message 
Pizzasgood


Joined: 04 May 2005
Posts: 6270
Location: Knoxville, TN, USA

PostPosted: Mon 23 Jun 2008, 19:45    Post subject:  

I knew rsync worked with single files (though i hadn't thought of using it directly on the pup_save.2fs files themselves, good idea!) What I was wondering about was using rsync to update the contents of a .tar file to match the contents of a normal directory, without extracting the .tar. I don't think you can do that, but I haven't had time to dig up any info on it yet.
_________________
Between depriving a man of one hour from his life and depriving him of his life there exists only a difference of degree. --Muad'Dib

Back to top
View user's profile Send private message Visit poster's website 
sunburnt


Joined: 08 Jun 2005
Posts: 5010
Location: Arizona, U.S.A.

PostPosted: Mon 23 Jun 2008, 20:30    Post subject:  

Hi Pizzasgood; Here's tar help:
Quote:
-A, --catenate, --concatenate append tar files to an archive
-c, --create create a new archive
-d, --diff, --compare find differences between archive and file system
--delete delete from the archive (not on mag tapes!)
-r, --append append files to the end of an archive
-t, --list list the contents of an archive
--test-label test the archive volume label and exit
-u, --update only append files newer than copy in archive
-x, --extract, --get extract files from an archive

Of paticular interest is -d & -u
Hope it helps... Terry
Back to top
View user's profile Send private message 
Bruce B


Joined: 18 May 2005
Posts: 11080
Location: The Peoples Republic of California

PostPosted: Tue 24 Jun 2008, 11:36    Post subject:  

FYI - I've not lost interest, quite to the contrary. I've been working very hard or R&D and this is most time consuming. I probably won't make a write up until I have some wrinkles ironed out.

Also, I never learned the intricacies of Frugal because I've been a pretty avid Full Install user.

I really do appreciate others expertise and interest. Not to mention names, because I don't want to leave anyone out.
Back to top
View user's profile Send private message 
HairyWill


Joined: 26 May 2006
Posts: 2949
Location: Southampton, UK

PostPosted: Tue 24 Jun 2008, 11:50    Post subject:  

fixed the save for multisession folders
I replaced multisavefunc in /etc/rc.d/functions4puppy
with
Code:
multisavefunc() {
dialog --yes-label "SAVE" --no-label "NO SAVE" --yesno "Press ENTER to save session...
Or, TAB then ENTER to not save ..." 0 0 >$DEVCONSOLE
if [ $? -eq 0 ];then
   mount /dev/$SAVEPART /mnt/data
   NEWDIR=/mnt/data/$(date '+%Y-%m-%d-%H-%M')
   echo -n "Saving session in $NEWDIR" >$DEVCONSOLE
   mkdir -p $NEWDIR
   cp -vrx /initrd/pup_rw/archive $NEWDIR
   cp -vrx /initrd/pup_rw/bin $NEWDIR
   cp -vrx /initrd/pup_rw/dev $NEWDIR
   cp -vrx /initrd/pup_rw/etc $NEWDIR
   cp -vrx /initrd/pup_rw/lib $NEWDIR
   cp -vrx /initrd/pup_rw/mnt $NEWDIR
   cp -vrx /initrd/pup_rw/opt $NEWDIR
   cp -vrx /initrd/pup_rw/root $NEWDIR
   cp -vrx /initrd/pup_rw/sbin $NEWDIR
   cp -vrx /initrd/pup_rw/usr $NEWDIR
   cp -vrx /initrd/pup_rw/var $NEWDIR
   echo " Finished saving session" >$DEVCONSOLE
   rm  $NEWDIR/etc/.XLOADED
   umount /mnt/data
else
   echo "Session not saved." >$DEVCONSOLE
fi
}
If I was clever I would have detected whether it was multisession CD or hdd and not had to delete the original multisession save. This will ask on shutdown if you want to save the session. If you hit the save button on the desktop you will get a session saved but it will not clear out /initrd/pup_rw so you will end up with duplicates on shutdown, probably not a big problem but wasteful on space.
_________________
Will
contribute: community website, screenshots, puplets, wiki, rss
Back to top
View user's profile Send private message 
sunburnt


Joined: 08 Jun 2005
Posts: 5010
Location: Arizona, U.S.A.

PostPosted: Tue 24 Jun 2008, 16:50    Post subject:  

HarryWill; I'm sure there's an easier way than using the MultiSession stuff, like Pizzasgood's doing.
But perhaps the MultiSession code isn't very complicated & lends itself to this.

The proposed changes to Puppy's file system setup splits up the "static" dirs. & the "dynamic" dirs.
The static ones would be SFS files & the dynamic ones in initramfs, a HD partition, or an image file.
The static ones aren't written to in a normal session, only when installed to.
Dynamic ones are written to every session, this doesn't change the use of rsync but does simplify it..

Last edited by sunburnt on Wed 25 Jun 2008, 01:22; edited 1 time in total
Back to top
View user's profile Send private message 
timothyli

Joined: 22 Jun 2008
Posts: 64
Location: Toronto, Canada

PostPosted: Wed 25 Jun 2008, 00:00    Post subject:  

I finally read the "How Puppy Works" page from BarryK's site and I was very impressed by its smart, sensible, and practical design.

I came to the conclusion that the current design may be "as-good-as-it-can-get" in terms of overall user experience, given a certain amount of RAM.

We may achieve the technical goal of running everything entirely in RAM (which the current design can "almost" do with pfix=ram), but I don't believe this would translate to any significant overall improvement in user experience, given the same amount of RAM. The (perhaps) faster access to certain apps and data would be traded off by slower load and save times.

If one wants to access a certain app in RAM, one could always remaster the system to ensure it is loaded in RAM.

Nevertheless, this is a very interesting problem to solve.

How about copying the pup_save to a RAM disk and then mounting it using unionfs as a layer?
Back to top
View user's profile Send private message 
sunburnt


Joined: 08 Jun 2005
Posts: 5010
Location: Arizona, U.S.A.

PostPosted: Wed 25 Jun 2008, 01:20    Post subject:  

timothyli; That's what Pizzasgood did above...

However... The proposed changes to Puppy would eliminate UnionFS & AUFS, this would improve a number of things.
Faster operation (less cpu intensive), more reliable operation, & way less ram usage.
The "Q & A" here is,,, If you can do the same thing without unions, then you've eliminated an unneeded complexity.
Back to top
View user's profile Send private message 
HairyWill


Joined: 26 May 2006
Posts: 2949
Location: Southampton, UK

PostPosted: Wed 25 Jun 2008, 03:24    Post subject:  

sunburnt wrote:
HairyWill; I'm sure there's an easier way than using the MultiSession stuff, like Pizzasgood's doing.
But perhaps the MultiSession code isn't very complicated & lends itself to this.
It is extremely simple.
It is possible to get this scenario working without modifying initrd.gz. I used to use multisession for several months when I first got interested in puppy. My current laptop doesn't have an optical drive, so I am partly motivated by being able to use puppy this way without needing a CD drive. It also provides a rough and ready file versioning system because files are never deleted from previous sessions.

Multisession is one of the capabilities that really makes puppy stand out.
My impression is that the multisession userbase is quite small. Maybe this development will increase the userbase by at least one and provide an easier mechanism for testing. Considering how easy it was to implement it wouldn't surprise me if Barry is reading this thread and chuckling because he already does testing this way.

The single layered filesystem seems much more revolutionary, I look forward to seeing further developments.

_________________
Will
contribute: community website, screenshots, puplets, wiki, rss
Back to top
View user's profile Send private message 
Aitch


Joined: 04 Apr 2007
Posts: 6825
Location: Chatham, Kent, UK

PostPosted: Wed 25 Jun 2008, 06:59    Post subject:  

Hi Guys

Can I resurrect an idea I threw up yonks ago, when fastboot was being discussed, whilst all this design theory is etheric?

My suggestion was more to do with the actual hardware probe phase, which every OS goes through each & every time I hit the on switch

If it's the same PC I turned off last night, why can't the probe begin with probing me
"Hello Aitch, did you change any hardware?"
If I answer no, it reloads an image without checks > 2seconds? <

[up to] 5 seconds later I'm using it ~~~

Similarly with shutdown
"Did you make any changes?"
no? bang - off

or am I dreaming?

I believe a hard drive is the only thing requiring a delay/write on shutdown

Chips is chips, yes/no?

Sorry I don't have the knowledge that's going on here, but I sure like the sunburnt/PG/BruceB/HairyWill/erikson/timothyli discussion Very Happy

Aitch
Back to top
View user's profile Send private message 
Display posts from previous:   Sort by:   
Page 5 of 6 [89 Posts]   Goto page: Previous 1, 2, 3, 4, 5, 6 Next
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » House Training » Users ( For the regulars )
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.1062s ][ Queries: 12 (0.0055s) ][ GZIP on ]