Banksy 3 - aka "The Personator"

For talk and support relating specifically to Puppy derivatives
Message
Author
User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

#31 Post by greengeek »

So do you still wish to continue loading from CD? If so that makes loading much slower than from HDD. With banksy being built from read only sfs I generally take the risk of loading it from USB or HDD frugal to get better load speed.

Did you notice a longer load time when trialling the WX version? It is a fatter version and each extra MB takes a finite time to load. One of my concerns with the idea of building a personal sfs from all of the current system changes is the resulting increase in personal sfs size and increase in load time. Up to the user to evaluate the cost/benefit ratio i guess.

Sylvander
Posts: 4416
Joined: Mon 15 Dec 2008, 11:06
Location: West Lothian, Scotland, UK

#32 Post by Sylvander »

@ greengeek

1. "So do you still wish to continue loading from CD?"
Yes.

2. "If so that makes loading much slower than from HDD."
I'll accept that so as to get the increased security.

3. "Did you notice a longer load time when trialling the WX version?"
No.

4. "Up to the user to evaluate the cost/benefit ratio i guess."
Seems OK to me.

User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

#33 Post by greengeek »

@Sylvander - how do you normally handle /mnt/home? (banksy doesn't actually use /mnt/home at all).

I'm just thinking that someone who is used to their personal files being stored in /mnt/home may be confused by that location being discarded in any future expansion of the banksy personalisation routine. I wouldn't want someone to lose a heap of important data in the belief that a "full" personalisation script was grabbing anything and everything. The responsibility for saving personal data is left 100% in the hands of the user when using banksy and this would not change with expansion of the initial personalisation (except for whatever data the user decided to store in the /opt/b3user directory which is the personal cache retained across the personalisation process).

User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

#34 Post by greengeek »

Note to self - investigate "ram2sfs" from Ted dog

Sylvander
Posts: 4416
Joined: Mon 15 Dec 2008, 11:06
Location: West Lothian, Scotland, UK

#35 Post by Sylvander »

greengeek wrote:@Sylvander - how do you normally handle /mnt/home?
My Banksy3 doesn't have any /mnt/home, which is normally on partition sda5 because most of my other Puppies have their pupsave files on sda5.
Hence, if I want to access anything on sda5 within Banksy3, I would use the pmounter command and choose to mount sda5, and then would be able to access content at that location.

Similar action needed if I wanted to access ANYTHING anywhere [e.g. "Acerose Password Vault" (APV) located on my Flash Drive = sdb1].
I find such things to be EASY to do.

User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

#36 Post by greengeek »

Sylvander wrote: What if...
It were possible to have Banksy operate initially just like any other Puppy in regard to cutomizing, so that it would be real easy make ANY kind of personalizations
Hi Sylvander, it will take me some time to achieve all the things you mentioned (particularly the "un-Banksying" idea) but I have gone one step closer with the addition of some different personalisation scripts which offer the option of capturing all personalisations.

You have probably noticed from some other threads that capturing every system change can be tricky so there may be some issues i have not detected, but as far as I can tell this new script of mine works really well to allow the new version to capture all system changes, encapsulate them in a readonly sfs, create a new iso and then automatically burn it direct to optical media.

I booted from the generic version then added a bunch of pets and using the new script made myself an iso that was over 450MB and everything worked perfectly.

For this version I decided to revert to the original banksy3 - it does lack some of the extras that are in b3max but the smaller size means the user can add exacty what they want (and it was easier/quicker to handle the files during the upgrade process).

The "personator" functionality is just the same as it was before, but I have added the new "grab all" script via a desktop icon which I have called the "impersonator":
- the personator script is for grabbing a small critical subset of personalisations.
- the impersonator script is for grabbing all personalisations so that the new iso "impersonates" exactly what you were already using.

I call the new banksy "The Impersonator"
("b3imp" for short)

Download links here:

EDIT : original link deleted. Replaced by better version here:
http://www.murga-linux.com/puppy/viewto ... 307#899307

This version does not have Wine and xfe installed but you should be able to add those exactly the same way you previously did. I am keen to know if it retains all the necessary stuff after you install all the goodies you need.
I will keep upgrading any issues I find but if you are able to test and find things that need fixing please let me know. Seems good so far.

Caveats:

- The "impersonator" script has been designed to do a "one time" grab of the personalisations - so it is necessary to boot the generic iso then add all desired changes and customisations in one session, then run the impersonator script. (It has not been tested for multiple "progressive" changes such as a savefile is intended for)

- The amount of RAM required to successfully complete the impersonator process is dependent on the amount of data that you add during the personalisation process. I think 1GB RAM would be the minimum, and my testing was on a 2GB machine.
.
Last edited by greengeek on Sun 17 Apr 2016, 18:00, edited 2 times in total.

Sylvander
Posts: 4416
Joined: Mon 15 Dec 2008, 11:06
Location: West Lothian, Scotland, UK

#37 Post by Sylvander »

WOW!
You are making wishes come true. :D

This I've got to try.
I'll be back ASAP having done the necessary.

-------------------------------------------------------------------------------

Problem at 1st attempt:
1. Downloaded the new b3impgeneric_RC1.iso & b3impgeneric_RC1.iso-md5.txt, with no problem.

2. md5sum checked as OK.

3. Burned the new number-1 CD-RW OK.

4. Booted the new CD-RW OK.

5. Personalized the new b3impgeneric_RC1 OK.

6. Switched in the blank/new number-2 CD-RW disk.
Clicked on the impersonator link OK.
Program ran OK, but at some point during the proces I noticed down at bottom right that the "personal-storage-free-space" was down at the lowest setting and flashing red.
My desktop has 8GB of RAM, so that surprises me.
I guess only 4GB is being used, right?

7. The making of the personalized ISO completed; burn to disk completed OK.

8. Tried to save the new ISO file to my sda6 internal partition, but pmounter could not run, reporting that there wasn't enough memory to run.

9. Rebooted to the new number-2 CD-RW, but the CD wouldn't boot.
Instead: WinXP began to boot.
This only happens if there is no bootable optical disk in the drive.

10. Booted into Slacko to check out the problem CD-RW.
The disk had been blanked using Pburn prior to burning using the "impersonator".
Pburn showed it as "Closed".
I used Pburn to blank the number-2 disk yet again, and it showed as green/empty yet again.

11. I'll make a 2nd attempt ASAP.
Any suggestions as to what to look out for?

User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

#38 Post by greengeek »

Sounds like I may have got something set up incorrectly. Unfortunately I won't be able to test further today (stuck on android).

Any chance you could try the process again but making only a single small change (such as adding a text file into /root)?

I do have a large swap partition - I wonder if that makers a difference. I will do more testing as soon as I am able.

Sylvander
Posts: 4416
Joined: Mon 15 Dec 2008, 11:06
Location: West Lothian, Scotland, UK

#39 Post by Sylvander »

greengeek wrote:...try the process again but making only a single small change (such as adding a text file into /root)?
Done, and it worked OK.

List of small personalizations:
"quicksetup" settings.
Desktop wallpaper changed.
Made a small text file in /root.
Free space dropped from 4 to 3 rings.
Iso was made OK.
Disk burned OK, it booted OK.
Typing from it now.
Free space showing 4 rings once again.

Will now make 2nd attempt at the full personalizations.

User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

#40 Post by greengeek »

Try plugging in a USB stick formatted as "Linux swap". I just tried adding about 400MB of personalisations and the processing exceeded my 2GB RAM and flowed into swap. I think Slackos nonPAE kernel will only use 3GB of your RAM so swap becomes critical if you add too much data.

Sylvander
Posts: 4416
Joined: Mon 15 Dec 2008, 11:06
Location: West Lothian, Scotland, UK

#41 Post by Sylvander »

1. 2nd attempt at full customization was successful. :D
DIFFERENCE:
I chose NOT install the WINE extras.
Upon making the ISO file...
"personal storage frre space" reduced from 4 green rings to 2 amber rings.
I guess the process succeeded because it didn't go into the red zone [no free space remaining].

2. Need to check if I have an empty Flash Drive I can use for a swap partition.

User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

#42 Post by greengeek »

Sylvander wrote: Done, and it worked OK.
List of small personalizations:
"quicksetup" settings.
Desktop wallpaper changed.
Made a small text file in /root.
Free space dropped from 4 to 3 rings.
This surprises me. I would not have expected such a minor customisation to have dropped your personal space to 3 rings when you have so much RAM. Could you hover over the "partview" icon before and during the process to see what it tells you about total personal storage available? Mine tells me I have 6.1GB of personal storage but now that I look at it I don't understand that figure as I have 2GB RAM and 10GB of swap. Not sure how that becomes 6.1GB storage.
What does yours say?

I only see the effect of free storage rings turning from green to orange when I load an huge amount of extra personalisation data and it loads up the memory during the automatic distillor/compressor/isolator process. I can probably tidy those scripts up a bit so that some tmp directories are deleted after use. But I still don't think you should see the stroage space depletion when only adding minor customisations. You should have at least 3GB of RAM available.
2nd attempt at full customization was successful.
OK, thats good news. First base anyway.
I chose NOT install the WINE extras.
Are you referring to the zenity and mpg123 dependencies or do you mean the mono and gecko extras that self-download when you start Wine?

I guess its possible that the mono and gecko leave some large downloaded file somewhere in the file system. Maybe that has an impact.
Upon making the ISO file...
"personal storage frre space" reduced from 4 green rings to 2 amber rings. I guess the process succeeded because it didn't go into the red zone [no free space remaining].
Yes, that would be my interpretation.
Need to check if I have an empty Flash Drive I can use for a swap partition.
I would be very interested to know if having a usb attached swap space changes the figures you see when you hover over the "partview" icon. Does the system see this as an addition to your personal space? Or is there something else I need to look at I wonder. More testing to follow...

Sylvander
Posts: 4416
Joined: Mon 15 Dec 2008, 11:06
Location: West Lothian, Scotland, UK

#43 Post by Sylvander »

1. " I would not have expected such a minor customisation to have dropped your personal space to 3 rings when you have so much RAM."
Same here.

2. "Could you hover over the "partview" icon before and during the process to see what it tells you about total personal storage available?"
Before doing any customizations:
4 green rings: 9.0G personal space, 9.0G free.
I have 8GB RAM [only 3.0 GB being used?] PLUS I added an 8GB swap partition in the free space on a 32GB Flash Drive.
So why isn't there 8+3 = 11GB?

3. "...do you mean the mono and gecko extras that self-download when you start Wine?"
Yes, these ask to be installed, and during the 2nd attempt I chose to NOT install those.

4. "I guess its possible that the mono and gecko leave some large downloaded file somewhere in the file system."
I'm guessing that when these are added to the ISO, they use up the last of the remaining "personal storage", so it goes into the red.

5. "I would be very interested to know if having a usb attached swap space changes the figures you see when you hover over the "partview" icon. Does the system see this as an addition to your personal space?"
Yes, it does.
I'll post this, and then close the web-browser, and go through the full customizations [including mono & gecko], taking notes of "free storage" as I go, then report back on those values [added below].

-------------------------------------------------------------------------------------------------------
I'm back to report on attempt-3:
a. 4-green rings: 9.0G personal storage, 9.0G free.

b. After installing Xfe = 9.0G free.

c. After installing WINE = 8.9G free.

d. After config of Xfe = 8.9G free.

e. Running winefile command, but:
Pre mono install = 8.8G free.
After installing mono = 8.6G free.
Pre gecko install = 8.6G free.
After installing gecko = 8.5G free.

f. During the impersonator process, free storage dropping bit-by-bit to 7.1G free [during making of the ISO].
No further decrease of free storage during remainder of the impersonator process.

g. Rebooted using the new CD-RW [attempt-3 CD], with Flash Drive removed.
Booted just fine.
At desktop...personal storage = 1.5G, 1.5G free.
Seems to be half what it aught to be.

--------------------------------------------------------------------------------------------------------

h. All is looking good.
All personalizations appear to be present and functioning.

User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

#44 Post by greengeek »

Hi Sylvander, thanks for the thorough testing and feedback. My interpretation of the results is that mono and gecko may have consumed a reasonable amount of space for the download, then additional space for the extracted files grafted into puppy itself. Probably my impersonator script should have ditched the original downloaded files but I have no idea where they may be.

I will try to investigate where Wine stores this stuff.

However, I sense that there is an underlying question with regard to personal space calculation and handling. I realise now that all my banksy development has been done while I have had a 10GB swap partition on internal hard disk. I have just tried to turn off that swap but it made no difference to my 6GB personal storage shown by part view. Possibly it needs to be unavailable at boot time to properly update the space calculation. More testing required - I will try a different machine and use the USB swap method.

I am glad you at least have a working system. It would be nice if I can figure out why 1.5GB of your storage is AWOL. Surely that must take forever to load? And a cdrw only holds 650MB doesn't it, rather than 1.5GB? I doubt Wine has stolen the lot. Might be some artifact of the way I built banksy.

I will report back with any meaningful results.

Sylvander
Posts: 4416
Joined: Mon 15 Dec 2008, 11:06
Location: West Lothian, Scotland, UK

#45 Post by Sylvander »

1. " I have no idea where they may be."
My guess would be in the /tmp folder.
Perhaps Pfind would find them.

2. "It would be nice if I can figure out why 1.5GB of your storage is AWOL. Surely that must take forever to load?"
It is somewhat slow, but I figured that was because it must load the files from a slow optical disk/drive.

3. "a cdrw only holds 650MB doesn't it"
I believe my CD-RW's hold 700MB.

4. "I doubt Wine has stolen the lot."
Puppy is telling me that the latest Banksy3 CD-RW files are occupying 366MB.
That's bigger than the original disks' reported 209MB occupied, but nowhere near filling the 700MB.
366 - 209 = 157 MB.
366MB is much smaller than the 8.5GB - 7.1 GB = 1.4 GB of free storage that disappears during the making of the ISO.

User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

#46 Post by greengeek »

Sylvander wrote: It is somewhat slow, but I figured that was because it must load the files from a slow optical disk/drive.
<Yes, optical media loading is slow. If it was loading over a gig of data you would really notice it even more. I was thinking that a cdrw may be more like a DVD in terms of larger data capacity but you have confirmed that the actual content on the cdrw is well less than normal CD capacity anyway.

This suggests that the missing 1.5GB reported by partview is either erroneous or caused by something like the file expansion process after loading CD data, or maybe caused by some artifact of the unusual way banksy is layered. Maybe both. Either way it looks like having a swap partition is necessary during a personalisation process that involves significant addition of data/programs. Less critical once running from the completed personal CD - but I still need to look closer at personal storage calculation and usage.
.
366MB is much smaller than the 8.5GB - 7.1 GB = 1.4 GB of free storage that disappears during the making of the ISO.
There are good reasons why a lot of storage is consumed during that process - the /tmp/banksydistillor directory holds all of the added data ready to build into the new personal sfs, then it is formed into the new sfs which is also in /tmp, then that sfs is copied into an "isobuild" directory along with other necessary files (all of this also in /tmp) then that isobuild directory is compiled into the new iso (yet another large file sitting in /tmp). So you can see a heap of data accumulates in /tmp during the process.

I think I can smarten the process somewhat to make it more thrifty on RAM. More testing to follow. Till then I suggest the USB swap partition trick may be the key.

Also - installing the banksy files to HDD definitely speeds booting (although I know you are not comfortable with that structure...)

Sylvander
Posts: 4416
Joined: Mon 15 Dec 2008, 11:06
Location: West Lothian, Scotland, UK

#47 Post by Sylvander »

greengeek wrote:installing the banksy files to HDD definitely speeds booting (although I know you are not comfortable with that structure...)
I like:
a. Banksy running totally in RAM with no use of any writable drive.
b. The fact that the Banksy CD-RW is read-only.

I don't like:
c. Any writable storage media being used during normal use of Banksy.
e.g. Swap partition. [Might a swap partition be infected, and so infect a subsequent Banksy session?]
Might a full install of Banksy [to an internal HDD partition] be infected?

User avatar
greengeek
Posts: 5789
Joined: Tue 20 Jul 2010, 09:34
Location: Republic of Novo Zelande

#48 Post by greengeek »

Sylvander wrote:I like:
a. Banksy running totally in RAM with no use of any writable drive.
b. The fact that the Banksy CD-RW is read-only.
I agree.
I don't like:
c. Any writable storage media being used during normal use of Banksy.
e.g. Swap partition.
Also agree
[Might a swap partition be infected, and so infect a subsequent Banksy session?]
I have read that this is impossible because any code in the swap area becomes useless after shutting down a session. However, swap is writable so I'd have to say that my guess is that there is more risk with it than without it. (Although we have to bear in mind that RAM itself is writable so in that case maybe there is not much difference - unless there is some attack vector that can "collect" tainted data from the previous swap session. If that is possible then I would venture to suggest such complex hacker skills would most likely reside in the hands of nation states such as was revealed by the stuxnet virus - in which case I have no show of protecting my system against that level of penetration)

Having said that I would be happy to take the risk of using the usb swap just for the time it takes to get the personalisation/burn done if the amount of data being added required the extra space during the processing. (I hope to reduce the required space in the next impersonator script).
Might a full install of Banksy [to an internal HDD partition] be infected?
I consider it highly unlikely because the two sfs in use are both readonly (in contrast with normal puppies where the personal storage file or folder is writable). In order to corrupt/infect the banksy code it would be necessary to unsquash, modify, resquash, then overwrite one of the sfs files. I can't imagine that being very likely except by local hands-on operation.

I would rate the use of Wine as a greater risk than use of swap or HDD install - especially if permitting the mono and gecko downloads. I don't have any grounds for saying that - it's just a gut feeling. Do the mono and gecko downloads ask you any questions or explain what they are for? Not as far as I could tell. That in itself would concern me. I prefer to carry the most minimalistic payload possible to get the job done - less code, less exposure. That's my mantra anyway.

Your testing has brought to my attention that my system always has 10GB of swap partition available - I never thought about that being a security risk till now. I may not sleep well tonight :-)
.

Sylvander
Posts: 4416
Joined: Mon 15 Dec 2008, 11:06
Location: West Lothian, Scotland, UK

#49 Post by Sylvander »

1. "I have read that this is impossible because any code in the swap area becomes useless after shutting down a session."
I'd be happy if I knew that any swap partition in use was zero-filled as part of Banksy shut down, but I guess that's impractical.
My lack of knowledge leads me to imagine stuff remaining in the swap partition [left there by (Banksy or) some other Puppy] being brought to life during the latest Banksy session.

2. "we have to bear in mind that RAM itself is writable so in that case maybe there is not much difference"
But surely there IS a difference.
I believe the contents of SWAP is persistent, whereas the content of RAM is completely lost after a relatively short time [not persistent].

3. "I would be happy to take the risk of using the usb swap just for the time it takes to get the personalisation/burn done"
same here.

4. "I hope to reduce the required space in the next impersonator script"
I like it.

5. " In order to corrupt/infect the banksy code it would be necessary to unsquash, modify, resquash, then overwrite one of the sfs files. I can't imagine that being very likely except by local hands-on operation."
Nice to hear.

6. "I would rate the use of Wine as a greater risk than use of swap or HDD install"
Yes, I'm not happy that I use WINE, but I hate the prospect of attempting to [transfer the vault contents, and] switch to using a Puppy alternative that is also portable [and works really well] like "Acerose Password Vault" [APV].

7. "Do the mono and gecko downloads ask you any questions or explain what they are for? Not as far as I could tell."
Same here.
MONO
GECKO

8. "Your testing has brought to my attention that my system always has 10GB of swap partition available - I never thought about that being a security risk till now."
I hope I'm not raising false fears; it's just something that troubles me.
Hence, I'll avoid using a swap partitition until I come to know it poses no risk.

version2013
Posts: 503
Joined: Mon 09 Sep 2013, 00:00
Location: Florida, USA
Contact:

#50 Post by version2013 »

Sylvander wrote: 7. "Do the mono and gecko downloads ask you any questions or explain what they are for? Not as far as I could tell."
Same here.
MONO
GECKO
More links:
Mono
Gecko

Post Reply