Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info

READ-ONLY-MODE: PLEASE DO NOT POST NEW STUFF!
  New Forum: http://forum.puppylinux.com
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups    
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Fri 14 Aug 2020, 11:27
All times are UTC - 4
 Forum index » Advanced Topics » Cutting edge
PPM_links-15.4 - hacked PPM files supporting petx
Moderators: Flash, Ian, JohnMurga
This forum is locked: you cannot post, reply to, or edit topics.   This topic is locked: you cannot edit posts or make replies. View previous topic :: View next topic
Page 1 of 1 [12 Posts]  
Author Message
gyro

Joined: 28 Oct 2008
Posts: 1799
Location: Brisbane, Australia

PostPosted: Thu 01 Aug 2019, 08:09    Post subject:  PPM_links-15.4 - hacked PPM files supporting petx
Subject description: petx - 'pet' extended format
 

The current version is at http://www.murga-linux.com/puppy/viewtopic.php?p=1038775#1038775

The attached file 'ppm_links-15.tar' contains files that patch the Puppy Package Manager so that it behaves as outlined below.
It contains 5 files:

'./usr/local/petget/installpkg.sh'

1. Respects all symbolic links to directories in the local file system.
So on my xenialpup '/usr/lib/i386-linux-gnu/libassuan.so.0.7.2' is written as '/usr/lib/libassuan.so.0.7.2',
'/tmp/x.txt' is written as '/initrd/mnt/zram0/tmp/x.txt'.

2. Ignores 'PUPMODE' and writes all files to the '/' file system. Same for "frugal install" and "full install".
So doesn't muck around with whiteout files or stack layers.

3. Extracts all source formats to a directory in /tmp.
Produces a list of the contents of this directory as the '.files' file.
Processes each entry in the '.files' file.

'./usr/local/petget/removepreview.sh'

1. Processes each entry in the '.files' file, in reverse order, so that files get removed before the containing directory.
If a processed directory is empty in the local file system, it is removed.

2. Removes files from the '/' file system, respecting any symbolic links to directories in the local file system.

3. After removing a file/directory it checks if this actiion produced a whiteout file in the RW directory, directly.
If it's an "aufs" stack, any whiteout file is removed from the RW directory, directly.
If it's an "overlayfs" stack, the full real path to the whiteout file is written to an "exclude file" so that the file will be excluded from the RW directory during the next reboot.
If it's a "full install", no whiteout files should be detected.

'./usr/local/mi-utils/dir2list'
A CLI utility to echo a list of directories and files contained in a directory.
Run without parameters to see usage help.
The included 'installpkg.sh' redirects the output of this utility to the '.files' file.

'./usr/local/mi-utils/save-add'
A CLI utility to write specified files from a specified directory to the '/' file system.
Run without parameters to see usage help.
The included 'installpkg.sh' calls this utility to do the actual file writing and directory creation.

'./usr/local/mi-utils/save-remove'
A CLI utility to remove specified files from the '/' file system.
Run without parameters to see usage help.
The included 'removepreview.sh' calls this utility to do the actual removing.

To use:
Either, extract somewhere and then copy the 5 files to their appropriate locations.
or
Code:
tar xf ppm_links-15.tar -C /


Note1:
While the second method is simpler, I don't recommend doing this on your production Puppy.

Note 2:
I have not provided a '.pet' file, since this is clobbering the actual file executed to install a '.pet'.

gyro
ppm_links-15.tar
Description  Extract contents to use.
tar

 Download 
Filename  ppm_links-15.tar 
Filesize  12.62 KB 
Downloaded  234 Time(s) 

Last edited by gyro on Sun 06 Oct 2019, 08:05; edited 2 times in total
Back to top
View user's profile Send private message 
gyro

Joined: 28 Oct 2008
Posts: 1799
Location: Brisbane, Australia

PostPosted: Sun 04 Aug 2019, 06:52    Post subject: PPM_links-15.1.tar - hacked versions of some PPM files
Subject description: Respects symbolic links, uninstalled files simply disappear.
 

Fixes an issue when used with an "overlayfs" stack.

Changes:
The attached file 'ppm_links-15-1.tar' contains an extra file, './usr/local/mi-utils/save-exclude'.
The file './usr/local/petget/removepreview.sh' is modified.
The file './usr/local/mi-utils/save-remove' is modified.

'./usr/local/mi-utils/save-exclude'
A CLI utility to exclude specified files from the '/' file system.
Run without parameters to see usage help.
The included 'removepreview.sh' calls this utility to get rid of files in an "overlayfs" stack.

The updated './usr/local/petget/removepreview.sh',
now calls './usr/local/mi-utils/save-exclude' for an "overlayfs" stack,
but './usr/local/mi-utils/save-remove', otherwise.

'./usr/local/mi-utils/save-remove'
Now removes files from only an "aufs" stack, (or "full install").

The acid test is to install a '.pet' containing '/etc/rc.d/rc.shutdown', no problems anywhere.
But uninstalling this pet with 'ppm_links.15' in an "overlayfs" stack is a problem,
at shutdown the whiteout file for '/etc/rc.d/rc.shutdown' is still in place, no 'rc.shutdown', big problem.

This is all because directly modifying any layer in an "overlayfs" stack is forbidden.
While 'save-remove' can directly remove the whiteout files from the RW layer of an "aufs" stack.

To use:
Either, extract somewhere and then copy the 6 files to their appropriate locations.
or
Code:
tar xf ppm_links-15-1.tar -C /


Note1:
While the second method is simpler, I don't recommend doing this on your production Puppy.

Note 2:
I have not provided a '.pet' file, since this is clobbering the actual file executed to install a '.pet'.

gyro
ppm_links-15.1.tar
Description  Extract to use.
tar

 Download 
Filename  ppm_links-15.1.tar 
Filesize  12.77 KB 
Downloaded  238 Time(s) 
Back to top
View user's profile Send private message 
gyro

Joined: 28 Oct 2008
Posts: 1799
Location: Brisbane, Australia

PostPosted: Sun 06 Oct 2019, 08:02    Post subject: PPM_links-15.4 - hacked PPM files supporting petx
Subject description: petx - 'pet' extended format
 

This release is all about 'petx', although it still performs as per previous versions.

ppm_petx-15.4.tar:
contains hacked Puppy utility files to support 'petx'.
ppm_petx-15.4.sfs:
contains the 'petx' CLI utility, and sould be loaded as an extra-sfs.

What is 'petx'?
Here is the 'petx' usage message:
--------------------------------
petx v1
Support utility for an extended 'pet' package format,
Stored in a '.pet.sfs' file.

Filename:
<package name>.pet.sfs
'./downloads/pmcputemp-0.63a.pet.sfs'

Internal format:
'data' - a directory containg the actual files to be installed.
'meta' - a directory containg 'pet.specs', 'pinstall.sh', 'md5sum.txt', ...

Usage:
petx -h
or
petx pet2 <filename of '.pet' file>
petx 2pet <filename of '.pet.sfs' file>
petx dir2 <package name directory> - with 'petx' internal format
petx 2dir <filename of '.pet.sfs' file>
petx chk <filename of '.pet.sfs' file>
petx ls <filename of '.pet.sfs' file>
--------------------------------

A 'petx' file can be installed as either a '.pet' or a '.sfs'

An easy way to start is to load ppm_petx-15.4.sfs with 'sfs_load',
and then, in a console, convert a currrent '.pet' to 'petx' with a command like:
Code:
petx pet2 /path/to/top_prog-0.3.pet
then click on 'top_prog-0.3.pet.sfs' and select 'View contents', to look inside the '.sfs' and observe the contents.

The 'petx' format includes an internal 'meta/md5sum.txt' which can be checked with command like:
Code:
 petx chk ./top_prog-0.3.pet.sfs

To try using a 'petx' file, you can install the contents of ppm_petx-15.4.tar with:
Code:
tar xf ppm_links-15-1.tar -C /
or extract it somewhere and copy the files into place.
Now click on the 'top_prog-0.3.pet.sfs' file, the hacked 'filemnt' should show a dialog where you can, view the contents, install as a '.sfs. ar install as '.pet'.
With the included hacked PPM files and hacked 'sfs_load', all these options should work.

Note1:
The 'petx dir2' facility does not include code to generate a new 'pet.specs' file, like it should.
The only way to produce a 'petx' file using the current version of the 'petx' utility is to
generate a normal '.pet' file using 'dir2pet' and then convert it using 'petx pet2'

Note2:
The 'petx ls' facility lists the files contained in the internal 'data/' directory, not all the files in the 'petx' file.

Note3:
The 'petx 2dir' facility is really just an 'unsquashfs' but to a directory with the name of the package.

Note4:
The 'petx dir2' produces a new 'meta/md5sum.txt', in case any files in the package directory were changed.
So it is preferred over 'mksquashfs' for this functionality.

gyro
ppm_petx-15.4.tar
Description  simply extract
tar

 Download 
Filename  ppm_petx-15.4.tar 
Filesize  36.45 KB 
Downloaded  143 Time(s) 
ppm_petx-15.4.sfs.gz
Description  gunzip to produce '.sfs' file
gz

 Download 
Filename  ppm_petx-15.4.sfs.gz 
Filesize  2.36 KB 
Downloaded  163 Time(s) 
Back to top
View user's profile Send private message 
gyro

Joined: 28 Oct 2008
Posts: 1799
Location: Brisbane, Australia

PostPosted: Sun 06 Oct 2019, 08:17    Post subject:  

Using 'petx' file as though it were a normal '.sfs' file:
After mounting the file at '/mount/point/dir',
add '/mount/point/dir/data' to the stack,
instead of adding '/mount/point/dir' to the stack.

gyro
Back to top
View user's profile Send private message 
sc0ttman


Joined: 16 Sep 2009
Posts: 2806
Location: UK

PostPosted: Sun 06 Oct 2019, 08:48    Post subject:  

I'd be interested in making the petx format work by default in Pkg...

Pkg can already install SFS files as though they were normal .pet packages, so internal petx support would probably be a good idea..

_________________
Pkg, mdsh, Woofy, Akita, VLC-GTK, Search
Back to top
View user's profile Send private message 
s243a

Joined: 02 Sep 2014
Posts: 2626

PostPosted: Sun 06 Oct 2019, 09:53    Post subject:  

So one could use this if they wanted to test a package but didn't necessarily want to save it to the savefile or savefolder?

Also does it overwrite any files in /usr/local/petget, or are these all alternative files?

_________________
Find me on minds and on pearltrees.
Back to top
View user's profile Send private message Visit poster's website 
s243a

Joined: 02 Sep 2014
Posts: 2626

PostPosted: Sun 06 Oct 2019, 10:08    Post subject:  

s243a wrote:
Also does it overwrite any files in /usr/local/petget, or are these all alternative files?


Related to this. Aside from the memory advantages of directly writing to the savefile/savefolder, the ability to write directly to the savefile/savefolder allows one to make changes to packages that need to be changed all at once (e.g. glibc) and would break the system if they were changed incrementally by writing each file separately to "/".

For these reason, I think it is good if the petx files are separate files from the existing package manager files or if not separate then it would be good if there were some configuration options that one could change to get the old behaviour.

BTW does writing directly to "/" fix some bug in the existing package manager?

_________________
Find me on minds and on pearltrees.
Back to top
View user's profile Send private message Visit poster's website 
gyro

Joined: 28 Oct 2008
Posts: 1799
Location: Brisbane, Australia

PostPosted: Sun 06 Oct 2019, 13:13    Post subject:  

sc0ttman wrote:
I'd be interested in making the petx format work by default in Pkg...

Pkg can already install SFS files as though they were normal .pet packages, so internal petx support would probably be a good idea..
If you look inside the '/usr/local/petget/installpkg.sh' file from 'ppm_petx-15.4.tar' you will see how easy it is to use the 'petx' format internally,
('.pet' seems to be the worst).

Of course, once 'petx' is the internal format, a 'petx' file just needs to be mounted.

gyro
Back to top
View user's profile Send private message 
gyro

Joined: 28 Oct 2008
Posts: 1799
Location: Brisbane, Australia

PostPosted: Sun 06 Oct 2019, 13:22    Post subject:  

s243a wrote:
So one could use this if they wanted to test a package but didn't necessarily want to save it to the savefile or savefolder?

Also does it overwrite any files in /usr/local/petget, or are these all alternative files?
If you do a "tar tf" of 'ppm_petx-15.4.tar' you will see that it overwrites a number of currrent Puppy files, including a number in '/usr/local/petget'.

The concept is to be a replacement for the current.
So this makes no difference to the process of 'testing' a package.

gyro
Back to top
View user's profile Send private message 
gyro

Joined: 28 Oct 2008
Posts: 1799
Location: Brisbane, Australia

PostPosted: Sun 06 Oct 2019, 13:51    Post subject:  

@s243a,

Why do I write directly to '/'?

1. It makes the code a whole lot simpler.
The process is the same for both a frugal-install, and a full-install.
The process is the same for a stack that supports direct changing of a lower layer (e.g. aufs) and a stack that does not support direct changing of any layer (e.g. overlayfs).

2. In the past PPM has had problems with symbolic links that point to directories.
When focusing on just a single layer, it is all too easy to forget that such symbolic links might exist in a lower layer.
By writing to '/', (i.e. the stack), any such links are always visible.

3. The only technical advantage I see for writing directly to a particular layer is when Puppy is running with an odd pupmode.
Then writing to the stack will write into RAM and then be copied into the save layer via 'snapmergepuppy' at some later time.

gyro
Back to top
View user's profile Send private message 
gyro

Joined: 28 Oct 2008
Posts: 1799
Location: Brisbane, Australia

PostPosted: Sun 06 Oct 2019, 14:17    Post subject: extra-sfs files and 'meta' files  

An interesting "advantage" of using 'petx' files as 'extra-sfs' files is the capability of each 'extra-sfs' to have a'pinstall.sh', a 'puninstall.sh', and a 'pet.specs' stored in the 'meta' directory.
Because these files have to to accessed via the sfs's mount point, they never confusingly appear in the stack, and they never conflict with those of another sfs file.

An example:
Say we have a 'petx' file 'Dukto-6.0_1.pet.sfs'.
During the boot process it gets mounted at '/initrd/pup_sfs6',
and the directory '/initrd/pup_sfs6/data' gets appended to the stack.
Then the file '/initrd/pup_sfs6/data/usr/bin/dukto' will appear in the stack as '/usr/bin/dukto'.
But the file '/initrd/pup_sfs6/meta/pet.specs' will not appear in the stack, it will only be accessible as '/initrd/pup_sfs6/meta/pet.specs'.

gyro
Back to top
View user's profile Send private message 
gyro

Joined: 28 Oct 2008
Posts: 1799
Location: Brisbane, Australia

PostPosted: Thu 24 Oct 2019, 13:31    Post subject:  

I have attached 'ppm_petx-15.5.tar' and 'ppm_petx-15.5.sfs'.

This release is purely to fix a bug in the '/usr/local/petget/installpkg.sh' script.

Symptoms in PPM:
If a package is installed that does not include a 'pet.specs', it appears to fail, even though the package is in fact installed and the '.files' files are present in the 'root/.packages' directory.
There is no entry for the package in '/root/.packages/user-installed-packages', only a blank line.
The '/usr/local/petget/installpkg.sh' script failed to put any information into the $DB_ENTRY variable.

Since this script resides in 'ppm_petx-15.5.tar', there is no need to download 'ppm_petx-15.5.sfs' if you already have 'ppm_petx-15.4.sfs'.

gyro
ppm_petx-15.5.tar
Description  simply extract
tar

 Download 
Filename  ppm_petx-15.5.tar 
Filesize  36.45 KB 
Downloaded  122 Time(s) 
ppm_petx-15.5.sfs.gz
Description  gunzip to produce .sfs file
gz

 Download 
Filename  ppm_petx-15.5.sfs.gz 
Filesize  2.36 KB 
Downloaded  122 Time(s) 
Back to top
View user's profile Send private message 
Display posts from previous:   Sort by:   
Page 1 of 1 [12 Posts]  
This forum is locked: you cannot post, reply to, or edit topics.   This topic is locked: you cannot edit posts or make replies. View previous topic :: View next topic
 Forum index » Advanced Topics » Cutting edge
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.0695s ][ Queries: 12 (0.0158s) ][ GZIP on ]