Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   <b>READ-ONLY-MODE: PLEASE DO NOT POST NEW STUFF</b>READ-ONLY-MODE: PLEASE DO NOT POST NEW STUFF 

The time now is Sat 08 Aug 2020, 23:53
All times are UTC - 4
 Forum index » Off-Topic Area » Programming
Minimum Operating Systems to host package manager in chroot
Post new topic   Reply to topic View previous topic :: View next topic
Page 1 of 1 [12 Posts]  
Author Message
s243a

Joined: 02 Sep 2014
Posts: 2626

PostPosted: Wed 03 Apr 2019, 13:29    Post subject:  Minimum Operating Systems to host package manager in chroot  

When I was experimenting with a 64bit version of TazPup, I found it helpful to be able to use slacko64's package manager to download packages, in the event that tazpkg didn't have available in it's repo's something that I wanted to install (e.g. vlc). However, extracting the basesfs of Slacko64 obviously gave me many more files then I would need to run puppies package manager.

I've often wanted a slimer base system for puppylinux, where only the most core files were in the base sfs and the other less core packages were moved to the adrv (or other sfs files). Having the package manager as the target application provides a design goal for this objective. This isn't necessarily the best design goal for a minimal system, rather it is the goal which suited my application.

My initial idea was that PuPnGo might make a good base system and that I could add a package manager to it from either tazpkg or x-slacko-slim. The reason that I thought that x-slacko slim might be a good choice is that mistfire has adapated petget in it to be able to download and install packages from more repo's then a typical puppy and this includes void and tinycore packages/repos.

PuPnGo is based on puppylinux 412, so the most compatible repos would be the puppylinux 412 repos or if I was looking for a comparable SliTaz repo I would choose version 4 of SliTaz (based on comparing bash versions). Obviously comparing glib versions would be more applicable but I haven't done this yet.

The reason that I looked at bash was that I thought an older version of bash might save significant space and I thought that PuPnGo's bash might be smaller since it may have been compiled against ulibc.

It turns out though that the bash in PuPnGo is not smaller. I presume that this is because it is statically compiled. However, at least in the case of slitaz the dependency libtinfo is only 128k. Moreover, in either case bash is only around 1MB and while packages of this size can add up, bash is fairly core in my opinion (although not required for tazpkg).

Anyway core PupNGo is around 8MB but that doesn't even include busybox! Perhaps PuPnGo pulls busybox from initrd. I'm not sure. The complete tazpkg with all it's dependencies is only 13MB.

The dpendencies of tazpkg are:
Code:

  busybox (824K)
    slitaz-base-files (568K)
      gettext-base (116K)
        glibc-base (4M)
    ncurses-common (120K)


I'm not sure what the functionality differences are between the base PuPnGo and a minimum tazpkg but for my purposes PuPnGo isn't offering any clear advantages except for perhaps more static libs, which might be advantages if one is messing around with core dependencies.

Anyway, I'll look at how much things grow once I try to get a whole petget to work but for now here are some base systems for tazpkg:

tazpkg_all_deps 13MB (includes busybox, slitaz-base-files, glibc-base, ncurses-common)
tazpkg_slim 6.8MB (includes slitaz-base-files)
tazpkg_slimist_4 6MB (stripped down slitaz-base-files, using slitaz version 4 as the repo)
tazpkg_slimist_cooking 6MB (stripped down slitaz-base-files, using slitaz cooking as the repo. Cooking refers to rolling release version of Slitaz)
rootfs-tazpkg+bash 15.1MB (tazpkg_all_deps + bash + some preliminary files for petget)

The idea of tazpkg_slimist_4 was it might be a compatible starting point to combine with PuPnGo since as noted above the SliTaz 4 repo might have similar version of libraries as PupnGo. ALL SliTaZ files in the above systems are from either the cooking branch or the next32 branch. My intent was to use the next32 branch but by default the packages are configured to use the cooking branch so I'm not completely sure which versions I got.

These base systems are built using tazpkg since tazpkg allows you to install packages at an alternative root. The one exception being the last item on the above list (i.e. rootfs-tazpkg+bash) which was built by taking tazpkg_all_deps and then adding both bash-4.4.5 and libtinfo-5.9.tazpkg (a dependency for bash), in the SliTaz repo. These last two packages were added by some code that I'm working on:
install-min-pkgs.sh
mk-min-taz-functions.sh

The code is designed so that you can install packages from either a directory or a file. In the script, I use uextract to extract the files. I convert pet specs to tazpkg receipts and vis versa if one of them is available otherwise I use dir2pet to generate the pet specs. The downside here is that there is no option in dir2pet to not pack the archive. I'll have to do some mods to fix this.

Another downside is that uextract duplicates the extraction folder if it already exists but this is probably a bug in my code as I probably want to delete the old one first if it exists before running uextract.

Anyway, the above code (i.e. install-min-pkgs.sh) adds the information needed for both petget and tazpkg to know what is installed. The idea is that one can provide this type of info for one or multiple package managers:
Code:

  mkdir -p "$dest_root/root/.packages/builtin_files"
  mkdir -p "$dest_root/var/lib/tazpkg/installed/$pkg_short_name" 
  for spec_type in $(eval 'echo "${'$spec_types_arry_name'[@]}"'); do
      case "$spec_type" in
      pet|.pet)
        pet_specs_fm_dir "$source_dir" "$dest_root"/
        cp "$source_dir/pet.specs" \
           "$dest_root/root/.packages/builtin_files/$pkg_short_name"
        ;;
      tazpkg)
        tazpkg_specs_fm_dir "$source_dir"
       
        cp "$source_dir/$pkg_name" "$dest_root/var/lib/tazpkg/installed/$pkg_short_name/files.list"
        ;;
      esac
  done

Currently though, I haven't written a sync mechanism for adding this type of information to a new package manager once it is added to the system, so that the new package manager is up to date. Anyway, this code is very preliminary and should be run in a sandbox...although I haven't.

The sandbox helps to protect against coding errors where a source or destination root is not assigned properly, or if a symlink targets the system rootfs (remember that non relative symbolic links will point to the root system if one is not running in a chroot environment).

The purpose of the above systems is to have a small system that one can play with in a chroot environment to download packages and examine dependencies. All code is very preliminary so I recommend people wait before it is more mature before playing with it.

Last edited by s243a on Wed 03 Apr 2019, 14:10; edited 2 times in total
Back to top
View user's profile Send private message Visit poster's website 
wanderer

Joined: 20 Oct 2007
Posts: 1119

PostPosted: Wed 03 Apr 2019, 14:05    Post subject:  

hi s243a

I am very interested in what you are doing
I hope you will continue
if you need a tester or just a user opinion I am available

I once got a puppy ramdrive core down to 2 mgs
and it would boot and symlink a usr directory to the ramdrive
everything big was in the usr directory
which was uncompressed on the hard drive so you could mess with it

I did the same thing with moving everything to the save file
in a normal puppy

but I didn't have the expertise to add many applications
just dillo gtkedit and emelfm

I think what you are doing
could be very useful to the puppy community
both as a base and a distro
and could spawn a whole puppy genre

wanderer
Back to top
View user's profile Send private message 
oui

Joined: 20 May 2005
Posts: 3606
Location: near Woof (Germany) :-) - 3 PC's: DELL SX280 750 MB Pentium4, Acer emachines 2 GB AMD64. DELL XPS15

PostPosted: Wed 03 Apr 2019, 16:16    Post subject: Re: Minimum Operating Systems to host package manager in chroot  

s243a wrote:
The idea of tazpkg_slimist_4 was it might be a compatible starting point to combine with PuPnGo since as noted above the SliTaz 4 repo might have similar version of libraries as PupnGo. ALL SliTaZ files in the above systems are from either the cooking branch or the next32 branch. My intent was to use the next32 branch but by default the packages are configured to use the cooking branch so I'm not completely sure which versions I got.


As far as I know (i was so! I don't know if it continues to stay so today)

a- all packages from the releases are offered as binaries on http://mirrors.slitaz.org, being downloadable as "download all the web site" or web site part, and

b- SliTaz was/is relatively easy to use a local repository as pseudo website directly on the own computer using /var/www , in SliTaz always donnected per default!

I suppose it is important to proceed so as there is no security that changes did not happen from one minute to the next making the chances from an homogeneous work as null (it would be the same thing in NuTyx: You willl be tried to try to compile as, it is astonish, the changes at the git always happens as YOU are try a new compilation Embarassed ...). It would be necessary to introduce a version's selector in the scripts building in assembly language and to store a part of the more recent old versions...

.

Last edited by oui on Wed 03 Apr 2019, 16:46; edited 2 times in total
Back to top
View user's profile Send private message 
oui

Joined: 20 May 2005
Posts: 3606
Location: near Woof (Germany) :-) - 3 PC's: DELL SX280 750 MB Pentium4, Acer emachines 2 GB AMD64. DELL XPS15

PostPosted: Wed 03 Apr 2019, 16:25    Post subject: Re: Minimum Operating Systems to host package manager in chroot  

s243a wrote:
When I was experimenting with a 64bit version of TazPup, I found it helpful to be able to use slacko64's package manager to download packages, in the event that tazpkg didn't have available in it's repo's something that I wanted to install (e.g. vlc).


I am not software designer at all.

And I never open the motor hood or look under the kirt!

But as far I know, SliTaz64 has to be a 64 bit kernel running full with 32 bit stuff like the usual SliTaz distribution (declaration of Aleksej, the actual main developer else if he names himself with modesty "hobbyist coder", on the SliTaz forum).

Barry Kauler did also announce us such a design at his time!

If I am right, this would mean you develop an not existing SliTaz full 64 (*1 outside of the SliTaz community!

Would that be right?

(*1 with VLC over that! As I hate (*2 the VLC monopolist influence, the horror pure for me! I hope it will be easy to desintall VLC and use our old versions instead of VLC without damage at the system?

(*2 why I hate VLC? VLC is one of the software requiring a confirmation from user to submit to the VLC conditions. As I never use some save file / dir, and start each day sometimes more that one time each day a PUPPY FIRST START, as it was in Puppy in the long first years until release (about, I am not sure any more if it did happen at that release) 1.7 this detail is a great inconvenient and shows that VLC is absolutely no free but spy software and remember it each new start!
Back to top
View user's profile Send private message 
musher0

Joined: 04 Jan 2009
Posts: 15041
Location: Gatineau (Qc), Canada

PostPosted: Wed 03 Apr 2019, 19:30    Post subject:  

Thanks, s243a.

Interesting.

_________________
musher0
~~~~~~~~~~
"You want it darker? We kill the flame." (L. Cohen)
Back to top
View user's profile Send private message 
s243a

Joined: 02 Sep 2014
Posts: 2626

PostPosted: Wed 03 Apr 2019, 20:26    Post subject: Re: Minimum Operating Systems to host package manager in chroot  

oui wrote:
s243a wrote:
When I was experimenting with a 64bit version of TazPup, I found it helpful to be able to use slacko64's package manager to download packages, in the event that tazpkg didn't have available in it's repo's something that I wanted to install (e.g. vlc).


I am not software designer at all.

And I never open the motor hood or look under the kirt!

But as far I know, SliTaz64 has to be a 64 bit kernel running full with 32 bit stuff like the usual SliTaz distribution (declaration of Aleksej, the actual main developer else if he names himself with modesty "hobbyist coder", on the SliTaz forum).

Barry Kauler did also announce us such a design at his time!

If I am right, this would mean you develop an not existing SliTaz full 64 (*1 outside of the SliTaz community!

Would that be right?

(*1 with VLC over that! As I hate (*2 the VLC monopolist influence, the horror pure for me! I hope it will be easy to desintall VLC and use our old versions instead of VLC without damage at the system?

(*2 why I hate VLC? VLC is one of the software requiring a confirmation from user to submit to the VLC conditions. As I never use some save file / dir, and start each day sometimes more that one time each day a PUPPY FIRST START, as it was in Puppy in the long first years until release (about, I am not sure any more if it did happen at that release) 1.7 this detail is a great inconvenient and shows that VLC is absolutely no free but spy software and remember it each new start!


Please see my response, in the thread, " TazPup64 (pre-alpha7) - Verified Sound Works. ", note that; I haven't tried booting directly from the ISO yet. Instead I booted from the files in the ISO.
Back to top
View user's profile Send private message Visit poster's website 
rufwoof


Joined: 24 Feb 2014
Posts: 3725

PostPosted: Thu 04 Apr 2019, 16:14    Post subject:  

mpv is my first port of call. vlc only if I have to.

mpv plays sounds, videos and can preview images, in a folder full of .png's for instance mpv *.png ... and it steps through them one by one.

_________________
( ͡° ͜ʖ ͡°) :wq
Fatdog multi-session usb

echo url|sed -e 's/^/(c/' -e 's/$/ hashbang.sh)/'|sh
Back to top
View user's profile Send private message 
wanderer

Joined: 20 Oct 2007
Posts: 1119

PostPosted: Thu 04 Apr 2019, 21:26    Post subject:  

hi s243a

this is just a question from a non guru


what i understand you are doing is

1. you will have a core

2. and a package manager that will download and resolve applications
from various sources and load them onto the core


so you will only need to maintain the core and the package manager
to have an infinitely variable minimal build system


also would it be simpler to use apps symlinked to an initrd
rather than a layered file system


wanderer
Back to top
View user's profile Send private message 
s243a

Joined: 02 Sep 2014
Posts: 2626

PostPosted: Thu 04 Apr 2019, 22:01    Post subject:  

wanderer wrote:
hi s243a


also would it be simpler to use apps symlinked to an initrd
rather than a layered file system


wanderer


I'm open to different approaches but I need to learn more about the symlinked approach. Also making a fully running system is sort of a second step. The primary core systems that I have in mind are focused on a chroot environment and might possibly (but not necessarily) be re-organized when being transformed into a fully bootable system.
Back to top
View user's profile Send private message Visit poster's website 
wanderer

Joined: 20 Oct 2007
Posts: 1119

PostPosted: Thu 04 Apr 2019, 22:12    Post subject:  

s243a

what you are doing here is truly groundbreaking

it will revolutionize the build system

and allow people to create amazing distros

thanks for doing it

wanderer
Back to top
View user's profile Send private message 
musher0

Joined: 04 Jan 2009
Posts: 15041
Location: Gatineau (Qc), Canada

PostPosted: Fri 05 Apr 2019, 02:13    Post subject:  

Hi s243a and all.

@s243a: if considering the symlinked approach, perhaps you will find the
GNU command < stow > of interest.

As starting points:
https://linuxconfig.org/how-to-use-gnu-stow-to-manage-programs-installed-from-source-and-dotfiles

https://www.ecosia.org/search?q=GNU+stow&addon=opensearch

IHTH

_________________
musher0
~~~~~~~~~~
"You want it darker? We kill the flame." (L. Cohen)
Back to top
View user's profile Send private message 
s243a

Joined: 02 Sep 2014
Posts: 2626

PostPosted: Fri 05 Apr 2019, 02:49    Post subject:  

musher0 wrote:
Hi s243a and all.

@s243a: if considering the symlinked approach, perhaps you will find the
GNU command < stow > of interest.

As starting points:
https://linuxconfig.org/how-to-use-gnu-stow-to-manage-programs-installed-from-source-and-dotfiles

https://www.ecosia.org/search?q=GNU+stow&addon=opensearch

IHTH




Thankyou for the links and I'll consider all of this stuff but as I noted above; building a functional ISO is a secondary goal. The chroot system is the primary goal. For a functional ISO I will probably do a traditional puppy first and this is mostly because I already have scripts for this based on mistfire's build-kit for TazPup.

That said in a chroot environment, I probably don't want all the overhead of a layered file-system and may first try sym-links (and/or binds) in the chroot environment before trying a layered file system approach. Another option for the chroot approach is a hybird approach where one works directly with one of the layers but sym-links to a fully layered system. In this approach perhaps the PATH and LD_LIBRARY_PATH can be modified to search the fully layered system if the file isn't found directly on a given layer.

(As I note above?), I already have scripts to dissect a rootfs into separate units stored in separate directories (see thread, "stripping down a puppy") so I can aim to make the build system flexible enough that one can choose either a symlink or a layered file system approach.

Regarding the "stripping down thread"' this thread takes the opposite approach but despite this the strip down approach used in the other thread was still educational in that it lets me organize the packages of a working puppy into functional units and furthermore as I note above, the dissect_rootfs() function that I developed in the "strip down approach" can be used in the bottom-up approach for organizing packages into various units.

Code:

dissect_rootfs(){
  #arr_names=${1:-ALL_ArrNames}
  arry_names_name=${1:-ALL_ArrNames}
  for arr_name in $(eval 'echo "${'$arry_names_name'[@]}"'); do
    eval "arr=( \"\${"$arr_name"[@]}\" )"
    eval "arr_action=\$"$arr_name"_action"
    arr_action=${arr_action:-"$action"}
    #Move is faster but perhaps copy is safer
    arr_action=${arr_action:-cp}
    set -x
    eval 'target="$'$arr_name'_target"'
    target=${target:-"$curdir/$arr_name"}
    set +x
    if [ "${s_rootfs:0:1}" = "/" ]; then
      s_rootfs_prefixed="$s_rootfs"   
    else
      s_rootfs_prefixed="$curdir/$s_rootfs"     
    fi 
    for app in "${arr[@]}"; do
      echo "arr_action=$arr_action"
      case "$arr_action" in
      cp|mv|retry) 
          echo "app=$app"
          copy_built_in "$app"       
          ;;
      esac
      if [ -f "/tmp/trim_puppy/$app" ] && [ $chroot_remove_builtin -eq 1 ]; then
        file_list="$curdir/$s_rootfs/root/.packages/builtin_files/$app"
        rm "$file_list"
        cp "/tmp/trim_puppy/$app" "$file_list"
      fi
      #chroot "$s_rootfs_prefixed" remove_builtin "$app"
    done
  done
}
Back to top
View user's profile Send private message Visit poster's website 
Display posts from previous:   Sort by:   
Page 1 of 1 [12 Posts]  
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » Off-Topic Area » Programming
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.2018s ][ Queries: 12 (0.0143s) ][ GZIP on ]