Pkg - CLI package manager

Under development: PCMCIA, wireless, etc.
Message
Author
s243a
Posts: 2580
Joined: Tue 02 Sep 2014, 04:48
Contact:

#101 Post by s243a »

Lobster wrote:Is this packaging system compiled/available for 64bit eg Puppy8 BionicPup?

Sounds very useful 8)

Puppy Linux
Smaller, faster, Run Free
I think it's all script based...so no need to compile :)
....
there is one exception though. I does utilize compiled tools from puppies package manager (i.e. petget) to convert a Debian repo database file into the puppy format.
Find me on [url=https://www.minds.com/ns_tidder]minds[/url] and on [url=https://www.pearltrees.com/s243a/puppy-linux/id12399810]pearltrees[/url].

wiak
Posts: 2040
Joined: Tue 11 Dec 2007, 05:12
Location: not Bulgaria

#102 Post by wiak »

pkg worked to some extent at least in a quick very experimental test in FirstRib:

http://www.murga-linux.com/puppy/viewto ... 49#1029849

I have lots of other things I want to get on with so won't be taking pkg use in FirstRib further, but it does look maybe possible that a FirstRib Puppy flavour could be built using pkg as the package manager rather than Void Linux's xbps package manager. Depends how standalone (from actual Puppy) pkg could be made. As I mention in above thread, it would be nice if pkg could have fewer full util dependencies (i.e. work with busybox alone). Very promising though!

wiak

s243a
Posts: 2580
Joined: Tue 02 Sep 2014, 04:48
Contact:

#103 Post by s243a »

pkg tries to download the following file (which doesn't exist):

http://deb.devuan.org/merged/pool/DEBIA ... 1_i386.deb

the correct path should be:

http://deb.devuan.org/merged/pool/DEBIA ... 1_i386.deb

This might not be a fault of package but might be instead due to how petget converts the debian package database to the puppy format. The "deb9u1_" might not be part of the version field in the debian package database.

**I'll look into this more later.
Last edited by s243a on Thu 06 Jun 2019, 22:12, edited 1 time in total.
Find me on [url=https://www.minds.com/ns_tidder]minds[/url] and on [url=https://www.pearltrees.com/s243a/puppy-linux/id12399810]pearltrees[/url].

wiak
Posts: 2040
Joined: Tue 11 Dec 2007, 05:12
Location: not Bulgaria

#104 Post by wiak »

s243a wrote:pkg tries to download the following file (which doesn't exist):

http://deb.devuan.org/merged/pool/DEBIA ... 1_i386.deb

the correct path should be:

http://cdn-fastly.deb.debian.org/debian ... 1_i386.deb

This might not be a fault of package but might be instead due to how petget converts the debian package database to the puppy format.
If that proves to be the case I hope that fix gets appropriately prioritised since that conversion/filter is pretty much mission critical for reliable Debian-based Puppy builds.

wiak

s243a
Posts: 2580
Joined: Tue 02 Sep 2014, 04:48
Contact:

#105 Post by s243a »

wiak wrote:
s243a wrote:pkg tries to download the following file (which doesn't exist):

http://deb.devuan.org/merged/pool/DEBIA ... 1_i386.deb

the correct path should be:

http://cdn-fastly.deb.debian.org/debian ... 1_i386.deb

This might not be a fault of package but might be instead due to how petget converts the debian package database to the puppy format.
If that proves to be the case I hope that fix gets appropriately prioritised since that conversion/filter is pretty much mission critical for reliable Debian-based Puppy builds.

wiak
Well lets verify this is a legitimate error first. Here is the DB field from the pkgdb file in the devaun ascii repo:

Code: Select all

libwayland-client0|1.12.0-1+deb9u1|libwayland-client0_1.12.0-1+deb9u1_i386.deb|http://deb.devuan.org/merged/pool/DEBIAN/main/w/wayland/libwayland-client0_1.12.0-1+deb9u1_i386.deb|optional|libs|80096899735b1b88bd1fa57042116402|libc6, libffi6,
....
more to come.

Here is the entry in dup stretch:

/root/.packages/Packages-debian-stretch-main

Code: Select all

libwayland-client0_1.12.0-1+deb9u1|libwayland-client0|1.12.0-1+deb9u1||BuildingBlock|71K|pool/main/w/wayland|libwayland-client0_1.12.0-1+deb9u1_i386.deb|+libc6&ge2.10,+libffi6&ge3.0.4|wayland compositor infrastructure - client library|debian|stretch|
So the version looks the same on both devaun ascii and dpup stretch. Now I'll try to download it using petget.

I was able to download this using petget on dpup stretch.
Find me on [url=https://www.minds.com/ns_tidder]minds[/url] and on [url=https://www.pearltrees.com/s243a/puppy-linux/id12399810]pearltrees[/url].

s243a
Posts: 2580
Joined: Tue 02 Sep 2014, 04:48
Contact:

#106 Post by s243a »

I need to find a good tutotial on the sort utility. I don't understand:

Code: Select all

# sort and clean the search results
LANG=C cat $TMPDIR/pkglist | sort --field-separator='-' -k1,1df -k2gr -k3gr -k4gr | uniq > $TMPDIR/pkglist1
/usr/sbin/pkg#L2090

Note that pkglist is generated in this line:

Code: Select all

  # create the search results
  cut -f1 -d'|' ${HOME}/.packages/${REPOFILE} 2>/dev/null | grep "^$1" > $TMPDIR/pkglist
/usr/sbin/pkg#L2058

I note that the filed seperator in the sort statment is "-". In my example (i.e. libwayland-client0_1.12.0-1+deb9u1) there are two dashes. I wonder if this causes any issues?

Here is some related strange output:

Code: Select all

bash -x pkg --names libwayland-client0 2>&1 | tee wayland_names.log
....
+ cut -f1 '-d|' /root/.packages/Packages-devuan-ascii-main
+ grep '^libwayland-client0'
+ '[' false = true ']'
+ hide_blacklisted_pkgs_from_search_results
+ '[' false = true ']'
++ echo fbset petget rgb sysfiles sysklogd
++ sed -e 's/ /|/g'
+ blacklisted_pkgs_list='fbset|petget|rgb|sysfiles|sysklogd'
+ cat /tmp/pkg//pkglist
+ grep -v -E ''\''fbset|petget|rgb|sysfiles|sysklogd'\'''
+ mv /tmp/pkg//pkglist_without_blacklisted /tmp/pkg//pkglist
+ rm '/tmp/pkg//pkglist_*'
+ '[' false = false ']'
+ local ALIAS_LIST
+ local ALIAS
+ local ALIAS_RES
+ '[' libwayland-client0 '!=' '' -a -f /tmp/pkg//pkg_aliases ']'
++ grep -m1 libwayland-client0 /tmp/pkg//pkg_aliases
++ tr , ' '
+ ALIAS_LIST=
+ echo
+ read ALIAS
+ '[' '' = '' ']'
+ continue
+ read ALIAS
+ LANG=C
+ cat /tmp/pkg//pkglist
+ sort --field-separator=- -k1,1df -k2gr -k3gr -k4gr
+ uniq
+ mv /tmp/pkg//pkglist1 /tmp/pkg//pkglist
+ '[' -s /tmp/pkg//pkglist ']'
+ cat /tmp/pkg//pkglist
libwayland-client0_1.12.0-1
+ '[' '!' -f /tmp/pkg//pkglist ']'
+ rm /tmp/pkg//pkglist
Find me on [url=https://www.minds.com/ns_tidder]minds[/url] and on [url=https://www.pearltrees.com/s243a/puppy-linux/id12399810]pearltrees[/url].

s243a
Posts: 2580
Joined: Tue 02 Sep 2014, 04:48
Contact:

#107 Post by s243a »

s243a wrote:
wiak wrote:
s243a wrote:pkg tries to download the following file (which doesn't exist):

http://deb.devuan.org/merged/pool/DEBIA ... 1_i386.deb

the correct path should be:

http://cdn-fastly.deb.debian.org/debian ... 1_i386.deb

This might not be a fault of package but might be instead due to how petget converts the debian package database to the puppy format.
If that proves to be the case I hope that fix gets appropriately prioritised since that conversion/filter is pretty much mission critical for reliable Debian-based Puppy builds.

wiak
Well lets verify this is a legitimate error first. Here is the DB field from the pkgdb file in the devaun ascii repo:

Code: Select all

libwayland-client0|1.12.0-1+deb9u1|libwayland-client0_1.12.0-1+deb9u1_i386.deb|http://deb.devuan.org/merged/pool/DEBIAN/main/w/wayland/libwayland-client0_1.12.0-1+deb9u1_i386.deb|optional|libs|80096899735b1b88bd1fa57042116402|libc6, libffi6,
....
more to come.

Here is the entry in dup stretch:

/root/.packages/Packages-debian-stretch-main

Code: Select all

libwayland-client0_1.12.0-1+deb9u1|libwayland-client0|1.12.0-1+deb9u1||BuildingBlock|71K|pool/main/w/wayland|libwayland-client0_1.12.0-1+deb9u1_i386.deb|+libc6&ge2.10,+libffi6&ge3.0.4|wayland compositor infrastructure - client library|debian|stretch|
So the version looks the same on both devaun ascii and dpup stretch. Now I'll try to download it using petget.

I was able to download this using petget on dpup stretch.
I should note that the above correct record entry woof libwayland-client0 was from the repo database file grabbed by woof next. I looked at the related one that pkg is using and it's wrong:

Code: Select all

libwayland-client0_1.12.0-1|libwayland-client0|1.12.0-1||BuildingBlock|71K|pool/DEBIAN/main/w/wayland|libwayland-client0_1.12.0-1_i386.deb|+libc6&ge2.10,+libffi6&ge3.0.4|wayland compositor infrastructure - client library|devuan|ascii||
I do know that dpup stretch correctly updated the repo, so maybe I"m missing a needed file to properly update the repo.
Find me on [url=https://www.minds.com/ns_tidder]minds[/url] and on [url=https://www.pearltrees.com/s243a/puppy-linux/id12399810]pearltrees[/url].

s243a
Posts: 2580
Joined: Tue 02 Sep 2014, 04:48
Contact:

#108 Post by s243a »

wiak wrote:
s243a wrote:pkg tries to download the following file (which doesn't exist):

http://deb.devuan.org/merged/pool/DEBIA ... 1_i386.deb

the correct path should be:

http://cdn-fastly.deb.debian.org/debian ... 1_i386.deb

This might not be a fault of package but might be instead due to how petget converts the debian package database to the puppy format.
If that proves to be the case I hope that fix gets appropriately prioritised since that conversion/filter is pretty much mission critical for reliable Debian-based Puppy builds.

wiak
My preliminarily investigation suggests that sc0tman's "pkg" didn't properly catch an error about an invalid repo path and instead reported that the repo was updated. Above investigation suggests that petget probably correctly converts the debian repo to puppy format.

My error was likely due to woof-next not properly configuring my DISTRO_SPECS. See post:
http://murga-linux.com/puppy/viewtopic. ... 74#1029974

I agree with you that the DISTRO_COMPAT_REPOS files is more complicated than it should be. As an example in the past I had trouble adding the tor repo to puppylinux:
http://murga-linux.com/puppy/viewtopic. ... 6249dd1e2d
http://li969-200.members.linode.com/pup ... d1e969ef49
http://murga-linux.com/puppy/viewtopic. ... 78#1010478

The third link says how I was able to successfully add the tor repo...a brief look back at this fix and I think it might have been an ugly hack!
Find me on [url=https://www.minds.com/ns_tidder]minds[/url] and on [url=https://www.pearltrees.com/s243a/puppy-linux/id12399810]pearltrees[/url].

s243a
Posts: 2580
Joined: Tue 02 Sep 2014, 04:48
Contact:

#109 Post by s243a »

To get more log output:

put at the top of
/usr/local/petget/0setup

Code: Select all

set -x
exec &> >(tee -a "repo_update.log")
Then run the command:

Code: Select all

bash -x pkg --repo-update | tee repo_update.log
...
+ xDLFILE=Packages.xz
+ '[' -f Packages.xz ']'
+ case ${DISTRO_BINARY_COMPAT} in
+ '[' -eq 0 ']'
/usr/local/petget/0setup: line 278: [: -eq: unary operator expected
+ LANG=C
+ echo 'Downloaded file is corrupted. Deleting file and aborting.'
Downloaded file is corrupted. Deleting file and aborting.
The related section in the code:

Code: Select all

	   xDLFILE="$DLFILE"
	   if [ -f $DLFILE ];then
		case ${DISTRO_BINARY_COMPAT} in
...
		esac
		if [ $RETSTAT -eq 0 ];then
		 LANG=${LANGORG} echo "...success."
		else
		 LANG=${LANGORG} echo "Downloaded file is corrupted. Deleting file and aborting."
/woof-code/rootfs-packages/PKG/usr/local/petget/0setup#L279

Note that there is a logfile for 0setup:

Code: Select all

echo -n "" > /var/woof/0setup_fail_report_$RUNNINGPUP #RUNNINGPUP=yes or no. latter if woof.
LANG=${LANGORG} echo "This is a report on the last time the '0setup' script was run.
Date and time '0setup' last run: ${RUNDATE}
Compatible-distro and release-name: ${DISTRO_BINARY_COMPAT}, ${DISTRO_COMPAT_VERSION}
Mostly only errors get logged, so the less seen below, the better.
Log of last run of '0setup':
" >> /var/woof/0setup_fail_report_$RUNNINGPUP
/usr/local/petget/0setup#L214

The above error not only wasn't logged (although possibly due to my process substitution) but the error was incorrectly reported. The db file wasn't corrupted. The issue was that I had the wrong value for "DISTRO_BINARY_COMPAT". The correct value should be devuan.

So anyway, pkg reports that the repo update succeeded but this is mostly due to bad error reporting in 0setup.

Edit: I stand corrected. I eventually did get an error (prior to fixing distro specs). The error was:

Code: Select all

	Failed to download db file: 
	 http://packages.devuan.org/merged/dists//main/binary-/Packages.xz
	...exited from 0setup script.
so pkg could sand this log file for the word "Failed"
Last edited by s243a on Sat 08 Jun 2019, 00:45, edited 1 time in total.
Find me on [url=https://www.minds.com/ns_tidder]minds[/url] and on [url=https://www.pearltrees.com/s243a/puppy-linux/id12399810]pearltrees[/url].

s243a
Posts: 2580
Joined: Tue 02 Sep 2014, 04:48
Contact:

#110 Post by s243a »

I was able to get pkg to properly update the repo by fixing my /etc/DISTRO_SPECS

For more info see:
http://murga-linux.com/puppy/viewtopic. ... 98#1029998
Find me on [url=https://www.minds.com/ns_tidder]minds[/url] and on [url=https://www.pearltrees.com/s243a/puppy-linux/id12399810]pearltrees[/url].

User avatar
sc0ttman
Posts: 2812
Joined: Wed 16 Sep 2009, 05:44
Location: UK

#111 Post by sc0ttman »

s243a wrote:I was able to get pkg to properly update the repo by fixing my /etc/DISTRO_SPECS

For more info see:
http://murga-linux.com/puppy/viewtopic. ... 98#1029998
I've read through your previous posts, and others about Pkg..

I guess I should ignore the bug reports above, as it seems to be a messed up DISTRO_SPECS that was the problem..

But I've made note of trying to use only Busybox..

Thanks for the testing and input.

Feel free to do a PR at gitlab, so that the devuan-ascii repos are by default listed in the `sources-all` file... So that Pkg supports these repos "out of the box"..

----------

I'm making a note here to Woof-CE dev wdlkmpx on github... No idea who you are on the forum (sorry) ...

I've replied to your comments on GitHub about Pkg:

https://github.com/puppylinux-woof-CE/w ... 9fc62d8ee1
[b][url=https://bit.ly/2KjtxoD]Pkg[/url], [url=https://bit.ly/2U6dzxV]mdsh[/url], [url=https://bit.ly/2G49OE8]Woofy[/url], [url=http://goo.gl/bzBU1]Akita[/url], [url=http://goo.gl/SO5ug]VLC-GTK[/url], [url=https://tiny.cc/c2hnfz]Search[/url][/b]

s243a
Posts: 2580
Joined: Tue 02 Sep 2014, 04:48
Contact:

#112 Post by s243a »

sc0ttman wrote:
s243a wrote:I was able to get pkg to properly update the repo by fixing my /etc/DISTRO_SPECS

For more info see:
http://murga-linux.com/puppy/viewtopic. ... 98#1029998
I've read through your previous posts, and others about Pkg..

I guess I should ignore the bug reports above, as it seems to be a messed up DISTRO_SPECS that was the problem..

But I've made note of trying to use only Busybox..

Thanks for the testing and input.

Feel free to do a PR at gitlab, so that the devuan-ascii repos are by default listed in the `sources-all` file... So that Pkg supports these repos "out of the box"..

----------

I'm making a note here to Woof-CE dev wdlkmpx on github... No idea who you are on the forum (sorry) ...

I've replied to your comments on GitHub about Pkg:

https://github.com/puppylinux-woof-CE/w ... 9fc62d8ee1
There still might be some issues but I have to correctly identify them. I've got pkg working in conjunction with mistfire's version of the puppy package manager (i.e. ppm, sometimes called petget). However, pkg isn't correctly identifying builtin files. I think the issue might be due to that the find command does not by default follow symlinks. In newer versions of puppy /root/.packages is linked to /var/packages. A quick search of the files in pkg identifyies some problem areas related to this:

Code: Select all

# grep -rn . -e 'builtin' | grep find
./CHANGELOG:110:- fixed find_deps(): remove builtins from list unless HIDE_BUILTINS=false
./CHANGELOG:297:	# this fixes finding names of pkgs that were installed over builtins
./usr/sbin/pkg:2469:  local builtins_without_busybox="`find ${HOME}/.packages/builtin_files/* -maxdepth 1 -type f | grep -v "/busybox"`"
./usr/sbin/pkg:2516:  # now check pkg names (not pkg contents) of builtins (EXCEPT busybox) for '$file'.. cos maybe we didnt find the file inside the file lists
./usr/sbin/pkg:2722:  [ ! -f "$PKG_FLIST" ] && PKG_FLIST="`find $HOME/.packages/builtin_files/ -maxdepth 1 -type f -name "$PKGNAME_ONLY"`"
Note that I also had this issue with mistfire's package manager. See post:
http://murga-linux.com/puppy/viewtopic. ... 73#1030473

Also note that in the GUI front ends for pkg, the search function doesn't seem to do any filtering. I think though that this isn't either implemented yet or deliberate.
Find me on [url=https://www.minds.com/ns_tidder]minds[/url] and on [url=https://www.pearltrees.com/s243a/puppy-linux/id12399810]pearltrees[/url].

User avatar
sc0ttman
Posts: 2812
Joined: Wed 16 Sep 2009, 05:44
Location: UK

#113 Post by sc0ttman »

s243a wrote:pkg isn't correctly identifying builtin files. I think the issue might be due to that the find command does not by default follow symlinks.
I will add a fix which detects the actual .packages folder... (My OS still uses ~/.packages... )
s243a wrote:Also note that in the GUI front ends for pkg, the search function doesn't seem to do any filtering. I think though that this isn't either implemented yet or deliberate.
There is a known issue in Pkgdialog (the ncurses UI) when the search term has a space in it ... But search *should* mostly work... does for me in Stretch RC2...
Gpkgdialog (the gtkdialog GUI) should work fine for package searches...


Of course, I first and foremost intend to get Pkg working in official Puppies ...
Your devuan acsii version is an 'edge case', for now...


So I'll look into those issues at some point...

But again, feel free to make a Pull Request (or "Merge request" in Gitlab lingo) if you find the fixes yourself...
I'll be happy to merge them in.
[b][url=https://bit.ly/2KjtxoD]Pkg[/url], [url=https://bit.ly/2U6dzxV]mdsh[/url], [url=https://bit.ly/2G49OE8]Woofy[/url], [url=http://goo.gl/bzBU1]Akita[/url], [url=http://goo.gl/SO5ug]VLC-GTK[/url], [url=https://tiny.cc/c2hnfz]Search[/url][/b]

s243a
Posts: 2580
Joined: Tue 02 Sep 2014, 04:48
Contact:

#114 Post by s243a »

How do I blacklist packages from being installed? Pkg is installing libc6 on me which is breaking things.
Find me on [url=https://www.minds.com/ns_tidder]minds[/url] and on [url=https://www.pearltrees.com/s243a/puppy-linux/id12399810]pearltrees[/url].

s243a
Posts: 2580
Joined: Tue 02 Sep 2014, 04:48
Contact:

#115 Post by s243a »

I noticed an error that is causing double slashes.

the variable whoami isn't defined until line 100

Code: Select all

whoami=$(whoami)
/usr/sbin/pkg#L100

but this variable is used to define the tmp directory on line 40 before this variable is set:

Code: Select all

TMPDIR=/tmp/pkg/${whoami}  # set the tmp dir
/usr/sbin/pkg#L40

Since whoami isn't defined prior to line#40 this causes the variable TMPDIR to have a trailing slash. A search through the code for:

Code: Select all

${TMPDIR}/
or

Code: Select all

$TMPDIR/
will show many places where there will be a double slash in the path.

Edit 1: I did a quick test

Code: Select all

echo test > /tmp/pkg//test
and the double slash doesn't seem to cause any problems. The actual path ends up being:

Code: Select all

/tmp/pkg/test
I wonder though if this might cause errors with other functions.

Edit 2: doing another test, it seems that one can have the double slash in single quoted strings and have it still work:

Code: Select all

grep -v "^\$" '/tmp/pkg//USRPKGLIST'
libjsoncpp1_1.7.4-3
Therefore:

The code doesn't seem to be wrong here but it makes the tracing output confusing because it looks like there was a missing variable or something.
Find me on [url=https://www.minds.com/ns_tidder]minds[/url] and on [url=https://www.pearltrees.com/s243a/puppy-linux/id12399810]pearltrees[/url].

s243a
Posts: 2580
Joined: Tue 02 Sep 2014, 04:48
Contact:

#116 Post by s243a »

s243a wrote:How do I blacklist packages from being installed? Pkg is installing libc6 on me which is breaking things.
I think my issue is that I don't have a libc6 entry in woof-installed-packages (I'll fix this).

As a side note, one can see the builtin packages with the command:

Code: Select all

HIDE_BUILTINS=false pkg --list-installed | grep ^hicolor
or they can do something like:

Code: Select all

grep ^hicolor woof-installed-packages
**hicolor is only used as an example.

The difference is that in the first case all packages will be shown (builtins, devx files and also user installed packages). In the latter case only builtin packages will be shown.

**BTW although I was missing the petspec record for libc6, in my opinion having the file list should be enough to prevent the package from being installed...but this is a minor point.
Find me on [url=https://www.minds.com/ns_tidder]minds[/url] and on [url=https://www.pearltrees.com/s243a/puppy-linux/id12399810]pearltrees[/url].

s243a
Posts: 2580
Joined: Tue 02 Sep 2014, 04:48
Contact:

#117 Post by s243a »

sc0ttman wrote:
s243a wrote:pkg isn't correctly identifying builtin files. I think the issue might be due to that the find command does not by default follow symlinks.
I will add a fix which detects the actual .packages folder... (My OS still uses ~/.packages... )
In the meantime I took a stab at this (commit e4b000bda57f13a6eba5a2f3fb2bd43e856189ce)
(changes incorporated into tiny_puduan_ascii-PreAlpha11)

I probably made some changes which weren't necessary. The find command seems to work okay with symlinks when no special options are provided as long as the search depth is 0. That is it will return any shell glob pattern that matches the patterns given to the find command.

Also I had assumed that pkg looks at the builtin file lists to see what package is already installed but some tests suggest that this might not be the case. Pkg might need to have the file in one of the files like *-installed-packages to prevent re-installation. My short term fix was to modify the file
/var/packages/PKGS_MANAGMENT as follows:

Code: Select all

case $DISTRO_BINARY_COMPAT in #110705
 debian|devuan|ubuntu|trisquel|raspbian) #130614
  PKG_NAME_IGNORE="adduser debconf passwd libc6 ${PKG_NAME_IGNORE}"
 ;;
esac
**libc6 is added to PKG_NAME_IGNORE, which is like block list for package installation.

I can consider doing a pull request once I've tested things enough. I'm not that good yet at git (and svn) commands though :(
Find me on [url=https://www.minds.com/ns_tidder]minds[/url] and on [url=https://www.pearltrees.com/s243a/puppy-linux/id12399810]pearltrees[/url].

s243a
Posts: 2580
Joined: Tue 02 Sep 2014, 04:48
Contact:

#118 Post by s243a »

sc0ttman wrote: Of course, I first and foremost intend to get Pkg working in official Puppies ...
Your devuan acsii version is an 'edge case', for now...
Not sure if the following is an "edge case issue" or w woof-CE issue but I'm in pupmode 13 (ii.e. usb flash mode) and files seem be copied into /initrd/pup_ro1/ (which shouldn't exist)

Code: Select all

+ cp -a --remove-destination '*' /initrd/pup_ro1/
I know that the package manager writes to the actual save layer but in my case the save layer is:

Code: Select all

/initrd/mnt/dev_save
I think this might be due to a woof-CE change.

Here is a more complete trace output:

Code: Select all

+ rmdir /home/spot/Downloads/firefox-esr_60.6.3esr-1~deb9u1_i386
+ mkdir -p /home/spot/Downloads/firefox-esr_60.6.3esr-1~deb9u1_i386
+ '[' '!' -f firefox-esr_60.6.3esr-1~deb9u1_i386.deb ']'
+ cp firefox-esr_60.6.3esr-1~deb9u1_i386.deb /home/spot/Downloads/firefox-esr_60.6.3esr-1~deb9u1_i386/firefox-esr_60.6.3esr-1~deb9u1_i386.deb
+ cd /home/spot/Downloads/firefox-esr_60.6.3esr-1~deb9u1_i386/
+ '[' -f /home/spot/Downloads/firefox-esr_60.6.3esr-1~deb9u1_i386/firefox-esr_60.6.3esr-1~deb9u1_i386.deb ']'
+ dpkg-deb --contents /home/spot/Downloads/firefox-esr_60.6.3esr-1~deb9u1_i386/firefox-esr_60.6.3esr-1~deb9u1_i386.deb
+ grep -v '/$'
+ tr -s ' '
+ cut -f6 '-d '
+ grep -v '^$'
+ sed -e 's/^.//g'
+ '[' -z '' ']'
+ '[' -f /root/.packages/firefox-esr_60.6.3esr-1~deb9u1_i386.files ']'
++ grep -m1 i386-linux-gnu /root/.packages/firefox-esr_60.6.3esr-1~deb9u1_i386.files
+ pkg_has_archdir=
+ '[' i386-linux-gnu '!=' '' -a -f /root/.packages/firefox-esr_60.6.3esr-1~deb9u1_i386.files -a '' '!=' '' ']'
+ dpkg-deb -x /home/spot/Downloads/firefox-esr_60.6.3esr-1~deb9u1_i386.deb /initrd/pup_ro1/
++ cat /tmp/pkg//pkg-cp-errlog
++ grep 'tar: Exiting with failure status due to previous errors'
+ '[' '' '!=' '' ']'
+ '[' -d /DEBIAN ']'
+ dpkg-deb -e /home/spot/Downloads/firefox-esr_60.6.3esr-1~deb9u1_i386/firefox-esr_60.6.3esr-1~deb9u1_i386.deb /DEBIAN
+ rm firefox-esr_60.6.3esr-1~deb9u1_i386.deb
+ cd -
+ '[' '!' -d firefox-esr_60.6.3esr-1~deb9u1_i386/ ']'
+ cd firefox-esr_60.6.3esr-1~deb9u1_i386/
+ cp -a --remove-destination '*' /initrd/pup_ro1/
+ '[' -s /tmp/pkg//pkg-cp-errlog ']'
+ cat /tmp/pkg//pkg-cp-errlog
+ grep -E 'Cannot create symlink to|cannot overwrite non-directory'
here is the command that I used:

Code: Select all

 bash -x pkg -i -f firefox-esr_60.6.3esr-1~deb9u1_i386.deb 2>&1 | tee /root/firefox_inst.log
Edit: After further examination this might be an issue with mistfire's package manager. I search for pup_ro1 returns nothing that seems relevant in the gitlab branch for pkg. I did notice though that pup_r01 is used in sfs_loadr.

Code: Select all

   6) SAVE_LAYER='/pup_rw'; PUP_HOME='/pup_rw'; PUPSAVE='sda1,ext2,/';;
   12) SAVE_LAYER='/pup_rw'; PUP_HOME='/mnt/dev_save';;
   13) SAVE_LAYER='/pup_ro1'; PUP_HOME='/mnt/dev_save';;
   77) SAVE_LAYER='/pup_ro1'; PUP_HOME=''; PUPSAVE='sr0,iso9660,/2011-01-27-20-26';;
/usr/sbin/sfs_loadr#L1697

I think that PUP_HOME should either be /mnt/home or /initrd/mnt/dev_save/

**note that /mnt/home is a symlink pointing to /initrd/mnt/dev_save
Find me on [url=https://www.minds.com/ns_tidder]minds[/url] and on [url=https://www.pearltrees.com/s243a/puppy-linux/id12399810]pearltrees[/url].

User avatar
sc0ttman
Posts: 2812
Joined: Wed 16 Sep 2009, 05:44
Location: UK

#119 Post by sc0ttman »

s243a wrote:
sc0ttman wrote: Of course, I first and foremost intend to get Pkg working in official Puppies ...
Your devuan acsii version is an 'edge case', for now...
Not sure if the following is an "edge case issue" or a woof-CE issue
If you are not running a standard Puppy (which you are not), then for Pkg it's 100%
an "edge case"
- because Pkg is designed (mainly) for standard Puppy Linux releases.

I have no idea if your custom OS (half Puppy, half something else) has everything setup
correctly... I don't even know if it was built with Woof... Hence, your OS is the "edge-case"
as far as Pkg development is concerned.

Anyway, I tried to work out if it's a Pkg issue....
I know that the package manager...
Which one?? Pkg? PPM? PPM 3.0?
..writes to the actual save layer but in my case
the save layer is:

Code: Select all

/initrd/mnt/dev_save
.....

Edit: After further examination this might be an issue with mistfire's package manager.

I search for pup_ro1 returns nothing that seems relevant in the gitlab branch for pkg.
Then I think you should probably post it at the PPM 3.0 thread, or even better work out if it's
actually a Woof-CE issue before posting at either... I guess..?
I did notice though that pup_r01 is used in sfs_loadr.

Code: Select all

   6) SAVE_LAYER='/pup_rw'; PUP_HOME='/pup_rw'; PUPSAVE='sda1,ext2,/';;
   12) SAVE_LAYER='/pup_rw'; PUP_HOME='/mnt/dev_save';;
   13) SAVE_LAYER='/pup_ro1'; PUP_HOME='/mnt/dev_save';;
   77) SAVE_LAYER='/pup_ro1'; PUP_HOME=''; PUPSAVE='sr0,iso9660,/2011-01-27-20-26';;
/usr/sbin/sfs_loadr#L1697

....

I think that PUP_HOME should either be /mnt/home or /initrd/mnt/dev_save/

**note that /mnt/home is a symlink pointing to /initrd/mnt/dev_save
Yep.. That is what I have on my system (Stretch RC something..) and most Pups I'd imagine..
But it depends on the install method/type... Sorry but I can't reproduce those issues..

...More generally ...

I think maybe you're posting "too soon", if you don't mind me saying so...

The first long series of posts you posted to the Pkg thread turned out to be your /etc/DISTRO_SPECS
not being setup correctly, which wouldn't have happened in a standard Puppy...

And sorry, but I also don't want this thread mixed up with mistfires PPM 3.0 (which literally runs
against to the goals of this project).. Pkg is not designed to support PPM 3.0 (unless it becomes
Puppys official/main PPM).. even then, Pkg aims to become independent of any PPM..

...

I think you should probably test Pkg on a standard Puppy to compare to your OS before posting any
issues here in order to make sure the issues are with Pkg and not your custom OS..

It's nice that you're sharing all your devuan-ascii Woof-next/[tazPup?/TinyCore?] endeavors on the forum,
but it should probably be in it's own thread, and anything posted here should relate to Pkg on a
standard Puppy release (that is, a Puppy built in the usual way, with a standard Woof-CE setup)..

.. or at least until you are 100% sure the Pkg problem exists in a standard Puppy too...

And FYI, people:

There are already a whole list of Issues for Pkg (see first post), it would be nice if people try to tackle those,
cos some are dead simple - add a dir2deb function, add a --list-builtins command, and so on...

Any Pkg Issues tackles should be added a Merge Requests (Pull Requests/PRs) on the GitLab page..
[b][url=https://bit.ly/2KjtxoD]Pkg[/url], [url=https://bit.ly/2U6dzxV]mdsh[/url], [url=https://bit.ly/2G49OE8]Woofy[/url], [url=http://goo.gl/bzBU1]Akita[/url], [url=http://goo.gl/SO5ug]VLC-GTK[/url], [url=https://tiny.cc/c2hnfz]Search[/url][/b]

User avatar
sc0ttman
Posts: 2812
Joined: Wed 16 Sep 2009, 05:44
Location: UK

#120 Post by sc0ttman »

s243a wrote:In the meantime I took a stab at this (commit e4b000bda57f13a6eba5a2f3fb2bd43e856189ce)

I probably made some changes which weren't necessary.
You should probably define the right package dir at the top of the /usr/sbin/pkg script as a variable,
then use that variable later throughout the script..

Or even better, define it in the main config RC script: ~/.pkg/pkgrc

Then it can be set to various different places - easier to run Pkg in chroot, ssh, local folder for unit tests ;) , etc..

Also I had assumed that pkg looks at the builtin file lists to see what package is already
installed but some tests suggest that this might not be the case.
Yes it does - as stated in this thread a few posts back - Pkg will work out if a package is 'builtin'
and refuse to do anything with it in that case...

Here are some of the places Pkg checks builtin packages:

Code: Select all

# cat /usr/sbin/pkg | grep -A2 -B2 '/builtin'
  local FILENAME_ONLY=''
  local DIRNAME=''
  local builtins_without_busybox="`find ${HOME}/.packages/builtin_files/* -maxdepth 1 -type f | grep -v "/busybox"`"

  # get user input
--
  # if needed, search inside builtin file lists (EXCEPT busybox) for ' $FILENAME_ONLY'
  [ "$PKG_FILE_LIST" = '' ] && [ "$PKGNAME" = "" ] \
    && PKGNAME="$(basename `grep -l "^ $FILENAME_ONLY\$" $builtins_without_busybox | sed "s#$HOME/.packages/builtin_files/##g"` 2>/dev/null)"

  # if that didn't work, search inside builtin file lists of busybox for ' $FILENAME_ONLY'
  [ "$PKG_FILE_LIST" = '' ] && [ "$PKGNAME" = "" ] \
    && PKGNAME="$(basename `grep -l "^ $FILENAME_ONLY\$" ${HOME}/.packages/builtin_files/busybox | sed "s#$HOME/.packages/builtin_files/##g"` 2>/dev/null)"

  # apps in ADRV sfs dont have their files listed in their $HOME/.packages/builtin_files/$pkgname.. so..


  # now check pkg names (not pkg contents) of builtins (EXCEPT busybox) for '$file'.. cos maybe we didnt find the file inside the file lists
  [ "$PKGNAME" = "" ] \
    && PKGNAME="`echo "$builtins_without_busybox" | grep -m1 "^${FILENAME_ONLY}\$" | sed "s#$HOME/.packages/builtin_files/##g" 2>/dev/null`"

  # now look for '$file' in user installed repo list
--
  # try /$file_* in builtins
  [ "$PKGNAME" = "" ] \
    && PKGNAME="$(basename `grep -l " ${FILENAME_ONLY}_" ${HOME}/.packages/builtin_files/* | sed "s#$HOME/.packages/builtin_files/##g"` 2>/dev/null)"

  # try $file-* in builtins
  [ "$PKGNAME" = "" ] \
    && PKGNAME="$(basename `grep -l " ${FILENAME_ONLY}-" ${HOME}/.packages/builtin_files/* | sed "s#$HOME/.packages/builtin_files/##g"` 2>/dev/null)"

  # try $file.* in builtins
  [ "$PKGNAME" = "" ] \
    && PKGNAME="$(basename `grep -l " ${FILENAME_ONLY}\." ${HOME}/.packages/builtin_files/* | sed "s#$HOME/.packages/builtin_files/##g"` 2>/dev/null)"

  # try *file in builtin pkgs
  [ "$PKGNAME" = "" ] \
    && PKGNAME="$(basename `grep -l "${FILENAME_ONLY}"\$ ${HOME}/.packages/builtin_files/* | sed "s#$HOME/.packages/builtin_files/##g"` 2>/dev/null)"


--

  # check if we need to (and can) get our pkg contents from builtins/PKGNAME
  local  need_to_check_builtins=`echo "$PKG_FLIST" | grep -m1 'packages/builtin_files/'`

  # if the pkg is a builtin, we will re-format the output of LIST to match the others (full paths on each line)
--
    rm $TMPDIR/pkg_file_list 2>/dev/null

    # first we get the package contents from $HOME/.packages/builtin_files/$PKGNAME_ONLY
    cat "$PKG_FLIST" 2>/dev/null | while read line
    do
--

  # try the builtins files (exact match)
  [ ! -f "$PKG_FLIST" ] && PKG_FLIST="`find $HOME/.packages/builtin_files/ -maxdepth 1 -type f -name "$PKGNAME_ONLY"`"

  # try PKGNAME (exact match) in user installed pkgs
--
    # its a file path, we will cat it, if not, its the pkg contents
    # themselves, we echo it
    if [ "`echo "$PKG_FLIST" | grep -m1 '/builtin_files/'`" != '' -o "`echo "$PKG_FLIST" | grep -m1 "packages/$PKGNAME"`" != '' ];then
      print_cmd=cat
    fi
--

    # check builtins for PKGNAME_ONLY, if it exists, its a builtin pkg
    if [ -d ${HOME}/.packages/builtin_files/ -a "$pkg_found" = false ];then
      pkg_found=`LANG=C is_builtin_pkg "${PKGNAME_ONLY}"`
      [ "$pkg_found" = true ] && install_status="installed (builtin)"
 
Pkg might need to have the file in one of the files like *-installed-packages to prevent re-installation.
On a normal Puppy, Pkg *should* refuse to install a package if it's already a 'builtin'
My short term fix was to modify the file /var/packages/PKGS_MANAGMENT as follows:

Code: Select all

case $DISTRO_BINARY_COMPAT in #110705
 debian|devuan|ubuntu|trisquel|raspbian) #130614
  PKG_NAME_IGNORE="adduser debconf passwd libc6 ${PKG_NAME_IGNORE}"
 ;;
esac
**libc6 is added to PKG_NAME_IGNORE, which is like block list for package installation.
IIRC.. That is indeed the right way to "blacklist" packages - Pkg respects the PKG_NAME_IGNORE stuff, so does PPM...

Unfortunately, it's one of those areas where Pkg relies on Woof/PPM config files :/
I can consider doing a pull request once I've tested things enough. I'm not that good yet at git (and svn) commands though :(
I'll get around to testing, fixing these things at some point.. So please do if I'm too slow...
But I log all the issues, so I don't forget..
[b][url=https://bit.ly/2KjtxoD]Pkg[/url], [url=https://bit.ly/2U6dzxV]mdsh[/url], [url=https://bit.ly/2G49OE8]Woofy[/url], [url=http://goo.gl/bzBU1]Akita[/url], [url=http://goo.gl/SO5ug]VLC-GTK[/url], [url=https://tiny.cc/c2hnfz]Search[/url][/b]

Post Reply