Better command line tools, and pkg manager suggestions

What features/apps/bugfixes needed in a future Puppy
Message
Author
User avatar
sc0ttman
Posts: 2812
Joined: Wed 16 Sep 2009, 05:44
Location: UK

#16 Post by sc0ttman »

musher0 wrote:there is no way I'll remember all those options by heart
Hopefully you won't have to ..

The options are designed to match the words/sounds you'd use to tell pkg what to do..
The idea is you should be able to eventually predict what the options are going to be:

Tell pkg to do a package build:

Code: Select all

pkg -pb vlc
Show pkg contents of vlc:

Code: Select all

pkg -c vlc
Tell pkg to get deps of vlc:

Code: Select all

pkg -e vlc
Tell pkg to list the deps of vlc:

Code: Select all

pkg -le vlc
Tell pkg to create sfs with combined VLC and deps:

Code: Select all

pkg -sc vlc
And many other options should be familiar:

--dir2pet, --dir2sfs, etc

Also, pkg will auto-complete options:

Code: Select all

# pkg --deps[TAB TAB]
--deps           --deps-all       --deps-check     --deps-download

Code: Select all

# pkg --[TAB TAB]
--add-source           --help                 --repo
--all                  --help-all             --repo-convert
--all-pkg              --install              --repo-dep-scope
--ask                  --install-all          --repo-file-list
--bleeding-edge        --list-deps            --repo-info
--contents             --list-downloaded      --repo-list
--deb2pet              --list-installed       --repo-pkg-scope
--delete               --names                --repo-update
--delete-all           --names-all            --search
--deps                 --names-exact          --search-all
--deps-all             --names-exact-all      --sfs2pet
--deps-check           --pet2sfs              --sfs-combine
--deps-download        --pet2tgz              --show-config
--dir2pet              --pkg-build            --tgz2pet
--dir2sfs              --pkg-build-list       --txz2pet
--dir2tgz              --pkg-combine          --uninstall
--download             --pkg-installed        --uninstall-all
--examples             --pkg-repack           --update-sources
--force                --pkg-status           --version
--get                  --pkg-update           --which
--get-only             --recursive-dep-check  --workdir
All short options are the first letters of each word:

Code: Select all

--pkg-build    -> -pb
--pkg-combine    -> -pc
--get-only    -> -go
More examples:

Code: Select all

 pkg -u "PKG1 PKG2"		uninstall multiple packages from the system
 pkg -d "PKG1 PKG2"		download multiple packages from the repo
 pkg -pb "PKG1 PKG2"		compile, build & install multiple pkgs
 pkg -li "PKG1 PKG2"		list installed pkgs matching PKG1 or PKG2
 pkg -s SEARCH			list pkgs in current repo matching SEARCH
 pkg -sa SEARCH			list pkgs in all repos matching SEARCH
 pkg -n SEARCH			search name only, list all matching pkgs
 pkg -g filezilla		get & install filezilla and deps
 pkg -a -g filezilla		ask to download filezilla and each of its deps
 pkg -a -la			ask to delete all downloaded packages
 pkg -a -e PKGNAME		install deps of PKGNAME, ask each time
 pkg -a -d qupzilla-1.2.0	ask to download qupzilla-1.2.0
 pkg -go filezilla		download filezilla & deps, don't install
 pkg -e PKGNAME			install all deps of PKGNAME, dont ask
 pkg -d qupzilla-1.3.1		download qupzilla-1.3.1 no questions
 pkg -l qupzilla-1.3.1		delete the downloaded qupzilla-1.3.1
 pkg -pb jwm			compile, build & install the 'jwm' package
 pkg -la			delete all downloaded packages without asking!
 pkg -li vim | pkg -ps -	Get info on all installed Vim pkgs
 pkg -li vim | pkg -wr -	Get repo of an installed Vim pkg 
 pkg -li | pkg -a -u -		Ask to uninstall installed packages one by one 
 pkg --dir2sfs /path/to/dir/	convert a local directory to a .sfs package
 pkg -i /path/to/file.pet	install pet package, include the extension!
 pkg --tgz2pet /path/to/file	convert a local .tar.gz file to PET package
And so on...
Last edited by sc0ttman on Fri 08 Sep 2017, 22:23, edited 2 times in total.
[b][url=https://bit.ly/2KjtxoD]Pkg[/url], [url=https://bit.ly/2U6dzxV]mdsh[/url], [url=https://bit.ly/2G49OE8]Woofy[/url], [url=http://goo.gl/bzBU1]Akita[/url], [url=http://goo.gl/SO5ug]VLC-GTK[/url], [url=https://tiny.cc/c2hnfz]Search[/url][/b]

User avatar
sc0ttman
Posts: 2812
Joined: Wed 16 Sep 2009, 05:44
Location: UK

#17 Post by sc0ttman »

Another fix that needs to be made:

Change the line ~1094:

Code: Select all

ash *.petbuild
to

Code: Select all

bash *.petbuild
Some .petbuild files in /usr/share/petbuild contains stuff like:

/usr/bin/{file1,file2}

I was using ash to run the petbuild scripts (in order to compile the source),
but I guess bash is required...
[b][url=https://bit.ly/2KjtxoD]Pkg[/url], [url=https://bit.ly/2U6dzxV]mdsh[/url], [url=https://bit.ly/2G49OE8]Woofy[/url], [url=http://goo.gl/bzBU1]Akita[/url], [url=http://goo.gl/SO5ug]VLC-GTK[/url], [url=https://tiny.cc/c2hnfz]Search[/url][/b]

User avatar
sc0ttman
Posts: 2812
Joined: Wed 16 Sep 2009, 05:44
Location: UK

#18 Post by sc0ttman »

musher0, what pup are you testing on? .. Bits of your DISTRO_** files are diff and unfamiliar to me ..

I'd like to test Pkg out on that pup... (and others)..

I will need a good, varied selection of pups to test..

Debian, Ubuntu, Slack, other .. All will have different repos that need supporting in /etc/pkg/sources-all

Some may have different ~/.packages/* files .. Will need to get a look at all the options!
[b][url=https://bit.ly/2KjtxoD]Pkg[/url], [url=https://bit.ly/2U6dzxV]mdsh[/url], [url=https://bit.ly/2G49OE8]Woofy[/url], [url=http://goo.gl/bzBU1]Akita[/url], [url=http://goo.gl/SO5ug]VLC-GTK[/url], [url=https://tiny.cc/c2hnfz]Search[/url][/b]

musher0
Posts: 14629
Joined: Mon 05 Jan 2009, 00:54
Location: Gatineau (Qc), Canada

#19 Post by musher0 »

Hi scotman.

I'm on this Pup:
https://augras.eu/documents/ws-default/ ... Os_de_base
Five months old already!

DpupStretch-7fr, despite the "fr" mention, is a bilingual French-English Pup.
I haven't tested all of it in English, but the LANG variable should default to
the language you choose on 1st run. I haven't removed anything, only
added the French locales. Should you need any translation, just holler!

(Contrary to the pups by petihar [Triton] and Médor [Slaxen] who remove
as much as possible of the English to try to provide French-only Pups.)

You have a choice of three kernels for DPupStretch-7fr on that download
page: the last number in the iso file name being the kernel version.

You may end up on 1st run with the wmx window manager, I can't
remember off-hand. If you find wmx confusing, do < xwin jwm > at the
initial black console, jwm is still there and can still be used.

Have fun!

~~~~~~~~~~~~~~

Thanks for the mnemonics lesson! I'll try, but I think I'll still need the
separate less panel in a corner for a few days while I'm getting used to
your "pkg". :)

~~~~~~~~~~~~~~

Will do the edit on line ~ 1094.

BFN.
musher0
~~~~~~~~~~
"You want it darker? We kill the flame." (L. Cohen)

User avatar
sc0ttman
Posts: 2812
Joined: Wed 16 Sep 2009, 05:44
Location: UK

#20 Post by sc0ttman »

Pkg needs a cleaner way to get

pkg-name # vlc-1.3.4abc-i486-s700, vlc-nogui-2.3.4-noarch
pkg-ver # 1.3.4abc
pkg-name-only # vlc , mplayer, gtk+2,
pkg-arch # i486
pkg-build-number # 1 .. may be null
pkg-suffix # s700 .. may be null

It first involves getting the version in a cleaner way,
(one func to do the job, rather than snippets around the Pkg code)..

from split.sh (taken from new2dir)

Code: Select all

#difficult task, separate package name from version part... 
#not perfect, some start with non-numeric version info...
xNAMEONLY="`echo -n "$xPKGDIR" | sed -e 's/\-[0-9].*$//g'`"
#...if that fails, do it the old way...
[ "$xNAMEONLY" = "$xPKGDIR" ] && xNAMEONLY="`echo "$xPKGDIR" | cut -f 1 -d "-"`"
Last edited by sc0ttman on Sun 10 Sep 2017, 13:04, edited 1 time in total.
[b][url=https://bit.ly/2KjtxoD]Pkg[/url], [url=https://bit.ly/2U6dzxV]mdsh[/url], [url=https://bit.ly/2G49OE8]Woofy[/url], [url=http://goo.gl/bzBU1]Akita[/url], [url=http://goo.gl/SO5ug]VLC-GTK[/url], [url=https://tiny.cc/c2hnfz]Search[/url][/b]

User avatar
sc0ttman
Posts: 2812
Joined: Wed 16 Sep 2009, 05:44
Location: UK

#21 Post by sc0ttman »

musher0 wrote:I'm on this Pup:
https://augras.eu/documents/ws-default/ ... Os_de_base
Five months old already!
Great, cheers..

I will try it out over the next few days.. I'm guessing Pkg doesn't work very well at all on that Pup,
if the slightly different ~/.packages/ files are anything to go by :shock: lol

Anyway, thanks for testing it out, I'm sure it's more a pain than a benefit to be the first!
[b][url=https://bit.ly/2KjtxoD]Pkg[/url], [url=https://bit.ly/2U6dzxV]mdsh[/url], [url=https://bit.ly/2G49OE8]Woofy[/url], [url=http://goo.gl/bzBU1]Akita[/url], [url=http://goo.gl/SO5ug]VLC-GTK[/url], [url=https://tiny.cc/c2hnfz]Search[/url][/b]

musher0
Posts: 14629
Joined: Mon 05 Jan 2009, 00:54
Location: Gatineau (Qc), Canada

#22 Post by musher0 »

Hi scotman!

On the other hand, someone has to do it! ;) And I have the motivation --
because PPM has let me down too many times! :twisted: (Let's call it
frustration rather than revenge !!!)

I built a new DPupStretch-7 yesterday evening and will test your "pkg" on
this fresh build as well.

BFN.
musher0
~~~~~~~~~~
"You want it darker? We kill the flame." (L. Cohen)

User avatar
sc0ttman
Posts: 2812
Joined: Wed 16 Sep 2009, 05:44
Location: UK

#23 Post by sc0ttman »

I built a new DPupStretch-7 yesterday evening and will test your "pkg" on
this fresh build as well.
Great, thanks, I think i might test on that one..

Well, I really don't think Pkg will prove more reliable than PetGet ...
But I am sure large parts of it (dep resolution, for example) can be simplified, reduced and improved all at the same time.
Over the next few weeks, I will probably steal lots of code from PetGet..

But really, thanks for testing, I appreciate it.
Hopefully it will iron out some silly stuff early on ..

----------------

Using Git Repos as package sources

Any suggestions on hwo to `download package source from a Git repo`?

How to get the package name? And package version?

Arch Linux can do this in its build-from-source system, according to some Pkgfiles...

Can PetBuild do this? .. I can't see anything in its code..
Last edited by sc0ttman on Sun 10 Sep 2017, 13:16, edited 4 times in total.
[b][url=https://bit.ly/2KjtxoD]Pkg[/url], [url=https://bit.ly/2U6dzxV]mdsh[/url], [url=https://bit.ly/2G49OE8]Woofy[/url], [url=http://goo.gl/bzBU1]Akita[/url], [url=http://goo.gl/SO5ug]VLC-GTK[/url], [url=https://tiny.cc/c2hnfz]Search[/url][/b]

User avatar
rockedge
Posts: 1864
Joined: Wed 11 Apr 2012, 13:32
Location: Connecticut, United States
Contact:

#24 Post by rockedge »

Using Pkg-1.9.0alpha-noarch
I installed the PET and every attempt to run it on a woof-ce recently produced xenial type pup is met with this error:

Code: Select all

root# pkg -h 
Warning: Invalid work dir in ~/.pkg/pkgrc, using ~/pkg
/usr/sbin/pkg: line 542: syntax error: unexpected "(" (expecting "}")
root# source ~/.bashrc 
root# pkg --update-sources 
/usr/sbin/pkg: line 542: syntax error: unexpected "(" (expecting "}")
root# pkg --repos
/usr/sbin/pkg: line 542: syntax error: unexpected "(" (expecting "}")
root# 
system ->
DISTRO_NAME='xenialpup'
DISTRO_VERSION=7.0.6
DISTRO_BINARY_COMPAT='ubuntu'
DISTRO_FILE_PREFIX='xenial'
DISTRO_COMPAT_VERSION='xenial'
DISTRO_XORG_AUTO='yes'
DISTRO_KERNEL_PET='Huge_Kernel'
DISTRO_DB_SUBNAME='xenial'
DISTRO_ARCHDIR='i386-linux-gnu'
DISTRO_TARGETARCH='x86'
BUILD_FROM_WOOF='testing;8ff16a9;2017-08-19 22:20:17 +0800'

User avatar
sc0ttman
Posts: 2812
Joined: Wed 16 Sep 2009, 05:44
Location: UK

#25 Post by sc0ttman »

Nice :oops:

Cheers, will get a Xenial Pup and have a look...

EDIT: Was on Windows... Now booted X-Tahr 6.0.2 ...

I checked and it was a 'function' keyword.. Remove the word 'function'.
Tahr (and Xenial) ash don't support it, Slackos does..

Tahr-6.0.2 is using: BusyBox v1.21.0 (2013-02-18 15:57:06 WST)
Slacko-6.9.x is using: BusyBox v1.25.0 (2016-07-24 11:33:13 GMT-8)

What busybox is your Xenial using?

In the mean time... I have made some Debian-Pup fixes to Pkg..

The bash TAB auto completion didn't work, at first..
Had to manually do this:

Code: Select all

bash
. /etc/bash_completion.d/pkg
pkg -h
exit
then re-open terminal...

Got Pkg working with TAB completion, changing repos,
listing package contents, etc..

Then started installing stuff from the puppy-tahr
and common repos easy enough... They use PETs though, easy..

The next problem I encountered was when trying to install 'snappy' from
the trusty-main repo..

Had to add a trailing slash to the puppy-tahr URLs ~/pkg/sources-all
(also made same changes in /etc/pkg/ to keep them up to date):

Code: Select all

puppy-tahr|pet|Packages-puppy-tahr-official|http://distro.ibiblio.org/puppylinux/pet_packages-tahr/||||noarch common trusty-main trusty-universe akita wary51x quirky 

trusty-main|deb|Packages-ubuntu-trusty-main|http://ftp.filearena.net/pub/ubuntu/ubuntu/||||puppy-tahr trusty-universe noarch common akita wary51x

trusty-universe|deb|Packages-ubuntu-trusty-universe|http://ftp.filearena.net/pub/ubuntu/ubuntu/||||puppy-tahr trusty-main noarch common akita wary51x
Then I updated the list of avail repos, switched to trusty-main repo:

Code: Select all

pkg --update-sources
pkg -r trusty-main
Then I downloaded snappy and its deps...

Another problem after downloading snappy and deps...

Pkg did not extract the all files in the .debs into the filesystem...
Yet reported no errors..

Will have to investigate later.. Maybe some DEBIAN post-install stuff was not executed?

Pkg uses dpkg, and much of the same code as installpkg.sh, I will have
look at how /usr/local/petget/installpkg.sh handles it a bit more..

..............

BUG:

After the DEB failed to install, I tried converting the libclutter-gst-*
DEB package to a PET:

# pkg --install libclutter-gst-1.0-0_1.6.0-2build1_i386.pet
Package 'libclutter-gst-1.0-0_1.6.0-2build1_i386' not yet downloaded.
You could install one of the following packages with `pkg -i PKGNAME`:
libclutter-gst-1.0-0_1.6.0-2build1_i386-bkp.deb
libclutter-gst-1.0-0_1.6.0-2build1_i386.pet
#
# rm ~/pkg/libclutter-gst-1.0-0_1.6.0-2build1_i386.pet
#
# pkg -i libclutter-gst-1.0-0_1.6.0
Install the package: libclutter-gst-1.0-0_1.6.0-2build1_i386:
Success: libclutter-gst-1.0-0_1.6.0-2build1_i386 installed.


POSSIBLE CAUSES:
- repo ext type of tahr-* is DEB not PET
- having a PET and DEB in ~/pkg/ may be causing issues

TO FIX:
- install could (should?) ignore the repo $EXT completely
- make sure the backup of PKG gets removed if conversion successful
- Remove all but the most essential does-file-exist checks in install()


..............

Another note:

Here's a fix to go in the install() func, which fixes binaries built using
--enable-default-pie not working..

Discussion here: http://murga-linux.com/puppy/viewtopic. ... 786#946786

Fix is here: https://github.com/puppylinux-woof-CE/w ... ac6ebf1d09

----------------

Pkg was missing the trusty-multiverse repo in its database, for some reason..

I replaced the trusty repo entries with the following in ~/.pkg/sources-all and /etc/pkg/sources-all:

Code: Select all

puppy-tahr|pet|Packages-puppy-tahr-official|http://distro.ibiblio.org/puppylinux/pet_packages-tahr/||||noarch common trusty-main trusty-universe trusty-multiverse akita wary51x quirky 

trusty-main|deb|Packages-ubuntu-trusty-main|http://ftp.filearena.net/pub/ubuntu/ubuntu/||||puppy-tahr trusty-universe trusty-multiverse noarch common akita wary51x

trusty-universe|deb|Packages-ubuntu-trusty-universe|http://ftp.filearena.net/pub/ubuntu/ubuntu/||||puppy-tahr trusty-main trusty-multiverse noarch common akita wary51x

trusty-multiverse|deb|Packages-ubuntu-trusty-multiverse|http://ftp.filearena.net/pub/ubuntu/ubuntu/||||puppy-tahr trusty-main trusty-universe noarch common akita wary51x
Then I did `pkg --update-sources`... Then I tested it:

Code: Select all

# pkg -g gnuboy-x
Checking 'trusty-multiverse' repo mirrors...
Download from trusty-multiverse repo: gnuboy-x_1.0.3-7_i386.deb:  
Using URL:  http://ftp.filearena.net/pub/ubuntu/ubuntu/pool/multiverse/g/gnuboy/
Downloading gnuboy-x_1.0.3-7_i386.deb. Please wait: 100%
Install the package: gnuboy-x_1.0.3-7_i386:  
Success: gnuboy-x_1.0.3-7_i386 installed.
#
# pkg -c gnuboy-x_1.0.3-7_i386
/usr/share/doc/gnuboy-x/changelog.Debian.gz
/usr/share/doc/gnuboy-x/CREDITS
/usr/share/doc/gnuboy-x/LIBERTY
/usr/share/doc/gnuboy-x/FAQ
/usr/share/doc/gnuboy-x/HACKING.gz
/usr/share/doc/gnuboy-x/copyright
/usr/share/doc/gnuboy-x/examples/classic.rc
/usr/share/doc/gnuboy-x/examples/sample.rc
/usr/share/doc/gnuboy-x/examples/palettes.rc
/usr/share/doc/gnuboy-x/examples/filters.rc
/usr/share/doc/gnuboy-x/CONFIG.gz
/usr/share/man/man6/xgnuboy.6.gz
/usr/games/xgnuboy
#
# xgnuboy --help

gnuboy 1.0.3
Usage: xgnuboy [options] romfile

      --source FILE             read rc commands from FILE
      --bind KEY COMMAND        bind KEY to perform COMMAND
      --VAR=VALUE               set rc variable VAR to VALUE
      --VAR                     set VAR to 1 (turn on boolean options)
      --no-VAR                  set VAR to 0 (turn off boolean options)
      --showvars                list all available rc variables
      --help                    display this help and exit
      --version                 output version information and exit
      --copying                 show copying permissions
Last edited by sc0ttman on Sun 10 Sep 2017, 10:30, edited 4 times in total.
[b][url=https://bit.ly/2KjtxoD]Pkg[/url], [url=https://bit.ly/2U6dzxV]mdsh[/url], [url=https://bit.ly/2G49OE8]Woofy[/url], [url=http://goo.gl/bzBU1]Akita[/url], [url=http://goo.gl/SO5ug]VLC-GTK[/url], [url=https://tiny.cc/c2hnfz]Search[/url][/b]

User avatar
sc0ttman
Posts: 2812
Joined: Wed 16 Sep 2009, 05:44
Location: UK

#26 Post by sc0ttman »

# pkg -ps dvdrip
Please wait.. Gathering dependency information.

Name: dvdrip_0.98.11
Description: None
Size: 2634K
Status: not installed
In Repo: trusty-multiverse
Repo file: Packages-ubuntu-trusty-multiverse
Installed deps: perl-base_5.18.2
Missing deps: dvdrip-utils_0.98.11, eject-2.1.0, fping_3.8-1,
imagemagick-common_6.7.7.10, libav-tools_9.18,
libevent-execflow-perl_0.64, libevent-perl_1.21,
libevent-rpc-perl_1.04-2, libgtk2-ex-formfactory-perl_0.67,
libintl-perl_1.23, liblocale-gettext-perl_1.05,
lsdvd_0.16-4, ogmtools_1.5, transcode-doc_1.1.7-8

# ^ps^ed
pkg -ed dvdrip
Resolving dependencies...
..blah...
Checking 'trusty-universe' repo mirrors...
Download from trusty-universe repo: libav-tools_9.18-0ubuntu0.14.04.1_i386.deb:
Using URL: http://ftp.filearena.net/pub/ubuntu/ubu ... iba/libav/
Downloading libav-tools_9.18-0ubuntu0.14.04.1_i386.deb. Please wait:
Error: /root/pkg/libav-tools_9.18-0ubuntu0.14.04.1_i386.deb failed to download.
Check 'http://ftp.filearena.net/pub/ubuntu/ubu ... 1_i386.deb'


Pkg wanted

libav-tools_9.18-0ubuntu0.14.04.1_i386.deb

at that mirror, I can only find:

http://ftp.filearena.net/pub/ubuntu/ubu ... ls_9.[b]20[/b]-0ubuntu0.14.04.1_i386.deb

For some reason, --repo-update doesn't work like it does on Slacko,
needs fixing .. Hopefully that will fix the missing libav-tools (which I doubt!)

In order to properly test Pkg, I generally don't run PetGet, I steal code from it instead.
Last edited by sc0ttman on Sun 10 Sep 2017, 13:09, edited 2 times in total.
[b][url=https://bit.ly/2KjtxoD]Pkg[/url], [url=https://bit.ly/2U6dzxV]mdsh[/url], [url=https://bit.ly/2G49OE8]Woofy[/url], [url=http://goo.gl/bzBU1]Akita[/url], [url=http://goo.gl/SO5ug]VLC-GTK[/url], [url=https://tiny.cc/c2hnfz]Search[/url][/b]

musher0
Posts: 14629
Joined: Mon 05 Jan 2009, 00:54
Location: Gatineau (Qc), Canada

#27 Post by musher0 »

Wooo, that's a lot to absorb, there, scotman!

But "ground-level" examples like that are nevertheless quite appreciated.

Thanks.
musher0
~~~~~~~~~~
"You want it darker? We kill the flame." (L. Cohen)

User avatar
sc0ttman
Posts: 2812
Joined: Wed 16 Sep 2009, 05:44
Location: UK

#28 Post by sc0ttman »

I will go back over my posts in this thread, clean it all up ..
Leaving (mostly) only clear, concise suggestions...

I won't be posting more Pkg fixes/code here..

I will get Pkg working in Xenial/Tahr/Deb/Ubu pups, then I will
refactor the code a bit, make the internal funcs available as a lib..
This will make a new CLI available, which is more similar to Slackwares pkgtools,
faster and makes Pkg more programmable (and more testable :) ).

Making Pkg available 'as a lib' means you will be able to do *something* like:

Code: Select all

source /usr/lib/pkg/funcs # make Pkg funcs avail on command line
pkgsearch XYZ
pkgdownload PKG 
pkgdepends PKG 
pkginfo PKG  # ... and many more
Then I will update to pkg-1.9.1alpha, and put it in the PETs section..

I will post it along with a big list of bugs to be fixed before it leaves alpha.

In the mean time, please do post suggestions to this thread about:

- improving command line tools in puppy
- candidates for "separation of GUI and CLI", such as: pup-advert-blocker, puppyinstaller, sns, others?
- de-coupling puppy apps from X
- how to improve command line option handling (links to good recipes, ideas, snippets, etc)
- how to improve error code handling (links to good shell tutorials)
- what you want from a CLI package manager
- how to go about making a web-based package manager frontend
- how to best fork and edit the Salix package search site, then where to host
- desired features/workflow/options for Pkg
- suggest command/features you like in other pkg managers that you want in puppy
- etc
[b][url=https://bit.ly/2KjtxoD]Pkg[/url], [url=https://bit.ly/2U6dzxV]mdsh[/url], [url=https://bit.ly/2G49OE8]Woofy[/url], [url=http://goo.gl/bzBU1]Akita[/url], [url=http://goo.gl/SO5ug]VLC-GTK[/url], [url=https://tiny.cc/c2hnfz]Search[/url][/b]

User avatar
sc0ttman
Posts: 2812
Joined: Wed 16 Sep 2009, 05:44
Location: UK

#29 Post by sc0ttman »

Suggestion:

Better automated sanity checking of packages built by: dir2pet, new2dir, splitpkg, pkg

Petbuild should (might already?) do this.

Note to self: Pkg should do its own checks,
regardless of whether or not petbuild does/did it..

From Arch Linux: https://wiki.archlinux.org/index.php/Creating_packages
Checking package sanity

After testing package functionality check it for errors using namcap:

$ namcap PKGBUILD
$ namcap <package file name>.pkg.tar.xz

Namcap will:

Check PKGBUILD contents for common errors and package file hierarchy for unnecessary/misplaced files
Scan all ELF files in package using ldd, automatically reporting which packages with required shared libraries are missing from depends and which can be omitted as transitive dependencies
Heuristically search for missing and redundant dependencies

and much more.

Get into the habit of checking your packages with namcap to avoid having to fix the simplest mistakes after package submission.
Last edited by sc0ttman on Sun 10 Sep 2017, 17:45, edited 1 time in total.
[b][url=https://bit.ly/2KjtxoD]Pkg[/url], [url=https://bit.ly/2U6dzxV]mdsh[/url], [url=https://bit.ly/2G49OE8]Woofy[/url], [url=http://goo.gl/bzBU1]Akita[/url], [url=http://goo.gl/SO5ug]VLC-GTK[/url], [url=https://tiny.cc/c2hnfz]Search[/url][/b]

User avatar
sc0ttman
Posts: 2812
Joined: Wed 16 Sep 2009, 05:44
Location: UK

#30 Post by sc0ttman »

Custom pkgs could be maintained easily as petbuilds.

For example. ttuuxxx recently hacked the ./configure script of mplayer,
to make it compile for his Debian Stretch puppy build to make
an mplayer PET less than 2mb.

A petbuild could be maintained (and easily used by woof and end users),
called mplayer_ttuuxxx-1.2.3.petbuild which contained
his custom changes.

Everyone knows ttuuxxx compiles tiny, lightweight PETs with the fewest deps
possible.. If I saw a PET or buildscript available from him, I'd go for it over an
official DEB or TXZ any day..

A range of apps could be maintained in this way by different users,
and woof builders could choose which packages they want to build.

Using petbuilds in this way might greatly reduce the amount of lost effort,
while greatly increasing the repeatability/reproducibility of great work.

This is also a pretty simple and clear system, and when compared to a ports system,
where users maintain different build script repos, this way avoids the complexity of
having to compare and check build scripts of the same name from different sources.

From the CRUX Linux prt-get QuickStart:
2.1. Port directory listings and order

In order to build a stable and consistent system, it's important that ports are tested against each other. Because of that, prt-get requires you to specify the directories where prt-get should look for package, and the order of these directories are used to choose a port; if more than one port exists for a certain package, the first matching port is used.
[b][url=https://bit.ly/2KjtxoD]Pkg[/url], [url=https://bit.ly/2U6dzxV]mdsh[/url], [url=https://bit.ly/2G49OE8]Woofy[/url], [url=http://goo.gl/bzBU1]Akita[/url], [url=http://goo.gl/SO5ug]VLC-GTK[/url], [url=https://tiny.cc/c2hnfz]Search[/url][/b]

wiak
Posts: 2040
Joined: Tue 11 Dec 2007, 05:12
Location: not Bulgaria

#31 Post by wiak »

Hi sc0ttman,

I presume you are still working on this? Does it handle package dependencies (via pet specs or whatever?).

Really needing a good package manager for automating Pup builds in similar way to what mklive-stretch DebianDog does on top of debootstrap using apt-get. (In Puppy would be commandline package manager additions after initial woof-CE minimal build - chroot into sandbox3/rootfs-complete and then package manager adds into that prior to squashfs and iso making).

wiak

User avatar
sc0ttman
Posts: 2812
Joined: Wed 16 Sep 2009, 05:44
Location: UK

#32 Post by sc0ttman »

wiak wrote:Hi sc0ttman,

I presume you are still working on this?
Very much so, doing a big clean up and a refactor.. making the code more testable, reliable, etc..
wiak wrote:Does it handle package dependencies (via pet specs or whatever?)
Yes it does, works pretty well .. Pkg can install VLC from Slacko 14.2 repo, plus deps ,
which requires spanning multiple repos, some with packages in subdirs,
some not, some deps are PETs, some are .txz... but Pkg can handle it ;)

But.. I am gonna rip out the dep resolution stuff and adapt PetGets code,
so that both package managers *always* return the same dep results..
(Pkg sometimes pulls in too many, and netsurf-3.3 (slackware-extra repo) makes
Pkg get stuck in a loop!!! :oops:
wiak wrote:Really needing a good package manager for automating Pup builds in similar way to what mklive-stretch DebianDog does on top of debootstrap using apt-get. (In Puppy would be commandline package manager additions after initial woof-CE minimal build - chroot into sandbox3/rootfs-complete and then package manager adds into that prior to squashfs and iso making).

wiak
After more fixes and refactors etc, when it is safer to add new (testable) features,
I will work on being able to install into a different rootfs, so Pkg can 'provision' other
filesystems..

DESTDIR=/path/to/my/rootfs pkg [opts] PKGNAME

or

pkg --root=/path/to/my/rootfs [opts] PKGNAME

Alpine Package Manager can do this .. It's a nice feature.. I've made a list of all the nicest
features from other CLI package managers, plus a few I dreamed up ..

But I'm bugfixing and making it more reliable and easier to work on for now ..
The cleaner the code, the easier it will be to 'do it right' going into the future..
[b][url=https://bit.ly/2KjtxoD]Pkg[/url], [url=https://bit.ly/2U6dzxV]mdsh[/url], [url=https://bit.ly/2G49OE8]Woofy[/url], [url=http://goo.gl/bzBU1]Akita[/url], [url=http://goo.gl/SO5ug]VLC-GTK[/url], [url=https://tiny.cc/c2hnfz]Search[/url][/b]

wiak
Posts: 2040
Joined: Tue 11 Dec 2007, 05:12
Location: not Bulgaria

#33 Post by wiak »

sc0ttman wrote: But I'm bugfixing and making it more reliable and easier to work on for now ..
The cleaner the code, the easier it will be to 'do it right' going into the future..
Absolutely. Too much code released in a hurry just cos it works (despite being hacked together and added to without careful modularisation etc). Well-designed clean code provides as you say for the future in terms of maintainability and enhancement. Development is always faster in the long (if not short) term if the programmer concentrates on getting the core code optimally designed.

wiak

User avatar
sc0ttman
Posts: 2812
Joined: Wed 16 Sep 2009, 05:44
Location: UK

#34 Post by sc0ttman »

wiak wrote:
sc0ttman wrote: But I'm bugfixing and making it more reliable and easier to work on for now ..
The cleaner the code, the easier it will be to 'do it right' going into the future..
Well-designed clean code provides as you say for the future in terms of maintainability and enhancement.
I wouldn't ever call my code "well-designed" or "clean" lol, but I'm trying..

But I am going down the script, top to bottom (5000 lines of code), and I am
cleaning up each func (~ 70 of them at the mo), and :

* make sure each func only does 1 job, if possible
* making each func as small, readable as concise as possible
* good, simple, informative comments
* refactoring big funcs into smaller funcs
* simplifying any logic, removing any loops that aren't 100% necessary, etc
* replacing any repetitive routines/code with a single func for the job

I've fixed a bunch of things so far..

Pkg now reports exactly the same deps for pkgs as PetGet does (didn't have to steal PetGets code in the end)

Progress is decent .. Speed is OK, reliability is generally good ..
List of features is huge ;)


I see Pkg often being used (by me at least) not just/only/even as a package manager (its main purpose) but as a dev tool, for easy making
of PETs etc .. Lots of useful command for Puppy users who make their own PETs and SFS files:

--download PKGNAME .... easier way to download PETs from any supported online sources...

--dir2sfs DIR ... make an SFS easily
--dir2pet DIR ..... make a PET
--pkg-contents PKGNAME .... get contents (file list) of any pkg
--pkg-repack PKGNAME .... (re)create a PET package from any installed packages
--pkg-combine PKGNAME ... auto-combine a pkg with its deps into a 'pkg-.1.2.3_ALL.pet'
--sfs-combine PKGNAME ... auto-combine a package and its deps into an SFS :)


Plus some useful debugging cmds:

--which FILE ... give file, find out which pkg it comes from
--which-repo PKGNAME ... give a pkg name, its repo its returned (if found in any supported repos)

.. and lots of PET conversion funcs ...
...many more...


But... I've received literally zero suggestions for "must have" or "nice to have" features,
other than yourself asking for Pkg to work inside Woof (a good one!)..
So I'll just make what I wanna make, see if anyone bites..
[b][url=https://bit.ly/2KjtxoD]Pkg[/url], [url=https://bit.ly/2U6dzxV]mdsh[/url], [url=https://bit.ly/2G49OE8]Woofy[/url], [url=http://goo.gl/bzBU1]Akita[/url], [url=http://goo.gl/SO5ug]VLC-GTK[/url], [url=https://tiny.cc/c2hnfz]Search[/url][/b]

wiak
Posts: 2040
Joined: Tue 11 Dec 2007, 05:12
Location: not Bulgaria

#35 Post by wiak »

sc0ttman wrote: But... I've received literally zero suggestions for "must have" or "nice to have" features,
other than yourself asking for Pkg to work inside Woof (a good one!)..
So I'll just make what I wanna make, see if anyone bites..
Unfortunately, I've been so busy coding makepup I've not had time to check your package manager out yet. But I recognise the importance of getting a good/versatile commandline-based package manager for Puppy development, hence my encouraging you to progress it. Sometimes it is best to make a program the way that suits you best anyway, and once it is polished up a bit and stable, that's when further ideas will probably come in from users. Main thing is that it can be easily called up from other scripts, have flexible access to package repositories, and deal with dependencies as reliably as possible I feel.

I'll certainly be giving it a spin once I finally take a rest from makepup (only a few tiny extras I want to add to that for the moment - sometimes it is good to sit back and ponder for a bit before deciding on more...).

wiak

Post Reply