Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Fri 06 Dec 2019, 20:28
All times are UTC - 4
 Forum index » Advanced Topics » Cutting edge
s243a's fork of woof-next
Moderators: Flash, Ian, JohnMurga
Post new topic   Reply to topic View previous topic :: View next topic
Page 1 of 2 [19 Posts]   Goto page: 1, 2 Next
Author Message
s243a

Joined: 02 Sep 2014
Posts: 2188

PostPosted: Fri 16 Aug 2019, 12:51    Post subject:  s243a's fork of woof-next  

I forked woof next prior to the woof-CE team decided that it was a good idea. This caused a small amount of controversy since much of this discussion about this by the woof-CE team was via private messages.

My primary differences in philosophy include:
1. allow fall-backs for build install methods
2. modularize the roots-skeleton
3. better legacy support (my claim on point #3 seems to have offended one of the woof-CE developers).

I would like to go into implementation details about these differences but I would like to more focus on what needs to be done going forward.

_________________
Find me on minds and on pearltrees.
Back to top
View user's profile Send private message Visit poster's website 
s243a

Joined: 02 Sep 2014
Posts: 2188

PostPosted: Fri 16 Aug 2019, 12:51    Post subject:  

reserved
_________________
Find me on minds and on pearltrees.
Back to top
View user's profile Send private message Visit poster's website 
s243a

Joined: 02 Sep 2014
Posts: 2188

PostPosted: Fri 16 Aug 2019, 12:51    Post subject:  

reserved
_________________
Find me on minds and on pearltrees.
Back to top
View user's profile Send private message Visit poster's website 
s243a

Joined: 02 Sep 2014
Posts: 2188

PostPosted: Fri 16 Aug 2019, 12:51    Post subject:  

reserved
_________________
Find me on minds and on pearltrees.
Back to top
View user's profile Send private message Visit poster's website 
s243a

Joined: 02 Sep 2014
Posts: 2188

PostPosted: Fri 16 Aug 2019, 12:52    Post subject:  

reserved
_________________
Find me on minds and on pearltrees.
Back to top
View user's profile Send private message Visit poster's website 
s243a

Joined: 02 Sep 2014
Posts: 2188

PostPosted: Fri 16 Aug 2019, 13:21    Post subject:  

The first issue that I need to address is a minor issue but an important one.

On boot I get the errr:
Code:

/init: line 979: which : not found
WARNING: 'e2fsck' command not found



The line number is completely wrong. I think the following is the offending code:
Code:

 which "${fsck_app}" || {
   echo "WARNING: '${fsck_app}' not found"
   FSCKDPARTS="${FSCKDPARTS}${1}|"
   return
}

/s243a/woof-CE/.../initrd-progs/0initrd/init#L118

The which utility is part of busybox. So the issue here is likely related to the creating of the symlinks but this will only work if busybox is compiled with said applet. Woof-next use to get the architecture specific files from the following two locations:
Code:

INITRD_ARCH=${INITRD_ARCH:-$WOOFCE/woof-arch/x86/target/boot/initrd-tree0}

/s243a/woof-CE/.../builders/build-iso.sh#L42

Code:

cp -a $INITRD_ARCH/* $INITRD_CODE/* $initrdtmp

/s243a/woof-CE/.../builders/build-iso.sh#L124

Here's an example of what architecture specific files are included in woof-next
/s243a/woof-CE/tree/woof-next/woof-arch/x86/target/boot/initrd-tree0/bin

Anyway, while this error appears minor sometimes fixing what appears to be minor error in the init systems can fix thing which one wouldn't expect to be related.

So at this point I'm wondering what changes related from the zwn branch should be included to fix this. I don't think that in the zwn branch anything comes up related to this change as all changes to the build-iso.sh file appear to be in the initial commit. However, if I follow jamesbond's changes since my fork I might be able to find some record of this on github.

I will note that this file has been significantly reduced in size when removed to the zwn branch. The variable INITRD_ARCH seems to no longer exist and I think the architecture specific components might be now generated dynamically rather than prebuilt:
Code:

./build.sh -prebuilt -auto -arch ${WOOF_TARGETARCH:-default} ${INITRD_LANG} ${INITRD_KM}

/puppylinux-woof-CE/woof-CE/.../zwn/builders/build-iso.sh#L81

And if my hunch is right, I don't know how much work will be required to incorporate this new dynamic approach into my fork but it looks like a good idea.

_________________
Find me on minds and on pearltrees.
Back to top
View user's profile Send private message Visit poster's website 
s243a

Joined: 02 Sep 2014
Posts: 2188

PostPosted: Fri 16 Aug 2019, 15:33    Post subject:  

The next issue that I wanted to look at is related to my fork of sc0tmann's package. I'ave included sc0tmann's package as part of my woof-next fork so that I could use it as part of the build system. In contrast; the woof-CE provided an alternate way to add pet packages in the ZWN branch:

wdlkmpx wrote:

Well, i guess, i guess, i guess... (reinserting disc)

After commit 9724cf5 zwn can download and "install" pet packages to the chroot dir

Here is an example
734ce98

And i guess i'll just abandon the idea, there's many things to do


For consistancy I may copy this functionality but It kinds of goes against my mental roadmap. My version is to create a generic builder that can work with (or emluate) the functionality of many package managers rather than having seperate builders for each type of distribution (i.e. slackware and debian).

Anway, back to the issue at hand. I noticed that in package that if a package is already installed than in will be automatically deleted from the /root/pkg. This might cause some confusion if one wants to first download a package and then re-isntall it. There is an enviornmental varialbe that one can set as a workaround but this is counter-instuative.

The offending code is:
Code:

# if AUTOCLEAN=true silently delete all pkgs in WORKDIR that are already installed
if [ "$AUTOCLEAN" = 'yes' -a "`HIDE_BUILTINS=true list_installed_pkgs`" != '' ]; then
  clean_pkgs &>/dev/null
fi

/s243a/woof-CE/.../woof-code/rootfs-packages/PKG/usr/sbin/pkg#L7720
/sc0ttj/Pkg/.../usr/sbin/pkg#L7603


I will make a fix for this shortly.

**Note that I'm discussing my fork of pkg so any comments may or may not apply to sc0ttman's version.

_________________
Find me on minds and on pearltrees.
Back to top
View user's profile Send private message Visit poster's website 
s243a

Joined: 02 Sep 2014
Posts: 2188

PostPosted: Fri 16 Aug 2019, 23:42    Post subject:  

s243a wrote:
The next issue that I wanted to look at is related to my fork of sc0tmann's package. I'ave included sc0tmann's package as part of my woof-next fork so that I could use it as part of the build system. In contrast; the woof-CE provided an alternate way to add pet packages in the ZWN branch:

wdlkmpx wrote:

Well, i guess, i guess, i guess... (reinserting disc)

After commit 9724cf5 zwn can download and "install" pet packages to the chroot dir

Here is an example
734ce98

And i guess i'll just abandon the idea, there's many things to do


For consistancy I may copy this functionality but It kinds of goes against my mental roadmap. My version is to create a generic builder that can work with (or emluate) the functionality of many package managers rather than having seperate builders for each type of distribution (i.e. slackware and debian).

Anway, back to the issue at hand. I noticed that in package that if a package is already installed than in will be automatically deleted from the /root/pkg. This might cause some confusion if one wants to first download a package and then re-isntall it. There is an enviornmental varialbe that one can set as a workaround but this is counter-instuative.

The offending code is:
Code:

# if AUTOCLEAN=true silently delete all pkgs in WORKDIR that are already installed
if [ "$AUTOCLEAN" = 'yes' -a "`HIDE_BUILTINS=true list_installed_pkgs`" != '' ]; then
  clean_pkgs &>/dev/null
fi

/s243a/woof-CE/.../woof-code/rootfs-packages/PKG/usr/sbin/pkg#L7720
/sc0ttj/Pkg/.../usr/sbin/pkg#L7603


I will make a fix for this shortly.

**Note that I'm discussing my fork of pkg so any comments may or may not apply to sc0ttman's version.


Here is my fix (see commit e518085 )

I add an override to the autoclean when the --download option is used:
Code:

      --download|-d|download|d)
        AUTOCLEAN_override='no' 
        [ ! "$2" ] && pkg_download || for x in $opt; do [ "$x" -a "$x" != "-" ] && pkg_download "$x";  done;;

/s243a/woof-CE/.../woof-code/rootfs-packages/PKG/usr/sbin/pkg#L7592

and then it is used in the following place:
Code:

[ ! -z "$AUTOCLEAN_override" ] && AUTOCLEAN="$AUTOCLEAN_override"
# if AUTOCLEAN=true silently delete all pkgs in WORKDIR that are already installed
if [ "$AUTOCLEAN" = 'yes' -a "`HIDE_BUILTINS=true list_installed_pkgs`" != '' ];then
  clean_pkgs &>/dev/null
fi

/s243a/woof-CE/.../woof-code/rootfs-packages/PKG/usr/sbin/pkg#L7721

To recap, what this does is prevents previously installed packages from being deleted when downloaded via the --download option.

_________________
Find me on minds and on pearltrees.
Back to top
View user's profile Send private message Visit poster's website 
s243a

Joined: 02 Sep 2014
Posts: 2188

PostPosted: Sat 17 Aug 2019, 15:06    Post subject:  

Another issue that I should document is related to the enviornmental variable PERL5LIB, If this variable isn't set then urxvt complains that perl doesn't work.

Curently, I have the following line in /etc/profile:

Code:

export PERL5LIB="/usr/lib/urxvt:/etc/perl:/usr/local/lib/i386-linux-gnu/perl/5.24.1:/usr/local/share/perl/5.24.1:/usr/lib/i386-linux-gnu/perl5/5.24:/usr/share/perl5:/usr/lib/i386-linux-gnu/perl/5.24:/usr/share/perl/5.24:/usr/local/lib/site_perl:/usr/lib/i386-linux-gnu/perl-base"


to address this issue but there is a lot of release specific stuff here. I don't want to hard code all this stuff for each distro. Also the distro arch directories should only be included in the path if they are a unique directory. In some versions of puppy this is just a symlink and as a consequence these directories would not need to be included as part of the PERL5LIB enviornmental variable.

_________________
Find me on minds and on pearltrees.
Back to top
View user's profile Send private message Visit poster's website 
s243a

Joined: 02 Sep 2014
Posts: 2188

PostPosted: Sun 18 Aug 2019, 04:44    Post subject: Is "${PWD}" every wrong?
Subject description: and how our assumptions kill us.
 

Here is something that looks safe
Code:

CURDIR="${PWD}" # get current dir, before cd to WORKDIR

/s243a/woof-CE/.../woof-code/rootfs-packages/PKG/usr/sbin/pkg#L39
but is it? Probably but is PWD inherited by subprocesses or do we need to export it? What if we execute a command like follows:
Code:

urxvt -e pkg -i pkgname

is PWD still inherated by the subprocess if we don't export it? These questions might sound strange but I was working on a script to install a package (via pkg) by clicking on it and the last command of the script worked if I called the command directly from the command line but when the script was invoked by clicking on a .pet file the same command that worked from the command line (last command executed in my script) didn't work even though said command worked when being used directly on the command line.

Here is the script which is called when a .pet file is clicked..
https://pastebin.com/EPgPujar
but
Code:

pkg -i pkgname

did not work when it was called from a script that was invoked by clicking on a .pet file in rox.

WHen I finally figured out how to debug it, I noticed that I got
Code:

CURDIR=/root

rather than the directory in which I clicked on the script. Does this sound right? My solution wasn't to export either CURDIR or PWD since parts of the code seem to assume that the "dot pet" file is in the CURDIR. For example
Code:

tar $taropts "${CURDIR}/${PKGNAME}.tar.${TAREXT}" 2> $TMPDIR/$SELF-cp-errlog

/s243a/woof-CE/../woof-code/rootfs-packages/PKG/usr/sbin/pkg#L4676

This likely occured because sc0ttman borrowed some code from puppies package manager. Since I identified this assumption, I simply made the current directorary the directory where the package is located.

Code:

  if [ -f "$1" ];then
    PKGFILE="$1"
    CURDIR="$(dirname "$(realpath "$1")")"
    PKGNAME=`basename "$1" .$pkg_ext`
  else

/s243a/woof-CE/.../woof-code/rootfs-packages/PKG/usr/sbin/pkg#L4587

The code in the above link doesn't quite match the above code (it's missing the "CURDIR=" line) because I have yet pushed this change to github.

_________________
Find me on minds and on pearltrees.
Back to top
View user's profile Send private message Visit poster's website 
nic007


Joined: 13 Nov 2011
Posts: 3180
Location: Cradle of Humankind

PostPosted: Sun 18 Aug 2019, 07:34    Post subject:  

What is this, a monologue? Are you looking for feedback from other users?
Back to top
View user's profile Send private message 
s243a

Joined: 02 Sep 2014
Posts: 2188

PostPosted: Sun 18 Aug 2019, 11:34    Post subject:  

nic007 wrote:
What is this, a monologue? Are you looking for feedback from other users?


I need a spot to document and fix issues [2] but feedback is certainly welcome, although it is quite early at this point [1]. I will also be referencing this thread from my tiny_puduan thread as a way to provide more details about changes and issues.

Notes
---------------------------
1. The build process isn't yet seemless enough where I want to promote my fork of woof-next for general use. The most sginificant issue here is that there are some key distro/release specific related files that I have to merge into the chroot folder before certain things are installed. For instance I need to merge DISTROSPECS
https://github.com/s243a/woof-CE/tree/woof-next/woof-distro/x86/tiny_devuan/ascii/fs_basesfs/etc
into the chroot folder prior to installing the puppy package mangager or sc0ttman's package manager (i.e. pkg). I haven't yet done this because I was focussing lately on fixing issues with tiny_devaun which was built via this woof-next fork.
2. I may eventually move the project to gitlab so that this fork can have it's own issue tracking. However, first I want to get the code working well enough and possibly offer some pattaches to the zwn branch of woof-CE.

_________________
Find me on minds and on pearltrees.
Back to top
View user's profile Send private message Visit poster's website 
s243a

Joined: 02 Sep 2014
Posts: 2188

PostPosted: Mon 19 Aug 2019, 00:31    Post subject: Re: Is "${PWD}" every wrong?
Subject description: and how our assumptions kill us.
 

s243a wrote:
Here is something that looks safe
Code:

CURDIR="${PWD}" # get current dir, before cd to WORKDIR

/s243a/woof-CE/.../woof-code/rootfs-packages/PKG/usr/sbin/pkg#L39
but is it? Probably but is PWD inherited by subprocesses or do we need to export it? What if we execute a command like follows:
Code:

urxvt -e pkg -i pkgname

is PWD still inherated by the subprocess if we don't export it? These questions might sound strange but I was working on a script to install a package (via pkg) by clicking on it and the last command of the script worked if I called the command directly from the command line but when the script was invoked by clicking on a .pet file the same command that worked from the command line (last command executed in my script) didn't work even though said command worked when being used directly on the command line.

Here is the script which is called when a .pet file is clicked..
https://pastebin.com/EPgPujar
but
Code:

pkg -i pkgname

did not work when it was called from a script that was invoked by clicking on a .pet file in rox.

WHen I finally figured out how to debug it, I noticed that I got
Code:

CURDIR=/root

rather than the directory in which I clicked on the script. Does this sound right? My solution wasn't to export either CURDIR or PWD since parts of the code seem to assume that the "dot pet" file is in the CURDIR. For example
Code:

tar $taropts "${CURDIR}/${PKGNAME}.tar.${TAREXT}" 2> $TMPDIR/$SELF-cp-errlog

/s243a/woof-CE/../woof-code/rootfs-packages/PKG/usr/sbin/pkg#L4676

This likely occured because sc0ttman borrowed some code from puppies package manager. Since I identified this assumption, I simply made the current directorary the directory where the package is located.

Code:

  if [ -f "$1" ];then
    PKGFILE="$1"
    CURDIR="$(dirname "$(realpath "$1")")"
    PKGNAME=`basename "$1" .$pkg_ext`
  else

/s243a/woof-CE/.../woof-code/rootfs-packages/PKG/usr/sbin/pkg#L4587

The code in the above link doesn't quite match the above code (it's missing the "CURDIR=" line) because I have yet pushed this change to github.


This appears to be working correctly so I pushed it to github in commit db46b1a. The above pastebin link contains test code. The code on github has the debugging code commented out:
/s243a/woof-CE/blob/woof-next/woof-code/rootfs-packages/puppycore_noarch/usr/sbin/petget (on s243a's woof next fork)

One thing that this doesn't do yet is notify one when the installation has finished.

Edit: the related change to /usr/sbin/pkg is in commit
6910569

_________________
Find me on minds and on pearltrees.
Back to top
View user's profile Send private message Visit poster's website 
recobayu


Joined: 15 Sep 2010
Posts: 374
Location: indonesia

PostPosted: Mon 19 Aug 2019, 02:17    Post subject:  

Hi s243a,
I just look at github. The last commit I see is at 30 May 2019.
Is there a new you upload?
Thank you.
Back to top
View user's profile Send private message 
s243a

Joined: 02 Sep 2014
Posts: 2188

PostPosted: Mon 19 Aug 2019, 07:45    Post subject:  

recobayu wrote:
Hi s243a,
I just look at github. The last commit I see is at 30 May 2019.
Is there a new you upload?
Thank you.


Here's the link that you want:
https://github.com/s243a/woof-CE/tree/woof-next

from this link you can click on the compare button to see all the commits. The most recent commit is at the bottom of the list.

_________________
Find me on minds and on pearltrees.
Back to top
View user's profile Send private message Visit poster's website 
Display posts from previous:   Sort by:   
Page 1 of 2 [19 Posts]   Goto page: 1, 2 Next
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » Advanced Topics » Cutting edge
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.1336s ][ Queries: 12 (0.0079s) ][ GZIP on ]