[ANNOUNCE] src2pkg-2.1 released

Under development: PCMCIA, wireless, etc.
Message
Author
amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

[ANNOUNCE] src2pkg-2.1 released

#1 Post by amigo »

I've just released a new version of src2pkg which fixes some issues when using src2pkg with recent versions of glibc and/or gcc.

Puppy users shouldn't be affected by these problems unless they have upgraded their toolchain somehow to use glibc>=2.10 and/or gcc-4.4.x.

Still, this release also includes a few minor fixes and/or additions which have been made over the last three weeks since the last version of src2pkg was released.

The new installable pet package can be downloaded here.


Edit: the above version is updated from the original src2pkg-2.1-noarch-3.pet to include more changes for 64-bit compatibility, plus whatever other changes have gone in in the last few days.

As always, feedback is highly useful, so, if you are using, or have in the past used src2pkg your comments and suggestions are more than welcome. You can PM me here on the forum, or email me directly at:
< amigo AT ibiblio DOT org >
Gilbert

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#2 Post by amigo »

The original version of src2pkg posted in this thread has been updated, so if you have any probelms building src2pkg-helpers, use the updated version. See the top post for the link to the new package.
Any feedback would really be appreciated.

big_bass
Posts: 1740
Joined: Mon 13 Aug 2007, 12:21

#3 Post by big_bass »

Hey Gilbert here's a screen output on slaxer_pup

hope this helps

Joe

--------------


note: using the tgz package I havent tried the pet everything is auto built
this is just a screen output after you install for the first time what to expect to see








#

Code: Select all

src2pkg --setup
Notice - Creating src2pkg-helpers:
src2pkg uses a shared library and a few programs
when creating packages. These binaries will be
compiled on your system for best compatibility,
and installed. When done, src2pkg is ready for use.

TEMP_DIR=/usr/src/src2pkg/builds/src2pkg-helpers
Starting build in 5 seconds
Unpacking sources - OK
Creating libsentry - OK
Creating tar-1.13 - OK
Creating coreutils - OK
Creating unionfs-fuse - OK
- Creating Slackware-type tgz package -

Slackware package maker, version 3.14159.

Searching for symbolic links:
usr/libexec/src2pkg/bin/install -> ginstall

Making symbolic link creation script:
( cd usr/libexec/src2pkg/bin ; rm -rf install )
( cd usr/libexec/src2pkg/bin ; ln -sf ginstall install )

Unless your existing installation script already contains the code
to create these links, you should append these lines to your existing
install script. Now's your chance. :^)

Would you like to add this stuff to the existing install script and
remove the symbolic links ([y]es, [n]o)? n


This next step is optional - you can set the directories in your package
to some sane permissions. If any of the directories in your package have
special permissions, then DO NOT reset them here!

Would you like to reset all directory permissions to 755 (drwxr-xr-x) and
directory ownerships to root.root ([y]es, [n]o)? n

Creating Slackware package: /usr/src/src2pkg/src2pkg-helpers/src2pkg-helpers-1.0-i486-1.tgz

./
install/
install/doinst.sh
install/slack-desc
usr/
usr/share/
usr/share/doc/
usr/share/doc/src2pkg-helpers-1.0/
usr/share/doc/src2pkg-helpers-1.0/README
usr/libexec/
usr/libexec/src2pkg/
usr/libexec/src2pkg/bin/
usr/libexec/src2pkg/bin/unionfs
usr/libexec/src2pkg/bin/install
usr/libexec/src2pkg/bin/unlink
usr/libexec/src2pkg/bin/touch
usr/libexec/src2pkg/bin/stat
usr/libexec/src2pkg/bin/rmdir
usr/libexec/src2pkg/bin/rm
usr/libexec/src2pkg/bin/readlink
usr/libexec/src2pkg/bin/mv
usr/libexec/src2pkg/bin/mknod
usr/libexec/src2pkg/bin/mkdir
usr/libexec/src2pkg/bin/ln
usr/libexec/src2pkg/bin/link
usr/libexec/src2pkg/bin/groups
usr/libexec/src2pkg/bin/ginstall
usr/libexec/src2pkg/bin/cp
usr/libexec/src2pkg/bin/chown
usr/libexec/src2pkg/bin/chmod
usr/libexec/src2pkg/bin/cat
usr/libexec/src2pkg/bin/tar-1.13
usr/libexec/src2pkg/bin/version
usr/libexec/src2pkg/lib/
usr/libexec/src2pkg/lib/libsentry.so

Slackware package /usr/src/src2pkg/src2pkg-helpers/src2pkg-helpers-1.0-i486-1.tgz created.

Done
Notice - The finished package: src2pkg-helpers-1.0-i486-1.tgz
is located in: /usr/src/src2pkg/src2pkg-helpers
- Installing Slackware-type tgz package -

+==============================================================================
| Installing new package ./src2pkg-helpers-1.0-i486-1.tgz
+==============================================================================

Verifying package src2pkg-helpers-1.0-i486-1.tgz.
Installing package src2pkg-helpers-1.0-i486-1.tgz:
PACKAGE DESCRIPTION:
# src2pkg-helpers Binaries and Libs used by src2pkg
#
# This package contains the libraries and binary programs used
# by src2pkg for building packages. This package is built and
# installed by src2pkg itself when it is run the first time or
# when run with the command 'src2pkg --setup'. These programs are
# compiled on your system to insure compatibility with your
# kernel and glibc versions and machine architecture. The sources
# for src2pkg-helpers are part of the src2pkg package itself.
#
# Packaged by src2pkg
Executing install script for src2pkg-helpers-1.0-i486-1.tgz.
Package src2pkg-helpers-1.0-i486-1.tgz installed.


Done src2pkg is now ready to use.
#

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#4 Post by amigo »

The perfect build!

big_bass
Posts: 1740
Joined: Mon 13 Aug 2007, 12:21

#5 Post by big_bass »

Hey Gilbert
Any feedback would really be appreciated.
you asked for feedback this is a special thing
an advanced option

you already provided for using a config file /etc/src2pkg/src2pkg.conf

you can uncomment options you prefer from the list


well, for first time users that can be a
little uncomfortable until you get the hang of things

so one little line of code could be easily added to
help out and then this could be made into an extra options
menu hint, hint

well for starters it could activate lines of code like this example

Code: Select all

sed '230i\
[[ $PKG_DEST_DIR ]] || PKG_DEST_DIR="$CWD"' /etc/src2pkg/src2pkg.conf>>/etc/src2pkg/src2pkg.conf.personal
 
so if you check line 230 the edit was made for you :D

I know that the src2pkg.conf.personal option is not a valid option for now


but the idea is already there and a menu driven GUI would be easy to add with xdialog since I am working on a package manager
I thought about this would blend in nicely for src2pkg also

you can always PM me


Joe

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#6 Post by amigo »

In the conf file, I do mention that I personally set everythig to build in the current directory -but the conf file has grown so big that one needs to be 'in the mood' to read it through.

I did, not long ago, add a command-line option '-CWD' which does the same thing as uncommenting those 4-5 lines (vis-a-vis working driectories).

You can use a personal conf file, although if you are the only one using src2pkg on your system it doesn't make anymore sense to setup up a personal conf file, than to simply use the main one in /etc/src2pkg/src2pkg.conf -the main one does not get overwritten if you upgrade src2pkg.

I'd be interested in what you come up with as a GUI -especially if it uses dialog/Xdialog -that way it could be use from CLI or from X. Every now and then, a user asks about a GUI for src2pkg, but I simply can't get a picture in my mind of how it should look and work. I've done so much to streamline the command-line use that it seems to me that a GUI would just get in the way. Most users (as far as I can tell), only use 3-4 options to src2pkg. But, as I said, i would be interestes to see what you come up with -even just ideas...

If you want to use a personal conf file, just create it as:
$HOME/.src2pkg.conf
src2pkg even provides a way fopr administrators to 'lock out' users from using a personal conf file, or to override any settings that the admin considers dangerous.

What is even 'kewler' is that src2pkg allows you to create 'extensions' which add functionality at any one of or all of *32* different spots in the code. To be precise, you can create an extension which runs before and/or after each one of the 16 main package-creation steps.

extensions should be placed in the directory '$HOME/.src2pkg/extensions' which you have to create. For instance, if you want to add coe which does something just after the 'fake_install' instruction is run, you simply create a file named $HOME/.src2pkg/extensions/08.post and add the code in there. (fake_install is the eighth step). to run code just before the 6th step, you'd put in a file named:$HOME/.src2pkg/extensions/06.pre

Any code you put in an extension gets priority -that is if it is supposed to run before the name step, then it really gets run first. And if it is a 'post' extension, then it runs after all the rest of the code in the step.

Once again, an administrator can lock one or all users out from using extensions.

I think we need to get our heads together on the package manager -I have already done extensive work on pkgtools to make the database *much* more capable for tracking dependencies and keeping package meta-data, md5sums, etc. I've evn done some work on 'spkg' which is a clone of pktools written in C. It can install and remove packages *much* faster than the script-based tools -up to 30 times faster removing packages.
Of course, whatever you have in mind for a LiveCD distro will bring in some other factors, but I think you'd do well to incorporate your ideas into what I am working on. My database even makes it possible to create a compile-order list. Also, it uses the files which make it compatible with slap-get/gslapt and slackpkg. Of course, all the database files are generated by src2pkg (with a single switch: '-E').
I have also built-in the ability to use pre-install, pre-remove and post-remove scripts (in addition to pkgtools' post-install (doinst.sh))

I really am getting quite close to having a full enough environment for you to start using to create your extra packages -I actually already have enough packages, but need to get them wrapped into an installation CD.

big_bass
Posts: 1740
Joined: Mon 13 Aug 2007, 12:21

#7 Post by big_bass »

Hey Gilbert


In the conf file, I do mention that I personally set everything to build in the current directory -but the conf file has grown so big that one needs to be 'in the mood' to read it through.
well I printed out the 8 pages of options
so I could dig in deeper to build the GUI

I can already visualize it
and started on it


*BTW I did see afterwards where you allowed for the "CWD"
to be used in lines 207-211 but the sed command allows for a quick fix to get started






and you left a lot of liberty to build for many special uses
which is good

what needs to be set down now is a "format"
so that all the work that is done wont have to be rebuilt
the painful way

this is easy though and the GUI will "guide" the the packagers on the road so if they stray its because they didn't follow the format

so for an example if you send me your personal
/etc/src2pkg/src2pkg.conf

that must be used for your packages to be built
without extra command line options being used


maybe you run a lot at the command line
changing options when needed
the GUI makes that easier to communicate that into options
that can be clicked on :)
sort of like once you know how to compile the kernel you can do it manually but the menu is sure a big help and not really something
"extra" when you need it



I can process the GUI to default those values you prefer to be set for the GUI as checked
and the special options unchecked that dont compromise the package
format needed by the package management

some examples



so you get the visual




Joe
Attachments
src2pkg-gui.png
(33.83 KiB) Downloaded 1317 times

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#8 Post by amigo »

Actually, what you have going there looks like a good way to offer a GUI tool to setup the conf file. here's what mine looks like:

Code: Select all

PACKAGER="my name and email address"
SIGNATURE="Packaged by $PACKAGER"
[[ $ADD_SRC2PKG_SCRIPT ]] || ADD_SRC2PKG_SCRIPT="YES"
[[ $COMPRESSION_SIZE_LIMIT ]] || COMPRESSION_SIZE_LIMIT=50

[[ $SOURCES_DIR ]] || SOURCES_DIR="$CWD"
[[ $SRC_BUILDS_DIR ]] || SRC_BUILDS_DIR="$CWD"
[[ $PKG_BUILDS_DIR ]] || PKG_BUILDS_DIR="$CWD"
[[ $PKG_DEST_DIR ]] || PKG_DEST_DIR="$CWD"
[[ $BACKUP_DIR ]] || BACKUP_DIR="$CWD"

[[ $EXTRA_FLAGS ]] || EXTRA_FLAGS="-pipe -fomit-frame-pointer"
[[ $EXTRA_LDFLAGS ]] || EXTRA_LDFLAGS="--relax,--sort-common,--no-keep-memory"
AUTO_CONFIG_OPTIONS="sysconfdir localstatedir"
[[ $AUTO_DESKTOP ]] || AUTO_DESKTOP=YES
AUDIO_NOTIFICATION=SAY
Very minimal as you can see.

I usually start the building of a new source by dragging the tarbal onto the src2pkg-dnd icon on the destktop. This is setup to run src2pkg -N -Q to write a new script, prompting you to correct, if needed, the NAME and VERSION for the package. I usually try to download a source rpm or the debian patch for the sources, if available. Doing so means that src2pkg will write a much better slack-desc file when the package is built.

I then start the build by running:
src2pkg -X -E -A
While that gets started, I look around in the sources -having a look at the README, or any other files which might gives hints about any opzions to configure that I might need. If you have used an rpm or the debian patch, you can use the option '-ACF' to try and automatically add configure options taken from the debian/rules or NAME.spec file. Otherwise, I let the build run until it finishes or fails. (Using the -VV option is good on the first run to see the full output of the configure, make, make install steps.
if the build fails, then I do whatever is indicated by the failure -add some configure options or download and package any dependencies. To add extra options to configure, I then rename the NAME.src2pkg.auto file to simply NAME.src2pkg and then open it in my favorite editor, uncomment the line:
# EXTRA_CONFIGS=""
and add any extra config options there. You should not put the --prefix= option in EXTRA_CONFIGS, and you don't have to woryy about mandir or infodir either. libdir will be added automatically for you, depending on whether your are running a 64bit system. Only really individual options (to the package) need to be set in EXTRA_CONFIGS
I then save the file and re-run:
src2pkg -X -E -A

Once the build succeeds, you will have a pretty good slack-desc file if you used an rpm or had the debian patch(there are other ways that src2pkg can get text for the desc file in some cases). If it needs any fine-tuning I do that and then save it. The -A option causes src2pkg to generate a slack-desc which is included in the package and a *copy* is created in the current directory named new.slack-desc. If you modify this new.slack-desc file, you should then rename it to simply 'slack-desc'. That means that it will be used in the next build without gernerating a new one.

I usually then re-run the whole build with:
src2pkg -X -E
unless it is something which takes a long time to compile. In that case, I might do this:
src2pkg -X -E --resume=fake_install (or resume_configure_source)

You can add the -W option if you are confident that the package will then be right. For small package that compile pretty quickly, I usually run the build several times, from the beginning to be sure that the build goes smotthly all the way through. If any other options need to be changed, or special code needs to be added to the NAME.src2pkg script (like to add or remove extra files) I go along adding that code till everything is just as wanted.

A final build can be done with options like:
src2pkg -X -E -Z2 --splitppkg=devel,i18n -W
-E is for extra databse files which make the package compatible with slapt-get. -Z2 means manpages will be bzipped and binaries will be compressed with upx/upx-ucl. splitpkg explaind itself and -W removes all temoprary build files when done.

If you want to build several packages of different formats, you can do so without rebuilding the sources:
src2pkg -X -TGZ
src2pkg -X -PET --resume=fake_install
src2pkg -X -PET2 --resume=fake_install
etc, etc.

For a GUI for setting up the conf file, I'd just make a list of the options by going through the conf file and picking out the ones that look like good candidates for your users.
Of course for Puppy, you'll want to use PKG_FORMAT=PET (or PET2 for the new-style package database). For Puppy, you may want/need to set FHS_POLICY=NONE if you plan to build packages that install to /usr/local or other non-standard package paths.
If you build packages which install things to /root, then you will need the option FAIL_ON_BAD_DIRS=NO
(To enforce better package standards, do not set these last two above)

I could write many, many pages, but let's go slowly...

big_bass
Posts: 1740
Joined: Mon 13 Aug 2007, 12:21

GUI for conf

#9 Post by big_bass »

Hey Gilbert

here's a GUI for the config its all Xdialog

hope you can adapt it into your src2pkg



I am done with puppy

but not with linux

I am always available to help
a friend though

Joe
Attachments
GUI_src2pkg_conf2.tar.gz
(3.12 KiB) Downloaded 772 times
GUI.png
(41.02 KiB) Downloaded 1209 times

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#10 Post by amigo »

Yeah, I got your PM and have started looking at what you put together. There are some good ideas in there for how to put together something useful for non-geeky users to make things nicer. At least one slacker has asked about a GUI for src2pkg and I told him to feedback some ideas of how he thought it ought to look or what it should do, but nothing yet...

I'm about through with slackware, myself. Actually, I've only been using a small quantity of slack packages for years as my GUI is made up of a lot of stuff which doesn't come with slack. I'm still plugging away on KISS, but I don't always have a chance to do much on it. Still, I guess I'm just a few packages away from ahving an installable and extnedable system -but I still need to put together an installed intrd before creating any iso. Much of the delay is due to working on the package format, the adapted pkgtools and src2pkg features which drive/support that format. I've even done some work with spkg which is a pkgtools clone written in C -much, much faster -running that is, but also slower to hack LOL

Joe, have you managed to collect all the various sources needed for creating packages of all the puppy stuff you use? I have a whole lot of stuff here waiting for review (and rewrite since most of it is *so* sloppy). Some things should just be started over from scratch -for instance all the init scripts should just be chucked and re-done in a saner way which will run faster too. And the whole pup_save/sfs mess needs to be re-designed to work better. All the efforts to make puppy run on OLPC and other light hardware is a waste of time as the save-times and load-times for the large initrd mean that boot-times and shutdown-times are not optimal. I lnog ago came up with a better balance between boot-times and initial app startup times which works better and doesn't use up all the RAM these small machines have.

big_bass
Posts: 1740
Joined: Mon 13 Aug 2007, 12:21

#11 Post by big_bass »

Hey Gilbert



I am waiting on divine intervention :D

or I am not moving

Joe

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#12 Post by amigo »

Sorry, I can't make heads or tails of that response... PM or email me if it's a secret.

big_bass
Posts: 1740
Joined: Mon 13 Aug 2007, 12:21

#13 Post by big_bass »

Hey Gilbert

divine intervention is not explainable
well, not in simple scientific terms but here I go

I had a dream last night of a man trying to rescue a sunk ship
it was big and rusty it cost the man an incredible large sum of money
to lift the ship from the bottom of the deep ocean
(which was just a totally gutted out shell of a ship )
I saw this from a divers view
as if I was filming it as it was happening

In the dream I tried talking to this guy
showing him different things and asking him
what it was worth to see if he understood the value of things correctly
because I just couldn't understand why he spent so much money
on something he could have bought something new thousands
of times better and at small fraction of the cost
he paid for an old rusty shell
of a ship he recovered from the ocean

well, that was the divine intervention :D



when you calculate time into the equation
time is a limited resource

I spent a few years solving puzzles that keeps changing
as far as a mental exercise goes it made me sharp
but I am not the sharpest knife in the drawer because
as far a productivity goes I am very busy but going nowhere :lol:

its quite liberating to remove preset limits
and get out of the box

I agree with you on packaging
this is how I think about it
the OS is not the goal it is a "result"
of well planed packages

if you need help testing anything send it my way
I will be glad try it out and offer feedback

Joe

aragon
Posts: 1698
Joined: Mon 15 Oct 2007, 12:18
Location: Germany

#14 Post by aragon »

you guys are terrific. not only that you do those clever things and you know why you're doing it in your way, it now gets a lyrical component ...

aragon

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#15 Post by amigo »

Very interesting Joe, a perfect analogy for what you've been doing....
I also have been dragging up old rusty ships for a long time -if only to keep my mind sharp. Though I did learn early that it is sometimes better to simply have a good look at the rusty hulk without disturbing it and then re-create the good parts of it from a fresh start.

Say aragon, since you are just around the corner from me, perhaps you'd like to get together over coffee some weekend? I'd give a pretty Penny to make such a date with Joe -though I'd rather be around the corner from him instead of the other way around... AYYYY! Jalisco!

aragon
Posts: 1698
Joined: Mon 15 Oct 2007, 12:18
Location: Germany

#16 Post by aragon »

amigo wrote:Say aragon, since you are just around the corner from me, perhaps you'd like to get together over coffee some weekend?
yes we should. next 2 weeks are busy, but then it should be better. i'll send you a pm.
I'd give a pretty Penny to make such a date with Joe -though I'd rather be around the corner from him instead of the other way around... AYYYY! Jalisco!
i think i understand that one :lol:

aragon

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#17 Post by amigo »

Great, I'll be expecting your PM.

I lived in Mexico for 15 years and miss it a lot, so I'm quite jealous of Joe being there... We are friends even at this distance, but if I were still there I'll bet we'd 'paint the town red' -uh have a great time, I mean.

It'll be interesting to meet a Puppian, aragon -I only have one acquaintance here who even knows what Linux is.

big_bass
Posts: 1740
Joined: Mon 13 Aug 2007, 12:21

#18 Post by big_bass »

Hey Gilbert

I have about 13 years In Mexico
so my "Gringo Spanish" has improved a lot

a lot of good people here but "nada" for linux users
around this "barrio"
maybe in the the next "barrio" I hope

hey working on package tools giving it an Xdialog face lift
when I get it all together its looking good I 'll post a copy

being careful not to disturb that dialog monster
too much


Joe

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#19 Post by amigo »

Yeah, pkgtools is a mess. It would be nice to make it work using either dialog or Xdialog, but that would really take some work. I actually started some years ago on creating dialog/Xdialog 'widgets' as functions which could be used either way. The trouble is that the dimensions have to be tweaked when using dialog boxes with Xdialog.

I've been meaning to PM or email you with a copy of the latest hacked pkgtools I have. I think you'd find them to be more useful than the Slackware versions. I'm still (slowly) working on them and a probably gonna incorporate the optional use of spkg into them. spkg is a very fast clone of pkgtools written in C. Package removal is up to 30 times faster. I have hacked in most of the extra functionality which I have added to pktools themselves -like handling postrm scripts and the extra databse files *-required, *-supplies and others.

I have added a meta-data file to the pacage format which provides good info about each package -PACKAGER, date and system on which package was built. I have also written a script which generates a compile-order list, based on the info found in the pkg-required files.

I also wanted to 'flesh out' the reasons for suggesting that you use my altered versions of pktools:
I changed the location of the database from /var/log to /var/lib/pkgtools because this a.) conforms to the LFSH/LSB standards. b.) It allows you to then mount /var/log on tmpfs which is handy to keep /var/log from growing to huge sizes over time. Stuff in /var/log is really temporary info, but the package database is really semi-permanent info. c.) Putting the database in its' own location also reduces the clutter in /var/log because the new databse has several new directories for the new meta-data and other files. I have opted for having the info in various files (instead of a single file for each package or a single file for 'all* packages) in order to make them more readable and it also makes it much easier and cleaner to 'query' the database using grep and other such tools.

I have also commented on the weaknesses of the slitaz package format, but without elaborating. tazpkg's use of cpio is not good since cpio creates new directories with perms of 700 instead of 755, so any package that creates new directories would need to include post-install code which corrects the perms of the dirs. Also, tazpkg has no facility for a BUILD number or ARCH for your packages so it is pretty limited.

I'll try to put together a package for you of my hacked tools, if you like, so any fixes you make can be incorporated. BTW, you can easily convert to using the new database location by simply copying or moving the existing slack-style database files into the new location. Then, when you use src2pkg with the '-E' option, you'll get (and keep) those extra database files which provide the data needed to do dependency-resolution using slapt-get (or with some extra functionality to be added to pkgtools).
Ojala, y que te vaya bonito..

User avatar
Pizzasgood
Posts: 6183
Joined: Wed 04 May 2005, 20:28
Location: Knoxville, TN, USA

#20 Post by Pizzasgood »

EDIT: Disregard everything I say about AUFS in this post. See my second post after this one for clarification about my confusion. And don't use the aufs patch I attached....

So, I'm currently using src2pkg as part of my automated build system for CheesyRamHog Linux, and intend to continue using it as part of a simple source-based package management system (compiling and doing a -UNION install, and then processing the package it outputs to track the files, for now).


The first thing I noticed, testing in Puppy, was that the -C option doesn't work with -PET and -PET2. Some quick grepping shows that 15-make_package_pet does not use $PKG_DEST_DIR anywhere, so I figure that's the problem. I didn't bother looking into it beyond that because I don't intend to use the .pet support anyway (I'm going with .tgz for now, and might come up with my own format later if that isn't good enough - binary packages aren't a big part of this project, I mainly just need to compile things and then log what goes where when they are installed).


The second thing I noticed is that it doesn't have support for aufs. Normally in Puppy that's fine because it seems to use the fuse deal instead. But when I'm doing the chrooted phase of my build, that isn't available. I could compile it ahead of time, and probably will in the long run, but for now, I just rely on the native unionfs or aufs support in the host system's kernel. Since the version of Puppy I'm running uses aufs, src2pkg won't work with the -UNION option unless I patch it. It's a simple enough change that I'm wondering whether there is perhaps a reason it doesn't already support aufs? Is it known to be unstable when used with src2pkg? Or was this just an oversight? I've attached the patch I made, and have been using this to build my system and so far haven't really had any problems. I did have an issue after installing glibc, but I'm not sure if that's due to aufs or not. It compiled and installed fine, but then something broke and anything that tried to view my active mounts (like running the "mount" command, or running "df") would segfault, and trying to view regions of the filesystem involved with the build would cause weird things to happen. I had a kernel oops in /var/log/messages about include/linux/dcache.h:324. But after a reboot everything was happy again, and no other packages had this problem (glibc continued to have it any time I tried again). I haven't had a chance yet to try compiling glibc from a native non-aufs system yet to compare against.


Third thing: I'm confused as to the purpose of $EXCL_DIR in 08D-union_root_install. Judging by the name, I would guess that it's meant to hide directories that you don't want to appear in the unioned filesystem. But that doesn't make sense to me, because the mount command has $EXCL_DIR at the bottom of the stack. When I tried building, I got errors relating to /dev, /proc, etcetera. I tried reordering it to put $EXCL_DIR between $PKG_DIR and / instead, and for good measure I added .wh.__dir_opaque files in each of the directories that were created in $EXCL_DIR, and then it worked. I attached the patch so you can better see what I did. Maybe I'm missing something? Or maybe unionfs via fuse handles the layering differently from how aufs does it.


Fourth thing: When given an unrecognized option, src2pkg exits with an error code of 0. Is that intentional? I attached a patch that tells it to use "exit 1" instead. This way if there's a glitch in one of my scripts so that they try passing something bogus to src2pkg an error will be detected.


Fifth thing: Lots of packages provide a "make check" rule, but there doesn't seem to be a good way to run this short of making a .auto script and inserting it, which duplicates a lot of code. So I made a patch that adds a 'verify_build' function (in the /usr/libexec/src2pkg/07A-verify_build file). This function will run "$DEFAULT_MAKE_COMMAND check" or whatever has been defined in $VERIFY_COMMAND, logging the results if logging is enabled, and displaying a warning if it doesn't return 0. There is also an $EXIT_ON_VERIFY_FAILURE variable that when set to "YES" will tell it to exit if there is a failure. Running the check can be enabled by passing the -VER option to src2pkg (or setting VERIFY_BUILD="YES"), and $VERIFY_COMMAND can be set with the -t= or --test_build= option (which implies -VER). I've provided the patch, and also a separate patch for the man file (when it's decompressed). There are probably some example files or something that also need patching, and maybe the sample configuration file. I haven't actually gotten around to updating my build system to take advantage of this yet, so it hasn't been heavily tested. It does seem to work though.


Sixth issue: Some packages (I forget the one I had this problem with) have their configuration in a subdirectory, but shouldn't actually be built from it. They want to be built from their main area with some options given to make (I think it was "make -c src"). The problem is that src2pkg sees the subdirectory and just dives in automatically, so that later when the makefile refers to something in the main area via a "..", it doesn't located it and errors (IIRC, at this point it was in the bind-mounted OBJ_DIR in the unioned environment). My quickfix for that was to edit my .auto script to "touch $SRC_DIR/Makefile" so that src2pkg would stay put. It would probably be better to have a commandline option or flag variable to tell it not to go into a subdirectory even if it finds one. So far I've only had this issue with the one package, so I haven't gotten around to implementing that.


Seventh thing: When using -JAIL instead of -UNION, it seems that programs that would have otherwise installed over a symlink to busybox (for example, /usr/bin/realpath), the package it creates includes /bin/busybox (but is otherwise normal and does include the actual file that it should have). Maybe that's one of the known flaws with the -JAIL method. Figured I'd point it out just in case.



Otherwise I like this. Saves me from having to reinvent a couple wheels.


EDIT: I should mention that if there is a good reason to not use aufs, that would be okay for me. CRH itself will be using UnionFS. It's just my host system (Puppy 4.1.2) that is using aufs, and I intend to be migrating over into CRH in the very near future (hopefully by the end of the week, because it would be a lot more painful to try to do that after school fires back up next week (it's Spring Break ATM)).
Attachments
src2pkg-2.1-noarch-7-support_aufs.patch.gz
Don't use this patch, it is incorrect.
(362 Bytes) Downloaded 312 times
src2pkg-2.1-noarch-7-fix_exclusion.patch.gz
Rearrange the $EXCL_DIR variable and put whiteout files in its subdirs.
(450 Bytes) Downloaded 308 times
src2pkg-2.1-noarch-7-unrec_opt_exitcode.patch.gz
Use &quot;exit 1&quot; when there is an unrecognized option.
(459 Bytes) Downloaded 314 times
src2pkg-2.1-noarch-7-add_make_check.patch.gz
Add -VER and -t|--test_build options for using &quot;make check&quot;
(3.13 KiB) Downloaded 309 times
src2pkg-2.1-noarch-7-add_make_check_man_page.patch.gz
Update the (decompressed!) manpage to document the -VER and -t options I added.
(472 Bytes) Downloaded 301 times
Last edited by Pizzasgood on Sat 27 Mar 2010, 19:09, edited 1 time in total.
[size=75]Between depriving a man of one hour from his life and depriving him of his life there exists only a difference of degree. --Muad'Dib[/size]
[img]http://www.browserloadofcoolness.com/sig.png[/img]

Post Reply