Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Sat 20 Dec 2014, 19:20
All times are UTC - 4
 Forum index » Advanced Topics » Additional Software (PETs, n' stuff) » Compiling
Pet building system
Post new topic   Reply to topic View previous topic :: View next topic
Page 1 of 3 [40 Posts]   Goto page: 1, 2, 3 Next
Author Message
01micko


Joined: 11 Oct 2008
Posts: 7841
Location: qld

PostPosted: Sat 11 Oct 2014, 19:23    Post subject:  Pet building system
Subject description: a new (old) approach
 

UPDATE

I have started the git repository. Please fork and add your recipes!!! There are literally thousands needed. I have 4. Laughing Rome wasn't built in a day.

See https://github.com/puppylinux-woof-CE/petbuilds

Code:
git clone https://github.com/puppylinux-woof-CE/petbuilds.git


UPDATE: new2dir is no longer a requirement. Splitting is done by split.sh



----------------------------------------------------------------------------------------------------------

NEW!

New git repository for Puppy Linux initrd.gz programs.

https://github.com/puppylinux-woof-CE/initrd_progs


-----------------------------------------------------------------------------------------------------------

This should have been done years ago, actually before the very first version of puppy was released, but it wasn't and no one has addressed the problem.

We all now the old routine.. compile .. new2dir.. dir2pet, manual garbage. The idea of computers is to *automate* isn't it? That's why scripts/batch files/whatever were invented. Rolling Eyes

Anyway, for the old schoolers who like the manual approach I have kept the old functionality of new2dir and dir2pet, probably not a bad idea for noobs.

However, if you're sick to death of the old school method, I have patched new2dir and dir2pet to be automated with some cli options, well the cli options are essential, there is a little help on the cli until I document this fully (soon).

So.. now it is possible to write a little script (in a specific format) and automate your builds.. you could nearly build the whole system with it.

script goes in >>> pet package comes out. Simple as that, and splitting (doc, dev, nls) is taken care of.

This is in no way a competitor or replacement for amigo's excellent src2pkg. It's for those who want to build in a queue.

Of course a refined recipe is needed and I'll attach a sample jwm.buildpet.

Inspiration for this came from Tman, who did similar to this awhile ago, but not as automated, jamesbond.. and Iguleder who has been harping on about this for ages (with good reason).. and amigo of course.

It still has all the disadvantages of installwatch, in that the binaries are installed to your system. maybe one day I'll get rid of installwacth as many sources these days support DESTDIR and if they don't can be sandboxed. But until then this is a step in the right direction.

Sample recipe (attached too, and a bit lazy .. build triplet? )
Code:
# jwm
# this file requires new2dir and dir2pet updated @ 141012

URL=http://joewing.net/projects/jwm/snapshots/
PKG=jwm
VER=976
COMP=tar.xz
DESC="Joes Window Manager for X"
DEPS=+cairo,+rsvg,+libpng,+libjpeg,+xpm,+fribidi
CAT=Desktop

ARCH=`uname -m`
case $ARCH in
 *64) LIBDIR=lib64 ;;
 *) LIBDIR=lib ;;
esac

retrieve() {
   [ -f "${PKG}-${VER}.${COMP}" ] ||\
    wget -t0 -c ${URL}/${PKG}-${VER}.${COMP}
}

extract() {
   tar -xvf ${PKG}-${VER}.${COMP}
   [ "$?" -eq 0 ] || exit
}

build() {
   # since not X compiling forget build triplet
   cd ${PKG}-${VER}
   ./configure --prefix=/usr \
            --localstatedir=/var \
            --sysconfdir=/etc \
            --libdir=/usr/${LIBDIR}
   [ "$?" -eq 0 ] || exit
   make
   [ "$?" -eq 0 ] || exit
   newer2dir make install
   make clean
   cd -
}
   
package() {
   # add this recipe
   install -d -m 0755 ./$PKG-$VER-$ARCH/usr/share/doc
   cat ${PKG}.petbuild > ./$PKG-$VER-$ARCH/usr/share/doc/$PKG-build-recipe
   if [ "$DESKTOP" ];then
      install -d -m 0755 ./$PKG-$VER-$ARCH/usr/share/applications
      cat $DESKTOP > ./$PKG-$VER-$ARCH/usr/share/applications/$DESKTOP
   fi
   [ -f ./pinstall.sh ] && install -m 0755 pinstall.sh ./$PKG-$VER-$ARCH/
   [ -f ./puninstall.sh ] && install -m 0755 puninstall.sh ./$PKG-$VER-$ARCH/
   for p in `ls|grep "\-${ARCH}"|grep -v "files$"` ; do
      case $p in
         *_DEV*) DESC="$PKG development"; DEPS=+${PKG} ;;
         *_DOC*) DESC="$PKG documentation"; DEPS=+${PKG} ;;
         *_NLS*) DESC="$PKG locales"; DEPS=+${PKG} ;;
      esac   
      dir2pet -x -s -w="$DESC" -d="$DEPS" -c=$CAT -p=${p} 2>&1 >/dev/null
   done
}

# main
retrieve
extract
build
package   
   


Of course this is NOT one size fits all.. it is there to be tweaked for your sources. The new new2dir and dir2pet scripts are an requirement. (attached). Back up the originals first.

There is no warranty as usual and there is a high likelihood of BUGS. Please report/fix as necessary. Should work in all woof-CE builds, may work in some of Barry's woof builds, haven't tested.

Have fun.

PS: please invoke the recipe in it's OWN directory otherwise there could be unforeseen consequences.

UPDATE: installwatch sucks. There is an issue with the new2dir I uploaded, and I really can't work it out, but I have worked around it. There is now a symlink called newer2dir which is called to invoke the silent mode. CLI args are dropped (apart from the usual "make install").

REMOVED jwm attachment, all latest are at github
new2dir_dir2pet-141012-2.tar.bz2
Description 
bz2

 Download 
Filename  new2dir_dir2pet-141012-2.tar.bz2 
Filesize  10.31 KB 
Downloaded  50 Time(s) 

_________________
Woof Mailing List | keep the faith Cool |

Last edited by 01micko on Sun 19 Oct 2014, 21:13; edited 4 times in total
Back to top
View user's profile Send private message Visit poster's website 
amigo

Joined: 02 Apr 2007
Posts: 2293

PostPosted: Sun 12 Oct 2014, 04:03    Post subject:  

Yeah, recipes are the real bottleneck for any build system. The amount of code exclusive to a build should be very limited so that one doesn't have to write a build script from scratch for each package. Ideally, the system should be smart enough to build the simplest sources without any intervention -and with a single command. So, the answer is to use src2pkg as it already does these things -and will even write the build script for you so you always can refer back to it or easily edit it for tweaking the build.
What could be easier?:
'src2pkg URL-of-source'
will actually succeed in many cases. Of course, for many sources you will want/need to add a couple of configure options or a couple of lines of arbitrary code in order to achieve a 'perfect' package.
There are many build systems which address these needs, but all of them require at least a basic recipe -except for src2pkg- still the only system which really can build some things even without a recipe. And, it's 'smartness' can always be overridden when needed and it lets you customize configure/compile options on a system-wide, group or individual level.

BTW, src2pkg is now included in porteus -after being used and lauded for years by picky Slackware users...
Back to top
View user's profile Send private message 
Jasper


Joined: 25 Apr 2010
Posts: 1204
Location: England

PostPosted: Sun 12 Oct 2014, 04:48    Post subject:  

Hi guys,

It's my deficiency, but I'm not clear if this is intended to update to jwm v 976 (the latest version) or if jwm v 976 is needed to use it to make other pets.

My regards

PS I have src2pkg v 3.0 and to make and install the latest version of jwm-976 I use:
Code:
src2pkg -PET http://joewing.net/projects/jwm/snapshots/jwm-976.tar.xz

by running it in a terminal after loading my devx.sfs (devx_precise_5.6.sfs).
Back to top
View user's profile Send private message 
01micko


Joined: 11 Oct 2008
Posts: 7841
Location: qld

PostPosted: Sun 12 Oct 2014, 06:52    Post subject:  

Hello Jasper,

This "build system" is more targeted toward distro/package builders. Once you have a set of recipes worked out, you should be able to identically reproduce the package - and the recipe is copied to the package. Once a builder has done that he/she can then reproduce for another arch or a newer distro.

The jwm recipe is merely an example. I'll attach a couple more.

Puppy has never had build scripts of its own. I've (we've?) always had to go searching through Barry's Blog or his source archive to find out how to compile stuffs, and sometimes it's tricky or BK's instructions are(were) rather vague.

What I'll probably do with this, after my mods to new2dir and dir2pet are proven relatively stable, is push those to woof-CE and start a separate git repo for build recipes. Over time all packages in puppy should have a recipe, that's the theory anyway.

I'm not advocating that anyone should use this over src2pkg. It is a fine program in it's own right, ideal for noobs and experienced users, especially when they need something fast and haven't time to hunt down build scripts.

A bit clearer now?

Cheers

EDIT:: removed attachments, latest are at github

_________________
Woof Mailing List | keep the faith Cool |

Last edited by 01micko on Mon 13 Oct 2014, 20:20; edited 1 time in total
Back to top
View user's profile Send private message Visit poster's website 
Iguleder


Joined: 11 Aug 2009
Posts: 1927
Location: Israel, somewhere in the beautiful desert

PostPosted: Sun 12 Oct 2014, 07:11    Post subject:  

01micko wrote:
... and start a separate git repo for build recipes. Over time all packages in puppy should have a recipe, that's the theory anyway.


I think it's the 4th or 5th time someone tries to do this. I hope it's going to work this time.

The next step is a script that extracts the pet.specs files from all PETs in a directory, then creates a "Packages-puppy-%s-official" thingy.

Once we have this and recipes for all must-have packages (pretty much everything in the "common" repository - the kernel, Aufs' tools, BusyBox, etc'), the development of future puppies (especially new 64-bit ports) will be way easier.

_________________
My homepage
My GitHub profile
Back to top
View user's profile Send private message Visit poster's website MSN Messenger 
ICQ Number 
Jasper


Joined: 25 Apr 2010
Posts: 1204
Location: England

PostPosted: Sun 12 Oct 2014, 16:06    Post subject:  

Hi 01micko,

Thank you very much for taking the time and trouble to explain the principles. The fog has not cleared, but it's not so dense now.

My regards
Back to top
View user's profile Send private message 
01micko


Joined: 11 Oct 2008
Posts: 7841
Location: qld

PostPosted: Sun 12 Oct 2014, 20:25    Post subject:  

No worries Jasper

---------------------------------------------------------------------------------------------------------------

Git repo created,

See main post

_________________
Woof Mailing List | keep the faith Cool |
Back to top
View user's profile Send private message Visit poster's website 
01micko


Joined: 11 Oct 2008
Posts: 7841
Location: qld

PostPosted: Tue 14 Oct 2014, 19:38    Post subject:  

Just some stats...

    petbuilds : 32
    commits : 14
    contributors : 1 Exclamation
    forks : 0 Exclamation


Traffic
    views : 151 Question
    unique visitors : 26
    clones : 1 Idea

_________________
Woof Mailing List | keep the faith Cool |
Back to top
View user's profile Send private message Visit poster's website 
zigbert


Joined: 29 Mar 2006
Posts: 5813
Location: Valåmoen, Norway

PostPosted: Wed 15 Oct 2014, 02:17    Post subject:  

Even if things are moving slowly in the Puppy world, we're pushing in the right direction...
_________________
Stardust resources
Back to top
View user's profile Send private message Visit poster's website 
Geoffrey


Joined: 30 May 2010
Posts: 1511
Location: Queensland Australia ɹǝpu∩uʍop

PostPosted: Wed 15 Oct 2014, 02:23    Post subject:  

I was the clown that cloned Laughing just having a look at the build scripts.
_________________
Carolina: Recent Repository Additions
Create strikethrough text HERE
Back to top
View user's profile Send private message 
01micko


Joined: 11 Oct 2008
Posts: 7841
Location: qld

PostPosted: Wed 15 Oct 2014, 02:49    Post subject:  

Geoffrey wrote:
I was the clown that cloned Laughing just having a look at the build scripts.


When you are happy enough then you can fork Wink . When you make some send a pull request. I'm uploading sources to ibiblio as I go. crazy obscure ones like ifplugd and cddetect_quick (and other custom puppy source).

While you're at it, try the mtp_phone_connect BUT set the POPUP_NOTIFY varible (in the petbuild) to 1 for Carolina as you have a proper notification daemon builtin to xfce. Let me know what you think.

BTW, all testing is done on 64 and 32.

EDIT: petget (Carolina) would need to be patched for xz compression, or hack it out of dir2pet, your choice, unless it already does support xz?

_________________
Woof Mailing List | keep the faith Cool |

Last edited by 01micko on Wed 15 Oct 2014, 06:02; edited 1 time in total
Back to top
View user's profile Send private message Visit poster's website 
sc0ttman


Joined: 16 Sep 2009
Posts: 2386
Location: UK

PostPosted: Wed 15 Oct 2014, 05:24    Post subject:  

Sorry if this is off topic, but ..

In the past when using Akita, and again recently in Slacko beta, I've installed sbopkg and used it to successfully compile a ton of programs from source, using the SlackBuild recipes which spopkg reads to build pkgs... I only had to add 'makepkg' to /sbin, and it was all good..

Can we not leverage the amount of SlackBuilds available online?

Can we steal some code from sbopkg and make this pet builder system work with SlackBuilds and source recipes of various other kinds? (The same way the PPM can install pkgs from various distros)...

_________________
Akita Linux, VLC-GTK, Pup Search, Pup File Search
Back to top
View user's profile Send private message 
01micko


Joined: 11 Oct 2008
Posts: 7841
Location: qld

PostPosted: Wed 15 Oct 2014, 05:47    Post subject:  

Absolutely sc0ttman!

My samba build script will be pilfered straight from PV's slackbuild. I have been building it like that for awhile but manually adding the puppy specifics.

The main difference is the end, where slackware will leverage DESTDIR, we install directly and call new2dir (installwatch frontend). A bit of sed trickery and a slackbuild could be converted easily.

When in doubt I either use a slackbuild or src2pkg.

However, all the custom puppy progs (obscure ones I have mentioned and others developed more recently) do need special attention.

How about a build script for sjpplog? It looks nice (jamesbond's blog is a fine example http://www.lightofdawn.org/blog/ - note the new address ).

Smile

_________________
Woof Mailing List | keep the faith Cool |
Back to top
View user's profile Send private message Visit poster's website 
mavrothal


Joined: 24 Aug 2009
Posts: 1897

PostPosted: Wed 15 Oct 2014, 05:52    Post subject:  

Although all puppies can use slackware packages maybe dir2pet could be modified to use slackbuild output (mostly the install scripts and package info).

BTW line 218 of new2dir above, is a bit strange Razz

_________________
Kids all over the world go around with an XO laptop. They deserve one puppy (or many) too Very Happy
Back to top
View user's profile Send private message 
01micko


Joined: 11 Oct 2008
Posts: 7841
Location: qld

PostPosted: Wed 15 Oct 2014, 05:56    Post subject:  

http://t.co/haQr9bjjn2

Laughing

Dunno how that happened.. but it doesn't seem to affect anything. Confused

EDIT: fixed version pushed to woof-CE/testing

_________________
Woof Mailing List | keep the faith Cool |
Back to top
View user's profile Send private message Visit poster's website 
Display posts from previous:   Sort by:   
Page 1 of 3 [40 Posts]   Goto page: 1, 2, 3 Next
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » Advanced Topics » Additional Software (PETs, n' stuff) » Compiling
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.1142s ][ Queries: 12 (0.0127s) ][ GZIP on ]