Compiler GUI and other packaging tools

Stuff that has yet to be sorted into a category.
Message
Author
User avatar
Gedrean
Posts: 139
Joined: Fri 05 Jun 2009, 05:59

#136 Post by Gedrean »

amigo wrote:The --backup function does exactly as you describe -any file about to be overwritten is copied over to the backup area and is then available to be written back. This is a really cool feature and is fully usable with src2pkg using the '-SAFE' option -it guarantees that your system will be the same as before -even before packing is finished.
That's spectacular, and even so it's functionality is gonna get integrated into dir2pet -- where's the backup area that sentry uses?
As far as not making pets directly, I did post a patch here which makes src2pkg do that -except I didn't quite have all the details correct with the md5sums, I think. A small alteration with the 'correct' method as used for tgz2pet should fix it. Barring that, simply creating a normal tgz/tbz/txz package with src2pkg and then running tgz2pet would still be much simpler than running the several commands you are currently using.
But will it create the pet.specs entry, the .desktop entry, and the database entry?
Another thing you might find useful is that src2pkg uses a patched version of tar which works directly with pets -it 'knows' that they are tgz's so you don't have to do all the renaming and re-renaming. Also, the tar version used by src2pkg (very old tar-1.13) is used purposely because of some special behaviours which are not available in later versions -having to do with the way it preserves perms/properties of links, dirs and files. This is why slackware and src2pkg still use it.
Nice.
One of my collaborators has gotten src2pkg creating *.deb packages (which was in my plans all along -as well as rpm's), so I may roll the pet-creation code back in along with the other varieties. As mentioned already, the real 'problem' with building pets is that they break so many of the 'rules' of what constitutes a 'sane' package, so some of src2pkg's best investigations and corrections would have to be overridden to allow for pet creation.
But that's why PETs are so FUN! LOL.

Seriously though, if it creates FULLY compliant pets (including pet.specs and .desktop and database entry) then it'd be great. I still will continue to develop pcompile some, admittedly it's not quite as nice as your tool, but we've put this work into it already and I'd like to at least see it released to a version that does the functions we've already worked out for it.

Chances are I may throw my vote in for using src2pkg as a default builder in future Puppy versions, because it does sound nice. But this /is/ the discussion thread for pcompile.

I really do appreciate the input you've given, and sentry is perhaps the best thing I've seen yet - as it's something I was hoping for - to the point of spending an hour on google - to do. I appreciate that you've created an awesome tool, and believe me, if it can create a good tgz or even an uncompressed directory (as we can just dir2pet once ready, to create our petspecs) it'll probably be what I end up using once 0.2.0 is completed in pcompile (because I don't know enough about coding and all that to go through detecting the other formats, and because it's really just going to remain a simple tool I think for fast builds).

But this isn't the thread for proselytizing ($5 word). Appreciate all the help: yes. Constant advertisements for src2pkg: not so much.

Anyhow, rounding out this post, repeat my earlier question:

Where's sentry's "backup area" located?

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#137 Post by amigo »

You can set the backup area on the command line -either with senntry or with installwatch(I mean the later versions). In src2pkg I do it all through the environmental variables setting them directly. You'll find that using the backup function gets pretty complicated as you have to parse file lists to write the files back. src2pkg creates a tarball of backed-up files so they can be saved for reference if wanted.

I've taken the time today to patch the last release of src2pkg with the pet-creation functionality. If you'll both shoot me an e-mail to:
amigo AT ibiblio.org, I'll send you a copy of it. I worked over the problem with the md5sum 'business' so it should be working as expected.

Sorry for the proselytizing, but I have long been convinced that src2pkg would be *the* ideal tool for creating puppy packages -even if that meant dropping the pet format and using slackware-type packages. There really is no other tool which comes even close to src2pkg's capabilities. big_bass uses it for everything -its' one of the reasons that he has been able to completely rebuild a puppy into individual packages. Since Puppy has so many enthusiastic packagers -many of them novices, it would pay to use src2pkg because it makes basic packaging really easy, ensuring sane packages every time. And yet, it is flexible enough to allow for the creation of any sort of 'one-off' or compley packages. src2pkg has over 10,000 lines of code which have been carefully crafted over 5+ years, so it would take you a long time to create something so capable and easy to use.

The package I have created for you is based on the last release of src2pkg, so it lacks a few fixes from the latest development version. But I wanted to quickly give you something to start with so you could point out any problems. You'll, no doubt, want me to incorporate some interactive code for on-the-fly the creation of pet.specs and the database file. There is code in there already, but rather crude still.

src2pkg also includes a drag-n-drop AppDir for use with rox -saves a lot of typing of filenames. Just drop a tarball on it and get a package -what could be easier. If you like src2pkg, then maybe I get a GUI for it as a side benefit...

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#138 Post by technosaurus »

Amigo - Can the latest src2pkg use the txz slackpackages yet? I made a patch to pet get that uses txz packages and pxt (similar to pet - just txz with a md5sum)

I am also adding pxt support to Pcompile btw but pet will be the default for now.
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

User avatar
Gedrean
Posts: 139
Joined: Fri 05 Jun 2009, 05:59

#139 Post by Gedrean »

amigo wrote:You can set the backup area on the command line -either with senntry or with installwatch(I mean the later versions). In src2pkg I do it all through the environmental variables setting them directly. You'll find that using the backup function gets pretty complicated as you have to parse file lists to write the files back. src2pkg creates a tarball of backed-up files so they can be saved for reference if wanted.
Okay. I tried going through the code of the sentry version you linked us to earlier, and I couldn't find any way that the command line function sets the backup area.
I've taken the time today to patch the last release of src2pkg with the pet-creation functionality. If you'll both shoot me an e-mail to:
amigo AT ibiblio.org, I'll send you a copy of it. I worked over the problem with the md5sum 'business' so it should be working as expected.
Done.
Sorry for the proselytizing, but I have long been convinced that src2pkg would be *the* ideal tool....
I figured, but still. :)
The package I have created for you is based on the last release of src2pkg, so it lacks a few fixes from the latest development version. But I wanted to quickly give you something to start with so you could point out any problems. You'll, no doubt, want me to incorporate some interactive code for on-the-fly the creation of pet.specs and the database file. There is code in there already, but rather crude still.
No problem, definitely will play around and look at the code to figure out how to do that...
src2pkg also includes a drag-n-drop AppDir for use with rox -saves a lot of typing of filenames. Just drop a tarball on it and get a package -what could be easier. If you like src2pkg, then maybe I get a GUI for it as a side benefit...
Wait, src2pkg doesn't have its own GUI?

I smell project...

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#140 Post by amigo »

Yes, src2pkg can create tbz, tlz and txz packages and it could do this a year before slackware was able to handle those package types. I had already put that in for vector/zenwalk/etc users.

The backup location is set with the --root option to sentry(use --backup=yes to turn the feature on), or you can do it like this:
export INSTW_ROOTPATH=/path/to/backups
# turn the backup feature on
export INSTW_BACKUP=1
just give those two command in the terminal where you are running from, before running 'sentry make install'.
src2pkg also includes a program called 'trackinstall' which works like checkinstall, and a simple wrapper called 'tracklist' which will give the fullest, most-detailed listing of what happens when you run 'make install' -chmod, chown, mv, ln, mkdir and more are covered.

I have put zero time into a GUI for src2pkg. The command-line interface is so simple, and yet powerful -a GUI would only slow most users down. I did implemet the drag-n-drop functionality, though, and use it every day. I am ROX user, so for a long time dnd would only work with ROX-filer, but it now works with KDE/XFCE/GNOME as well.
Still, I sympathize with the desire for a GUI and will help you work it out -if you don't drive me mad in the process LOL. Most srcpkg users just use it in the simplest of fashions and it will work without problems for at least 80% of anything you throw at it. But for more complex builds, it can use build scripts which guarantee the repeatability of your builds and the scripts are arch-independent. You can use the same script to compile on other arches -even multi-lib distros. So when you have the hang of using it, you can easily copy over your build scripts to that puppy-ppc or puppy-x86_64 machine and rebuild every package with the same ease. there are example scripts included and even group-build scripts which show how to rebuild a whole tree full of many sources with a single command.

Preach, preach, preach.... Sorry, but if you try it sou'll see what I mean -big_bass thanks me big-time for src2pkg, every time I hear from him.

User avatar
Gedrean
Posts: 139
Joined: Fri 05 Jun 2009, 05:59

#141 Post by Gedrean »

amigo wrote:The backup location is set with the --root option to sentry(use --backup=yes to turn the feature on), or you can do it like this:
export INSTW_ROOTPATH=/path/to/backups
# turn the backup feature on
export INSTW_BACKUP=1
just give those two command in the terminal where you are running from, before running 'sentry make install'.
Awesome.
I have put zero time into a GUI for src2pkg. The command-line interface is so simple, and yet powerful -a GUI would only slow most users down. I did implemet the drag-n-drop functionality, though, and use it every day. I am ROX user, so for a long time dnd would only work with ROX-filer, but it now works with KDE/XFCE/GNOME as well.
Still, I sympathize with the desire for a GUI and will help you work it out -if you don't drive me mad in the process LOL. Most srcpkg users just use it in the simplest of fashions and it will work without problems for at least 80% of anything you throw at it. But for more complex builds, it can use build scripts which guarantee the repeatability of your builds and the scripts are arch-independent. You can use the same script to compile on other arches -even multi-lib distros. So when you have the hang of using it, you can easily copy over your build scripts to that puppy-ppc or puppy-x86_64 machine and rebuild every package with the same ease. there are example scripts included and even group-build scripts which show how to rebuild a whole tree full of many sources with a single command.
How lovely. As far as a GUI, it would mostly be a GUI-Wrapper. PCompile itself is mostly just an argument interpreter and wrapper for src2all, which has been integrated in now, so it is actually a script, but it doesn't do all that much. Anyhow, it's just that the GUI gives a chance for the less-command-line-oriented puppy users to say "Hey I want this program. Bam. Done."

That's what we were aiming for with pcompile.

This does sound nice though.
Got your email, will be testing and playing with it soon.

User avatar
Gedrean
Posts: 139
Joined: Fri 05 Jun 2009, 05:59

#142 Post by Gedrean »

Techno, in case you didn't get it earlier, let me know when you're done with dump2pet and dir2pet updates and cleanups so they behave nice.

I'll finish up with pcompile to 0.2.0 for making the interface and all that work nice. Hopefully the comments should make things work well.

After that I'm looking at src2pkg -- if I can EVER get it to install properly, lol. It may be a good choice for a unified option. Trying it out will give an idea.

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#143 Post by amigo »

gedrean, I've sent you a new copy with the changes which should make src2pkg install and setup properly.

User avatar
01micko
Posts: 8741
Joined: Sat 11 Oct 2008, 13:39
Location: qld
Contact:

#144 Post by 01micko »

Hi amigo,

I installed src2pkg yesterday in "dpup" (yes, yet another branch) and I discovered I needed 'makepkg'... which I found in none other than big_bass 'slaxer_pup', (incidentally, the ver of src2pkg in slaxer_pup was 1.9.8, now have 1.9.9) and also 'upgradepkg' (is that a 10 yr old script? Testament to Pat's coding), it also complains about something else but basically works. It seems to run through the '--setup' routine wvery time I try it though.

Maybe you need those files Gedrean?

Cheers
Puppy Linux Blog - contact me for access

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#145 Post by amigo »

01micko, this has been fixed so that running 'src2pkg --setup' will create and install a pet package.

email me and I'll send you a copy of the latest version -I am only distribtuing it privately right now while still developing the new *.pet and *deb support.

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#146 Post by technosaurus »

Gedrean - I initially just modified dir2pet to use basic xdialogs in place of the command line sections, but felt that it was still too annoying. I am currently working on combining the dir2pet and petspecs programs into one GUI with mouseover descriptions for each field. I just haven't had a lot of time to work on it lately due to 12+ hr days recently and 3 hrs of commute time. I am close to 50% complete, but with the amount of rewrite that I am doing it will need some testing.
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

User avatar
Gedrean
Posts: 139
Joined: Fri 05 Jun 2009, 05:59

#147 Post by Gedrean »

technosaurus wrote:Gedrean - I initially just modified dir2pet to use basic xdialogs in place of the command line sections, but felt that it was still too annoying. I am currently working on combining the dir2pet and petspecs programs into one GUI with mouseover descriptions for each field. I just haven't had a lot of time to work on it lately due to 12+ hr days recently and 3 hrs of commute time. I am close to 50% complete, but with the amount of rewrite that I am doing it will need some testing.
That is WICKED.

Keep in mind, too much combination could be very difficult to work with, as users that just want to create a petspecs file don't want to build to a pet just yet.
Remember also, gtkdialog has a "tooltip" option (undocumented) for most controls, so you don't need to hardcode most mouseover comments. And to be honest, you only really need it for labels and certain buttons.

Good luck and I look forward to playing with your new Brundlefly. ;)

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#148 Post by technosaurus »

I am planning to put 2 checkboxes for keeping existing pet.specs file and .desktop files - if one is found it will default to true. This will prevent accidentally overwriting a desktop file with locale support with a basic one.
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#149 Post by technosaurus »

some progress on gui - still needs quite some cleanup, but I need some sleep right now

TODO: tooltips, find executables, fileselection for icons,split out the packaging part to enable adoptpet and adoptpxt (maybe tgz, txz others later?)
Attachments
adopt.gz
NOT a real version - just for review for inclusion in Pcompile... works like dir2pet
(2.71 KiB) Downloaded 845 times
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

User avatar
Gedrean
Posts: 139
Joined: Fri 05 Jun 2009, 05:59

#150 Post by Gedrean »

Oh, techno, once you're ready the pcomp_script can be taken out of debug by removing the extra gtkdialog3 calls (they both call the invalid package message, around the make and make install and all those commands) - at this point I'm not really pushing a huge amount of my time into looking at this thread every day because I'm kinda at a stopping point for pcompile - unless you start finding new ideas. Adopt is a neat name though :)

Good luck with all this. I'll keep this thread on my watchlist but I'm gonna focus on other things, so if you have a request or idea go ahead and post it here - I should be able to play with it.

As it is, I'm right now trying to figure out how src2pkg is creating its slack.desc files -- because it's pulling that data from somewhere and I don't know where... Maybe amigo will see this and answer the unasked but implied question therein.

Those details could DEFINITELY be useful for pre-filling in other files like .desktop or such.

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#151 Post by amigo »

Aha! You've discovered one of the nicer features of src2pkg. It seems such a simple thing to create a slack-desc file. But it consumes a lot of time. Before creating the routines which 'mine' the description, writing the desc file usually tooik more time than the whole rest of the process of creating a package. so, even though it was a bit daunting at first, I started looking for ways to pull a description from existing files.

What you are seeing is probably pulled from an rpm *.spec file included in the sources, or from a debian/control file in the sources. These are the two most common and best places to get descriptive info. And for that very reason I usually use the debian patch for a given source tarball, or use a *.src.rpm I can find one. Often, after a first build which gets me the description, I thn stop using the patch or use the tarball instead of the src.rpm -I hate writing desc files.

src2pkg can use text found in *.spec, debian/control, *.lsm, PKGINFO or PKG-INFO(from python modules sources), description-pak files like used by checkinstall(some distros like vector linux use these) or from a NAME.txt file which accompanies a slackware binary package.

Somer real tricky code in there for wrapping the text and each type of description file needs its' own parser -but it's all done in pure bash so it is faster than calling externals.

You may have also noticed that src2pkg will discover what kind of license is being used by the software and write that info to the desc file, as welll as the URL of the homepage :-)

The code which wraps the text allows you to change the line-length if you want and you can also set the number of lines to use in a description file -although the standard length for slack packages is 9-13

As some users have suggested, it would be nice to be able to simply 'read' the README and get the info, but 'simply' is simply not that simple! Anyway, The whole routine is one of the most complex and *lovely* routines in src2pkg. I don't know whether to be more proud or pleased as it saves me loads of work.

User avatar
Gedrean
Posts: 139
Joined: Fri 05 Jun 2009, 05:59

#152 Post by Gedrean »

I figured that was the case. I was MORE impressed by the fact that it found out the license for SDL was LGPL -- but also that it was LGPL 2.1 -- and I couldn't find the 2.1 designator ANYWHERE in source files or text files anywhere.

So I don't know HOW it knew the LGPL version number. Did it just guess, or did it figure out based upon some of the wording in the license?

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#153 Post by amigo »

Yep, it greps for key phrases typical of each license type. Handy. huh?

A couple of quick pointers since you are up and running:

Use the option -Z? to reduce package size:
-Z1 uses bzip2 on man-pages
-Z2 uses upx-ucl, upx or exepak to compres binaries
-Z3 will link the License of the package to a common location (each package contains and installs the license, but subsequent installation which use the same license overwrite the same file.
-Z4 creates an archive of all docs.
Edit: The -Z options are cumulative: -Z4 does all of -Z1, -Z2 and -Z3.

Open /etc/src2pkg/src2pkg.conf and scroll down to line ~230
Uncomment and fill in all those extra compiler flags you want to use:
[[ $EXTRA_FLAGS ]] || EXTRA_FLAGS="-pipe"

If you want to use a bunch of extra options to configure scripts, uncomment and fill in line ~289:
AUTO_CONFIG_OPTIONS="sysconfdir localstatedir"
then use the -ACN option when running src2pkg. Note that src2pkg will examine the configure to script to see if these options are actually available and only those which are will be added to the config options.

Another note: you don't ever need to specify libdir, mandir or infodir as these are handled automatically. well, you can specify libdir, but you should never need to. Since src2pkg handles man-pages and info files -correcting their location when wrongly installed, it doesn't matter where they get installed -src2pkg will fix them according to which FSH_POLICY is in force.

You may find it nicer to uncomment these lines:
[[ $SOURCES_DIR ]] || SOURCES_DIR="$CWD"
[[ $SRC_BUILDS_DIR ]] || SRC_BUILDS_DIR="$CWD"
[[ $PKG_BUILDS_DIR ]] || PKG_BUILDS_DIR="$CWD"
[[ $PKG_DEST_DIR ]] || PKG_DEST_DIR="$CWD"
[[ $BACKUP_DIR ]] || BACKUP_DIR="$CWD"
They cause everything to be built in the current directory which makes it *much* easier to examine the package tree, or go into the sources for whatever reason.

If you have sound, you may wish to uncomment line ~347:
AUDIO_NOTIFICATION=SAY
Set to BEEP or PLAY or, if you have flite or festival (not likely) you can use SAY. Handy when you are watching something else while building packages in the background.
You can get a build script and flite sources here:
http://distro.ibiblio.org/pub/linux/dis ... flite-1.3/

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#154 Post by technosaurus »

Here is a full pet of the cumulative results. more, smaller scripts for building pets and pxts (requires xz and tar >= 1.22)

Fixes:
Fixed expansion for bad tarballs (ex. SDL)
Replaced ../local/pcompile/...... with /usr/local/pcompile/.... so you don't need to run it from within /usr/somedir
New GUI for making pet packages for the new format pet.specs
tweaks to default options

New utilities
adopt - fixes a dir for building a pet (used by adoptpet and adopt pxt)
adoptpet - fixes dir and actually builds pet
adoptpxt - fixes dir and actually builds pxt
dir2pet2 - assumes dir has been "adopted" and turns it into a pet
dir2pxt - assumes dir has been "adopted" and turns it into a pxt (uses xz compression instead of gz like pet packages do)
dumpdir - skips the "adopt" gui and assumes all is good
dmp2pet - runs dumpdir and dir2pet (for DEV, DOC, NLS or known good packages that already have proper menu entry and pet.specs)
dmp2pxt - runs dumpdir and dir2pxt (same as dmp2pet but better compression)
Attachments
pcompile-0.1.8.pet
Still a small TODO list but much better
(17.74 KiB) Downloaded 873 times
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#155 Post by technosaurus »

typo in gzip tarballs - here is the fix:

Code: Select all

sed -i "s/gunzip -k/gunzip/g" /usr/local/pcompile/pcomp_script
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

Post Reply