Compiler GUI and other packaging tools

Stuff that has yet to be sorted into a category.
Message
Author
User avatar
Gedrean
Posts: 139
Joined: Fri 05 Jun 2009, 05:59

#121 Post by Gedrean »

well, finding no solution I must be content with that behavior.

See if we can't clean up dir2pet some and we'll see what to do...

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#122 Post by technosaurus »

by capturing writes did you mean something like

Code: Select all

dump2pet [directory] > file
or is it?

Code: Select all

dump2pet [directory] 2> file
I think stderr and stdout require different > to print to a file.

I'll look at dir2pet as well - should be minor modifications to eliminate the issues. Probably just use Xdialogs unless I can figure out a way to get it to use a single gtkdialog effectively.
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#123 Post by technosaurus »

the way the pet directories are made .... the files are "watched" as they are installed and then copied to the appropriate directory. This is probably the best way to do it since only some programs support a --destdir= parameter
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

User avatar
Gedrean
Posts: 139
Joined: Fri 05 Jun 2009, 05:59

#124 Post by Gedrean »

Yeah that's what I basically meant.

Oh well, no problem. Give dir2pet a bit of a makeover in Xdialog (I haven't yet figured out how THAT works) and we'll call it 0.2.0 ?

EDIT:

I sorta meant having make install not actually install the program, but it's okay.

At this point just make dir2pet do its stuff in Xdialog instead of to stdout.

Maybe name it something else, and include it with our tool, so the original dir2pet remains for those who prefer stdout to Xdialog. For now we'll indicate that, and if dir2pet as a script gets overall adopted as a different way we aren't stepping on someone else's toes, I'd think. But that's up to you since you're doing this and 4.4, so you make the call on that.
Last edited by Gedrean on Tue 20 Oct 2009, 21:14, edited 1 time in total.

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#125 Post by amigo »

2 is stderr, 1 is stdout. You can redirect both at once like this:
command &> log.txt

User avatar
Gedrean
Posts: 139
Joined: Fri 05 Jun 2009, 05:59

#126 Post by Gedrean »

Oh yeah I almost forgot...
Here's my current pcompile files, 0.1.7 - some debug stuff was reinserted because of the changes to dir2pet.

Once you've got your dump2pet and dir2pet changes and stuff, and are happy with what you've got, post it up and I'll take out the edits for the debug, and give it back for an 0.2.0.'

EDIT: And now the actual file...

Oh yeah, and it's gained weight. It's like 104K extracted.
Attachments
pcompile-0.1.7.tar.gz
pcompile? 017? BAHFAIL!
(13.5 KiB) Downloaded 607 times

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#127 Post by technosaurus »

xdialog is like gtkdialog-lite with better documentation

I've downloaded the tarball & should have something put together soon. I just need to refresh my knowledge of xdialog.
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

ljfr
Posts: 176
Joined: Thu 23 Apr 2009, 08:35

new2dir

#128 Post by ljfr »

Hi,
a small user experience feedback:
for people compiling/packaging few packages at the same times, do not start more than one new2dir script (or derivative script) at the same time,
otherwise you will get a mix of packages.

regards,

NB:Regarding compilation flags, checking what others are doing :freeBSD, gentoo, slackware,... it seems they all go and advice for minimal customization.

User avatar
Gedrean
Posts: 139
Joined: Fri 05 Jun 2009, 05:59

Re: new2dir

#129 Post by Gedrean »

ljfr wrote:Hi,
a small user experience feedback:
for people compiling/packaging few packages at the same times, do not start more than one new2dir script (or derivative script) at the same time,
otherwise you will get a mix of packages.

regards,

NB:Regarding compilation flags, checking what others are doing :freeBSD, gentoo, slackware,... it seems they all go and advice for minimal customization.
We know that they advise that. Part of what the flags that were chosen is to do is to cut size down a lot (largely because Puppy is for older lower-profile PCs) or to compensate for slow computers by speeding up boot or run times.

And thanks for the suggestions, it does help us to know that. :)
Mostly this is intended as an autocompiler, and the flags, as noted, should work about 99% of the time -- so far they have, I haven't had compiles fail because of those flags. Any compiles that do fail I try without pcompile and get the same results with no flags, just config options. Mostly they're just either badly coded software, or software expecting a depend of a diff version than I have, or stuff that for one reason or another just won't compile in Puppy. Evar.

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#130 Post by amigo »

src2pkg can handle concurrent packaging -as many as you like.
Also, since your tools use installwatch, you should use the updated version or my fork of the project called sentry as it will catch a *lot* more of the files and changes than the old installwatch. Nearly all my changes were incorporated into the installwatch git repo, or you can just link installwatch to sentry.
You can get sentry here:

ftp://distro.ibiblio.org/pub/linux/dist ... .6.tar.bz2

Really, don't try to be so fancy with CFLAGS/LDFLAGS -using -Os will let gcc do all that for you without any other options -it knows which options do what -even when we don't. You'll save tons more size by carefully choosing config options anyway. Even aggressive stripping is not always a good idea -you try to shave a very few bytes off and wind up with something unusable. src2pkg does include the ability to strip aggressively, but it is not on by deafult. If you want to easily strip aggressively, just use 'sstrip' instead of strip.

User avatar
Gedrean
Posts: 139
Joined: Fri 05 Jun 2009, 05:59

#131 Post by Gedrean »

amigo wrote:src2pkg can handle concurrent packaging -as many as you like.
The only thing I have against src2pkg is it doesn't build a PET natively, it builds a different package type, then we can sorta convert to PET, but it seems a bit roundabout.
Also, since your tools use installwatch, you should use the updated version or my fork of the project called sentry as it will catch a *lot* more of the files and changes than the old installwatch. Nearly all my changes were incorporated into the installwatch git repo, or you can just link installwatch to sentry.
Thanks for the information. Installwatch comes on the devx SFS, so it's not really replaceable the way we want to, and new2dir (which was written by BarryK) is what we based 2dirs off of, and it uses installwatch -- so it's kinda hard to migrate. Though looking at Sentry is definitely in my to-do list and it may also get into techno's, as a replacement for installwatch in devx from now on, as long as it has the same interface and return style.

Question though, does that "all my changes were incorporated..." mean the new installwatch, and your Sentry, function identically?
Really, don't try to be so fancy with CFLAGS/LDFLAGS -using -Os will let gcc do all that for you without any other options -it knows which options do what -even when we don't. You'll save tons more size by carefully choosing config options anyway. Even aggressive stripping is not always a good idea -you try to shave a very few bytes off and wind up with something unusable. src2pkg does include the ability to strip aggressively, but it is not on by deafult. If you want to easily strip aggressively, just use 'sstrip' instead of strip.
Well that's good to take into consideration, but as I said the flags chosen here aren't really causing any problems yet. Probably for 0.2.0 the defaults might get changed to just the optimize, architecture, mtune and the like.

Anyhow, thanks for the info and I'll look into Sentry.

User avatar
Gedrean
Posts: 139
Joined: Fri 05 Jun 2009, 05:59

#132 Post by Gedrean »

Having glanced into "sentry" I have to ask: What does --backup do?

Does it, as I am hoping severely and suspecting and wondering, make copies of any overwritten files before they are overwritten, allowing the state of the machine to be reverted after installation, and not losing files that were installed?

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#133 Post by technosaurus »

I am all about upgrading any parts of the devx in 4.4, so I will have a look at updating installwatch/sentry. I have heard a couple of complaints about new2dir missing some things and also having some random parts of the source tree in the final dirs, so it is definitely worth a look.
amigo wrote:... just use 'sstrip' instead of strip.
Yes 'sstrip' does better than 'strip' but if you later run strip on an sstriped binary it becomes completely unusable (at least on the versions I tested), since the woof build environment does this (optionally) that would really be a bad idea and you only save like 1-2kb anyways over normal strip. removing the .notes and .comments sections left by gcc is pretty safe though to make up for the difference. I don't know if sstrip does that or not. --strip-unneeded is good enough and have not had any problems except with e3 which is configured to build out of the box as a binary that resembles an sstripped binary (not your typical gcc + elf format) Note that e3c does not have this problem because it makes a traditional style elf executable
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

User avatar
Gedrean
Posts: 139
Joined: Fri 05 Jun 2009, 05:59

#134 Post by Gedrean »

technosaurus wrote:I am all about upgrading any parts of the devx in 4.4, so I will have a look at updating installwatch/sentry. I have heard a couple of complaints about new2dir missing some things and also having some random parts of the source tree in the final dirs, so it is definitely worth a look.
I KNEW IT!

I've had built programs including bits of the source tree in the new2dir/2dirs/dir2pet group. Annoying as HELL.

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#135 Post by amigo »

All my changes except one (I think) were included upstream, but sentry uses exactly the same syntax, so you can simply symlinkg installwatch to sentry.

The --backup function does exactly as you describe -any file about to be overwritten is copied over to the backup area and is then available to be written back. This is a really cool feature and is fully usable with src2pkg using the '-SAFE' option -it guarantees that your system will be the same as before -even before packing is finished.

As far as not making pets directly, I did post a patch here which makes src2pkg do that -except I didn't quite have all the details correct with the md5sums, I think. A small alteration with the 'correct' method as used for tgz2pet should fix it. Barring that, simply creating a normal tgz/tbz/txz package with src2pkg and then running tgz2pet would still be much simpler than running the several commands you are currently using.
Another thing you might find useful is that src2pkg uses a patched version of tar which works directly with pets -it 'knows' that they are tgz's so you don't have to do all the renaming and re-renaming. Also, the tar version used by src2pkg (very old tar-1.13) is used purposely because of some special behaviours which are not available in later versions -having to do with the way it preserves perms/properties of links, dirs and files. This is why slackware and src2pkg still use it.

One of my collaborators has gotten src2pkg creating *.deb packages (which was in my plans all along -as well as rpm's), so I may roll the pet-creation code back in along with the other varieties. As mentioned already, the real 'problem' with building pets is that they break so many of the 'rules' of what constitutes a 'sane' package, so some of src2pkg's best investigations and corrections would have to be overridden to allow for pet creation.

User avatar
Gedrean
Posts: 139
Joined: Fri 05 Jun 2009, 05:59

#136 Post by Gedrean »

amigo wrote:The --backup function does exactly as you describe -any file about to be overwritten is copied over to the backup area and is then available to be written back. This is a really cool feature and is fully usable with src2pkg using the '-SAFE' option -it guarantees that your system will be the same as before -even before packing is finished.
That's spectacular, and even so it's functionality is gonna get integrated into dir2pet -- where's the backup area that sentry uses?
As far as not making pets directly, I did post a patch here which makes src2pkg do that -except I didn't quite have all the details correct with the md5sums, I think. A small alteration with the 'correct' method as used for tgz2pet should fix it. Barring that, simply creating a normal tgz/tbz/txz package with src2pkg and then running tgz2pet would still be much simpler than running the several commands you are currently using.
But will it create the pet.specs entry, the .desktop entry, and the database entry?
Another thing you might find useful is that src2pkg uses a patched version of tar which works directly with pets -it 'knows' that they are tgz's so you don't have to do all the renaming and re-renaming. Also, the tar version used by src2pkg (very old tar-1.13) is used purposely because of some special behaviours which are not available in later versions -having to do with the way it preserves perms/properties of links, dirs and files. This is why slackware and src2pkg still use it.
Nice.
One of my collaborators has gotten src2pkg creating *.deb packages (which was in my plans all along -as well as rpm's), so I may roll the pet-creation code back in along with the other varieties. As mentioned already, the real 'problem' with building pets is that they break so many of the 'rules' of what constitutes a 'sane' package, so some of src2pkg's best investigations and corrections would have to be overridden to allow for pet creation.
But that's why PETs are so FUN! LOL.

Seriously though, if it creates FULLY compliant pets (including pet.specs and .desktop and database entry) then it'd be great. I still will continue to develop pcompile some, admittedly it's not quite as nice as your tool, but we've put this work into it already and I'd like to at least see it released to a version that does the functions we've already worked out for it.

Chances are I may throw my vote in for using src2pkg as a default builder in future Puppy versions, because it does sound nice. But this /is/ the discussion thread for pcompile.

I really do appreciate the input you've given, and sentry is perhaps the best thing I've seen yet - as it's something I was hoping for - to the point of spending an hour on google - to do. I appreciate that you've created an awesome tool, and believe me, if it can create a good tgz or even an uncompressed directory (as we can just dir2pet once ready, to create our petspecs) it'll probably be what I end up using once 0.2.0 is completed in pcompile (because I don't know enough about coding and all that to go through detecting the other formats, and because it's really just going to remain a simple tool I think for fast builds).

But this isn't the thread for proselytizing ($5 word). Appreciate all the help: yes. Constant advertisements for src2pkg: not so much.

Anyhow, rounding out this post, repeat my earlier question:

Where's sentry's "backup area" located?

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#137 Post by amigo »

You can set the backup area on the command line -either with senntry or with installwatch(I mean the later versions). In src2pkg I do it all through the environmental variables setting them directly. You'll find that using the backup function gets pretty complicated as you have to parse file lists to write the files back. src2pkg creates a tarball of backed-up files so they can be saved for reference if wanted.

I've taken the time today to patch the last release of src2pkg with the pet-creation functionality. If you'll both shoot me an e-mail to:
amigo AT ibiblio.org, I'll send you a copy of it. I worked over the problem with the md5sum 'business' so it should be working as expected.

Sorry for the proselytizing, but I have long been convinced that src2pkg would be *the* ideal tool for creating puppy packages -even if that meant dropping the pet format and using slackware-type packages. There really is no other tool which comes even close to src2pkg's capabilities. big_bass uses it for everything -its' one of the reasons that he has been able to completely rebuild a puppy into individual packages. Since Puppy has so many enthusiastic packagers -many of them novices, it would pay to use src2pkg because it makes basic packaging really easy, ensuring sane packages every time. And yet, it is flexible enough to allow for the creation of any sort of 'one-off' or compley packages. src2pkg has over 10,000 lines of code which have been carefully crafted over 5+ years, so it would take you a long time to create something so capable and easy to use.

The package I have created for you is based on the last release of src2pkg, so it lacks a few fixes from the latest development version. But I wanted to quickly give you something to start with so you could point out any problems. You'll, no doubt, want me to incorporate some interactive code for on-the-fly the creation of pet.specs and the database file. There is code in there already, but rather crude still.

src2pkg also includes a drag-n-drop AppDir for use with rox -saves a lot of typing of filenames. Just drop a tarball on it and get a package -what could be easier. If you like src2pkg, then maybe I get a GUI for it as a side benefit...

User avatar
technosaurus
Posts: 4853
Joined: Mon 19 May 2008, 01:24
Location: Blue Springs, MO
Contact:

#138 Post by technosaurus »

Amigo - Can the latest src2pkg use the txz slackpackages yet? I made a patch to pet get that uses txz packages and pxt (similar to pet - just txz with a md5sum)

I am also adding pxt support to Pcompile btw but pet will be the default for now.
Check out my [url=https://github.com/technosaurus]github repositories[/url]. I may eventually get around to updating my [url=http://bashismal.blogspot.com]blogspot[/url].

User avatar
Gedrean
Posts: 139
Joined: Fri 05 Jun 2009, 05:59

#139 Post by Gedrean »

amigo wrote:You can set the backup area on the command line -either with senntry or with installwatch(I mean the later versions). In src2pkg I do it all through the environmental variables setting them directly. You'll find that using the backup function gets pretty complicated as you have to parse file lists to write the files back. src2pkg creates a tarball of backed-up files so they can be saved for reference if wanted.
Okay. I tried going through the code of the sentry version you linked us to earlier, and I couldn't find any way that the command line function sets the backup area.
I've taken the time today to patch the last release of src2pkg with the pet-creation functionality. If you'll both shoot me an e-mail to:
amigo AT ibiblio.org, I'll send you a copy of it. I worked over the problem with the md5sum 'business' so it should be working as expected.
Done.
Sorry for the proselytizing, but I have long been convinced that src2pkg would be *the* ideal tool....
I figured, but still. :)
The package I have created for you is based on the last release of src2pkg, so it lacks a few fixes from the latest development version. But I wanted to quickly give you something to start with so you could point out any problems. You'll, no doubt, want me to incorporate some interactive code for on-the-fly the creation of pet.specs and the database file. There is code in there already, but rather crude still.
No problem, definitely will play around and look at the code to figure out how to do that...
src2pkg also includes a drag-n-drop AppDir for use with rox -saves a lot of typing of filenames. Just drop a tarball on it and get a package -what could be easier. If you like src2pkg, then maybe I get a GUI for it as a side benefit...
Wait, src2pkg doesn't have its own GUI?

I smell project...

amigo
Posts: 2629
Joined: Mon 02 Apr 2007, 06:52

#140 Post by amigo »

Yes, src2pkg can create tbz, tlz and txz packages and it could do this a year before slackware was able to handle those package types. I had already put that in for vector/zenwalk/etc users.

The backup location is set with the --root option to sentry(use --backup=yes to turn the feature on), or you can do it like this:
export INSTW_ROOTPATH=/path/to/backups
# turn the backup feature on
export INSTW_BACKUP=1
just give those two command in the terminal where you are running from, before running 'sentry make install'.
src2pkg also includes a program called 'trackinstall' which works like checkinstall, and a simple wrapper called 'tracklist' which will give the fullest, most-detailed listing of what happens when you run 'make install' -chmod, chown, mv, ln, mkdir and more are covered.

I have put zero time into a GUI for src2pkg. The command-line interface is so simple, and yet powerful -a GUI would only slow most users down. I did implemet the drag-n-drop functionality, though, and use it every day. I am ROX user, so for a long time dnd would only work with ROX-filer, but it now works with KDE/XFCE/GNOME as well.
Still, I sympathize with the desire for a GUI and will help you work it out -if you don't drive me mad in the process LOL. Most srcpkg users just use it in the simplest of fashions and it will work without problems for at least 80% of anything you throw at it. But for more complex builds, it can use build scripts which guarantee the repeatability of your builds and the scripts are arch-independent. You can use the same script to compile on other arches -even multi-lib distros. So when you have the hang of using it, you can easily copy over your build scripts to that puppy-ppc or puppy-x86_64 machine and rebuild every package with the same ease. there are example scripts included and even group-build scripts which show how to rebuild a whole tree full of many sources with a single command.

Preach, preach, preach.... Sorry, but if you try it sou'll see what I mean -big_bass thanks me big-time for src2pkg, every time I hear from him.

Post Reply