Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Fri 01 Aug 2014, 22:40
All times are UTC - 4
 Forum index » Taking the Puppy out for a walk » Suggestions
The State of Package Management
Moderators: Flash, Ian, JohnMurga
Post new topic   Reply to topic View previous topic :: View next topic
Page 5 of 15 [222 Posts]   Goto page: Previous 1, 2, 3, 4, 5, 6, 7, ..., 13, 14, 15 Next

Should Puppy's package format be changed?
Yes, without backwards compatibility.
28%
 28%  [ 11 ]
Yes, with backwards compatibility.
25%
 25%  [ 10 ]
No, but the PET format should be standardized/stricter.
20%
 20%  [ 8 ]
No, the PET format works fine.
25%
 25%  [ 10 ]
Total Votes : 39

Author Message
Ray MK


Joined: 05 Feb 2008
Posts: 760
Location: UK

PostPosted: Mon 06 Feb 2012, 17:59    Post subject:  

Doesn`t anyone see the value of not using loose files in an O.S.?
Squash files can`t get viruses ( I believe...), and don`t corrupt easily.
SFS files require almost no package management, only config. files.
They stay compressed, load to ram, and can be swapped on-the-fly.
And they can be mounted from anywhere, hd, ram, usb, lan, and web.

One of Puppy`s big strengths is it`s main SFS file holding most of the O.S.
And the devx file ( "DEVeloper eXtra", I think...) is it`s second best setup.

Agreed - absolutely.
Especially now that we have so many good "on-the-fly" loaders/un-loaders.

Also good for those with ram challenged kit.

Best regards - Ray
Back to top
View user's profile Send private message 
jpeps

Joined: 31 May 2008
Posts: 3220

PostPosted: Mon 06 Feb 2012, 18:04    Post subject:  

sunburnt wrote:

SFS files require almost no package management, only config. files.
They stay compressed, load to ram, and can be swapped on-the-fly.
And they can be mounted from anywhere, hd, ram, usb, lan, and web.


I don't know about that. All the configs have to match the current environment perfectly, or it won't load. I recall TC's issues...every time there was some minor change (like a file using "-" vs "_"), it wouldn't boot. It's probably fine if every app is completely self contained.

What's to keep users from making and combining SFS apps now? I do. I recall having to remove whiteouts to get an SFS to load when there was a previous version in base (maybe something unique).
Back to top
View user's profile Send private message 
noryb009

Joined: 20 Mar 2010
Posts: 538

PostPosted: Mon 06 Feb 2012, 18:31    Post subject:  

Status update on makepuppypkg: I've done quite a bit of work on it, and it's working perfectly for me. However, I can't currently commit it to github, but I'll hopefully have it up on the weekend.

Quote:
but the new format would be need to be flexable enough to be changed/extended, rather than replaced in the future else people will grumble and eventually go elsewhere with complaints like 'it's too hard to add apps'.

Definitely. Having a pet.specs that is separated by "|" may seem like a good idea, but it becomes hard to add more fields without either breaking backward compatibility or adding the field to the end. Having one field per line would fix this, as outdated package managers would ignore unknown fields, and package managers could have default values for outdated packages.

Quote:
For example, for the opensuse package ConsoleKit-0.4.5-6.2.2.i586.rpm, (in xlm format) the pet.specs is:
<snip>
and this is for just one package!

As much as I would love to use it, I don't think that most puppy builders would be willing to spend that much time working on the package file for a single app

That is the pet.specs, but package builders do not have to worry about them. The program 'rpmbuild' reads a much nicer looking file and converts it to that file.

My 'makepuppypkg' scripts reads a file like arch's PKGBUILD.

Quote:
Does that mean each and every dependency must be uploaded to a puppy repository?

In other distributions, if a package in a repository has a dependency, it would either be in that repository or an upstream one (in arch linux, a "extra" package can use a dependency from "core", but not vice versa).

Quote:
And the devx file ( "DEVeloper eXtra", I think...) is it`s second best setup.

I agree with you saying that SFS files allow save files to stay small, but this quote also shows that bad part about the current SFS files - you get everything, or nothing.
SFS files would be great if either:
- every package in puppy is in it's own SFS file or
- every package in puppy is in it's own PET (or a new format) and users can make their own SFS files from it.
Back to top
View user's profile Send private message 
amigo

Joined: 02 Apr 2007
Posts: 2226

PostPosted: Tue 07 Feb 2012, 10:22    Post subject:  

Every file, dir and link on a fresh install (or bootable live) system, should be part of a single accountable 'unit'. Really, an SFS is no different from a package in some ways.
You nearly said it here:
"every package in puppy is in it's own PET"

The thing is that your concept of a single-file delivery doesn't hold up in practice -unless you are composing them from (usually) smaller units. An SFS which guarantees the usability of all its' 'members', should be composed of one or more 'packages'. The SFS itself is not really a good unit to use for packages simply because it's unhandy and CPU-intensive to work with from a creation standpoint.

An SFS is a file-system image, the same as an *.iso file or e2fs, e3fs. That aspect of it has no bearing on the contents of it. YXou could also create and distribute 'mega-packages' which combine the files of one or more 'units' of files.

Because of the very nature of how software/libs are created, there will nearly always be something 'external' needed by your delivered packge/FS-image. That means that the method of making your thing available and useful has to take that into account. It *is* possible (usually) to create fully static stuff, but if you go all the way wit that, then everything needs *loads* of stuff which winds up being duplicated -maybe thousands of time. How many progs are you running which depend on glibc? Do you have any idea?

The whole concept of shared libs is meant to avoid that, so statically including libs is nearly never the ritght thing to do.

But, the whole of idea of including everything necessary in one unit is simply wrong -unless you really want to upgrade, re-compile and re-assemble everything every time just one thing changes. If you have a program which is gonna need libs (for example), which are produced from other sources, you need to have a way to make sure those libs are available when your program is run -you need dependency-resolution. But resolving depends doesn't mean just having 'that library' on your system. It needs to be 'that library' exactly as the one used when you compiled your program -some stray library with the same name -even version, may not work, because of compile-time options and any further libs that get linked to simply because they exist at compilation time. Dependency resolution means you have a way to track the existence and origin of every file on the syetem.

While possible to create a list of all files included in your 'unit', if that unit is really composed of lots of little things from all over then you have an accounting mess. Processing any given source code is going to produce a limited number of files to be installed. The list of those files (as composed at built-time) is a unit -a package, if you will. Of course there can be split packages, if needed. But mega-packages which combine bits from various sources completely mess up the scheme -as a rule. There are exceptions there also, where several small utilities from several sources might be combined into one package. But when you start combining things which are, or are likely to be needed by any other programs, then your units start overlapping and accounting becomes difficult at best.

I'm tired, but instead of deleting this I'll let it stand just in case... You are going about it the wrong way around by addressing symptoms instead of the problem. The problem is that you have no way of accurately accounting for the necessary attributes of each and every item on your system at a particular time. That should be broken down into intelligible units. No, the user doesn't have to know or handle any of that, but 'devs' who don't do that have an un-maintainable mess.
Back to top
View user's profile Send private message 
jpeps

Joined: 31 May 2008
Posts: 3220

PostPosted: Tue 07 Feb 2012, 15:46    Post subject:  

amigo wrote:
You are going about it the wrong way around by addressing symptoms instead of the problem. The problem is that you have no way of accurately accounting for the necessary attributes of each and every item on your system at a particular time. That should be broken down into intelligible units. No, the user doesn't have to know or handle any of that, but 'devs' who don't do that have an un-maintainable mess.


Very true, and that will inevitably require proposed apps to be submitted to some central authority prior to posting....a horrible idea. Multiply the censored disappearing forum posting issue by 1000X. In the present system, there's no need to maintain a huge database replete with source code for every item. I have yet to experience a boot failure in puppy related some mismatch with an app dependency.
Back to top
View user's profile Send private message 
noryb009

Joined: 20 Mar 2010
Posts: 538

PostPosted: Tue 07 Feb 2012, 16:31    Post subject:  

Quote:
The problem is that you have no way of accurately accounting for the necessary attributes of each and every item on your system at a particular time.

With the current PET format, we don't know what each system item attributes are, but we could with a new format. There could be a "provides" list which has all the provided programs/dependencies and their versions.

Quote:
that will inevitably require proposed apps to be submitted to some central authority prior to posting....a horrible idea. Multiply the censored disappearing forum posting issue by 1000X.

It would "require" submitting a "build script" to a central authority, however this doesn't mean that the authority would have complete power. Any build script that works is better then nothing, and making a build script better is sometimes better. When a patch is in the gray area between being good or bad, it could be either be added as a comment in the build script for interested users to use, or hosted in a separate area which wouldn't be controlled by anyone (other then removing spam/malicious build scripts).

Quote:
In the present system, there's no need to maintain a huge database replete with source code for every item.

Barry currently hosts the sources he uses. We wouldn't have to with a new system either, but it would be beneficial for everybody to have patches for the original source to make the program work with puppy. Hosting patches in a semi-centralized spot would make compiling new package versions and packages for new architectures much easier.

Quote:
I have yet to experience a boot failure in puppy related some mismatch with an app dependency.

You might not have experienced a boot failure, but many packages don't work right, or don't even start, with a mismatched or not found dependency.
Back to top
View user's profile Send private message 
jpeps

Joined: 31 May 2008
Posts: 3220

PostPosted: Tue 07 Feb 2012, 16:38    Post subject:  

noryb009 wrote:

You might not have experienced a boot failure, but many packages don't work right, or don't even start, with a mismatched or not found dependency.


ldd /app

If they were compiled somewhere else, there's no guarantees anyway. There's also no guarantees an updated app will be backwardly compatible with everything. Even if they are, there's no guarantees that it will work correctly in every environment.
Back to top
View user's profile Send private message 
2byte

Joined: 09 Oct 2006
Posts: 357

PostPosted: Tue 07 Feb 2012, 17:57    Post subject:  

Setting aside what packager to use for a moment, does this rough outline sound reasonable for a start?

Have a repository of proven build scripts like slackbuilds.org, with no precompiled anything.

Build scripts would list all relevant information including:
Puppy version
Package version
Architecture 486, 586, x86_32, x86_64
Packages required (on top of the base system) to build the current one.
All configure arguments to be listed in the script.
General notes needed to successfully compile the application from a base system.
Add others here ….

In order for the build script to be accepted as 'approved' it must compile the application on a base system (as it exists in the distro.iso) using only the official devx and any listed dependencies. The dependencies (other than the base system and devx above) would have to have their own approved build scripts. In other words, a developer could get the scripts to compile the dependencies and his app to be tested on a base system with no other packages installed.

From here a separate repository of applications can be compiled for this distro release. In the package would be a reference to the build script used to create it. Once the dependencies are also in this repository the developer could install the approved dependency when building their package instead of starting with source.

I know there are missing details here but if we had something like this then maintaining our pups and packages would be much easier.

Am I getting warm?

_________________

Back to top
View user's profile Send private message 
jpeps

Joined: 31 May 2008
Posts: 3220

PostPosted: Tue 07 Feb 2012, 20:42    Post subject:  

Presently, no devx has to be loaded. Generally, with the exception of needing a lib or two, current pets work across different puppy flavors...so don't really need special build scripts. Compiling...lets say someone wants the latest mplayer...could take a long time..especially on older computers.

I'm guessing that if there was a great advantage to using build scripts, we'd be seeing more of them used already. Regarding the complexity of monitoring, approving, submitting, etc., I doubt that's going to happen (hopefully).
Back to top
View user's profile Send private message 
sunburnt


Joined: 08 Jun 2005
Posts: 5010
Location: Arizona, U.S.A.

PostPosted: Tue 07 Feb 2012, 21:02    Post subject:  

.
# Comments on Barry making another add-on SFS of common libraries.?

One for each of his major releases, so as to provide the needed libs. base.
.
Back to top
View user's profile Send private message 
noryb009

Joined: 20 Mar 2010
Posts: 538

PostPosted: Tue 07 Feb 2012, 23:17    Post subject:  

Quote:
ldd /app

You're missing a few steps:
1) google one dep and find a .PET
2) install the .PET
3) run ldd/app
4) go back to step 1

2byte: You are very close to what I'm picturing. slackbuilds.org is close. I'm picturing a two part respository: an official binary one for popular packages maintained by a team of volunteers (but the community would still be allowed to submit changes), and a second build script repository that doesn't have an approval process (but spam packages would still be deleted).

Many of those fields are needed in a build script, along with a few more.
Quote:
In order for the build script to be accepted as 'approved' it must compile the application on a base system (as it exists in the distro.iso) using only the official devx and any listed dependencies

There will be a list of packages (gcc, make, etc.) that can be used to make programs. If it needs an extra program to make the package (like bacon), it would have a separate "build_deps" field.

Quote:
From here a separate repository of applications can be compiled for this distro release.

This is partly correct - there will be a repository of applications for each puplet, but it would sit on top of a shared repository. The shared might include the firefox, but the puplet reposiory would include firefox which has a popup asking to make firefox the default browser.

Quote:
Compiling...lets say someone wants the latest mplayer...could take a long time..especially on older computers.

Compiling can take a very long time, which is why binaries are available. The problem with hosting a binary package for every program is that it takes up a lot of space. To save both of those, a combination can be used: popular programs can be binaries on a server, but less popular programs only have a build script, allowing users to compile it themselves. Mplayer is a common program, so there would be a binary for it. Some small projects aren't too popular, so they would have a build script (written by an interested user, not by the main developers) for other interested users to use.
If good packages are picked for the official binary repository, then most users would never have to use a build script.

Quote:
I'm guessing that if there was a great advantage to using build scripts, we'd be seeing more of them used already.

Almost every distribution uses build scripts.
Back to top
View user's profile Send private message 
Q5sys


Joined: 11 Dec 2008
Posts: 1047

PostPosted: Fri 10 Feb 2012, 17:36    Post subject:  

Aitch wrote:
Well, I dunno, but slackware has been good so far Very Happy
.....and Arch seems to have the best package management/update process I've come across....

http://www.linuxforums.org/forum/applications/172247-there-best-repository.html

The missing ingredient from puppy's package management seems to be a database

http://en.wikipedia.org/wiki/Package_management_system

Has Fedora/Yum been tried, or gentoo/portage? Laughing Laughing

/aside
jemimah, can I ask for kernel headers.sfs and or kernel source to be made available for saluki - people are still having compile problems occasionally, and these were omitted from both Lupu and Slacko, AFAIK

Aitch Smile


The arch package system is great... but it also has a tendency to break things about once a month. Then you are left with a non usable system and you have to spend time trying to figure out how to fix it. These issues are usually sorted within a day. But if you look in the arch support forums you'll see tons of old threads about pacman breaking the system and people trying to find out how to fix it. It seems that the main devs want you to check the site for possible issues that may arise from updating before you update; instead of updating and then searching for problems.

If this debate is ever sorted... Ive still got a server ready to go for the repo.

Aitch wrote:
Quote:

I'm with Jemimah
Quote:
The problem isn't the ppm, it's the repos.....The only way any of this works at all is if puppy developers compile and test the packages, and specify the dependencies correctly, and if the users stick to installing stuff from the tested repositories only.


That seems simple enough...? - though I don't compile/develop.....

thanks devs! Very Happy

Aitch Smile


Cant src2pet and dir2pet simply be altered to run LDD on the binary and sed that output into a file to be used in the package for dependency checks?

And lastly since it relates to this issue... a thread about a package site.

_________________



My PC is for sale
Back to top
View user's profile Send private message 
amigo

Joined: 02 Apr 2007
Posts: 2226

PostPosted: Sat 11 Feb 2012, 03:14    Post subject:  

It's not pacman which breaks the system -it's the content of the packages. That's the problem with a rolling-update system. Some things, when rebuilt, require the immediate rebuilding of lots of other things. If the rolling updates were released as a complete 'tranch' of such related upgrades, then it would be more reliable.

You have the right idea about usinng ldd to figure out the depends, but that info must be cross-referenced against a list which shows which package contains the files you need. Plus, the output from ldd doesn't tell the whole strory. Many times a program depends on another binary program in order to function -'man' is a good example. It needs 'groff' in order be funtional, but groff is not a library which shows up as a dependency of 'man'.
Back to top
View user's profile Send private message 
Q5sys


Joined: 11 Dec 2008
Posts: 1047

PostPosted: Sat 11 Feb 2012, 03:58    Post subject:  

amigo wrote:
It's not pacman which breaks the system -it's the content of the packages. That's the problem with a rolling-update system. Some things, when rebuilt, require the immediate rebuilding of lots of other things. If the rolling updates were released as a complete 'tranch' of such related upgrades, then it would be more reliable.


The issues Ive had with pacman have all been from updates to the core repo. Changes to pacman itself seem to be nightmarish at times... other times its seamless. I know about a month ago they required you to create a new pacman.conf file because of some weird change they made. Then a few months before that you had an issue with /etc/mtab. Before that I know I had an issue with mkinitcpio.
Those type of issues are what cause most of the headaches of people that I know that use arch. Otherwise they love it.
Those things listed above I wouldnt really consider issues with a package. They are from changes to the core system which require special action above and beyond normal updates. However a user has no idea about them until he gets a failed upgrade... or suddenly finds his system will not load properly after an upgrade. Sometimes I wish pacman had a feature where major changes like that would alert the user before installing.

_________________



My PC is for sale
Back to top
View user's profile Send private message 
01micko


Joined: 11 Oct 2008
Posts: 7785
Location: qld

PostPosted: Sat 11 Feb 2012, 04:00    Post subject:  

To address amigo's concern I made a script awhile ago which checks deps of deps and also has a switch to let you know if the dep is in the main puppy-sfs.

here

Probably needs some work..

_________________
Woof Mailing List | keep the faith Cool |
Back to top
View user's profile Send private message Visit poster's website 
Display posts from previous:   Sort by:   
Page 5 of 15 [222 Posts]   Goto page: Previous 1, 2, 3, 4, 5, 6, 7, ..., 13, 14, 15 Next
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » Taking the Puppy out for a walk » Suggestions
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.1378s ][ Queries: 14 (0.0091s) ][ GZIP on ]