Some PPM versioning problems
Posted: Mon 27 Aug 2012, 12:44
Forum member elroy seems to have run into a bug in the Puppy Package Manager:
Note that he seems to have two problems:
A. PPM offers to install a version that does not meet the specification.
B. PPM doesn't offer to install a version that does meet the specification.
When I have tried this it offers to install both. So I do not know if problem B is a bug or something else. The remainder of this post concerns itself only with problem A.
Looking into this, I have found a few problems with the version checking code in dependencies.sh. Although I am running Racy 5.2.2, I'm reporting it here (in the 5.3 thread) because I've tried the current suite of petget scripts from Woof2 and the same problems arise.
1. The major problem seems to be that, although findmissingpkgs.sh creates the /tmp/petget_missingpkgs_patterns_with_versioning file, which contains the versioning requirements of the missing dependency packages, dependencies.sh never reads it.
2. The format of the entries in the /tmp/petget_missingpkgs_patterns_with_versioning file differs from what the code in dependencies.sh expects. The format in the file looks like this:
But the format expected by the code in dependencies.sh looks like this:
3. Repositories often have multiple versions of a package. Before the versioning code was added, a single grep could find all versions in a repo's database. But with the added need to check versioning, a loop is needed to individually compare the versions of each dependency package with the requirements of the package being installed. (Without the loop strange things happen: like comparing the required version string with a string that is a concatenation of the multiple versions in the repo, separated by newline characters.)
Looking over the code, I have come up with a few ideas on how to deal with these problems. There are certainly other, perhaps better, ways, but I'll submit these ideas as suggestions.
I am attaching a revised dependencies.sh (based on the version that I grabbed from Woof2 fossil. I'm also attaching a diff file, but it is short, so I'll paste it here as well. I've split it up into three pieces and have added notes that I hope will explain my not too twisted logic.
One line was modified to read file with versioning requirements.
One line added to convert the format into what dependencies.sh expects.
The grep is now used to initialize a new variable, $DB_ENTRIES, which can have multiple entries, instead of $DB_ENTRY. The latter is now set to the individual entries as the while loop (that replaced the if block) progresses.
Another new variable, $FOUND_IN_DB, keeps track of whether or not a good package was found in this repo's database.
Previously, when a good package was found, the code would break out of the loop that looped through the various repo databases. But now those breaks would break out of the new loop that loops through the various versions of the dependency package in the current database. We don't want to do that, since we want to provide a list of all compatible versions in the current database, so we just set a flag to indicate that at least one was found in the current data base. Then after that loop finishes we test it, and if it is good we break out of the loop that loops through the various repo databases.
- - - - - - - - - - - - - - - - - - - -
I hasten to add that I am not very familiar with the scripts in petget. Although my changes seem to be working okay in my preliminary testing, I am hoping that someone more familiar with the code (Hi Barry!) will look this over to ensure that it makes sense and doesn't add more bugs than it squashes.
EDIT, 2012-Sep-07: In fact, there was at least one bug in my code. By using petget_missingpkgs_patterns_with_versioning, I neglected any dependencies of the dependencies (and any dependencies of the dependencies of the dependencies, etc.) that were added to petget_missingpkgs_patterns by the code earlier in dependencies.sh. Thankfully, Barry noticed that and "re-implemented" the fix with code that works properly. (See Barry's blog: PPM: deps versioning re-fixed)
So the attached files are now obsolete. The latest dependencies.sh can be found in Woof2 fossil. [END OF EDIT]
Also, anyone testing this needs to be aware that just plugging my modified dependencies.sh into an old Puppy will probably not work. One also needs to get the current versions of the other files in /usr/local/petget/ from Woof2 fossil.
(See How to make my .pet download the correct Java version?)elroy wrote:I have a problem with a .pet I’ve created. It requires a version of java that’s greater than or equal to a said version; upon install (while downloading - trying to locate missing deps) it insists on downloading an earlier version.
Note that he seems to have two problems:
A. PPM offers to install a version that does not meet the specification.
B. PPM doesn't offer to install a version that does meet the specification.
When I have tried this it offers to install both. So I do not know if problem B is a bug or something else. The remainder of this post concerns itself only with problem A.
Looking into this, I have found a few problems with the version checking code in dependencies.sh. Although I am running Racy 5.2.2, I'm reporting it here (in the 5.3 thread) because I've tried the current suite of petget scripts from Woof2 and the same problems arise.
1. The major problem seems to be that, although findmissingpkgs.sh creates the /tmp/petget_missingpkgs_patterns_with_versioning file, which contains the versioning requirements of the missing dependency packages, dependencies.sh never reads it.
2. The format of the entries in the /tmp/petget_missingpkgs_patterns_with_versioning file differs from what the code in dependencies.sh expects. The format in the file looks like this:
Code: Select all
|udev|eq167-patched_t2-w5|
Code: Select all
|udev&eq167-patched_t2-w5|
Looking over the code, I have come up with a few ideas on how to deal with these problems. There are certainly other, perhaps better, ways, but I'll submit these ideas as suggestions.
I am attaching a revised dependencies.sh (based on the version that I grabbed from Woof2 fossil. I'm also attaching a diff file, but it is short, so I'll paste it here as well. I've split it up into three pieces and have added notes that I hope will explain my not too twisted logic.
Code: Select all
--- old/dependencies.sh 2012-08-25 17:49:22.955287915 -0400
+++ new/dependencies.sh 2012-08-25 21:39:16.027231096 -0400
@@ -129,9 +129,10 @@
rm -f /tmp/petget_missing_dbentries* 2>/dev/null
#for depPATTERN in `cat /tmp/petget_missingpkgs_patterns` #ex depPATTERN=|kdelibs| ex2: |kdelibs>2.3.6|
#111107 01micko: fix for '||' messing things up...
-for depPATTERN in `grep '[a-zA-Z]' /tmp/petget_missingpkgs_patterns` #ex depPATTERN=|kdelibs| ex2: |kdelibs>2.3.6|. 120221 jemimah
+for depPATTERN in `grep '[a-zA-Z]' /tmp/petget_missingpkgs_patterns_with_versioning` #ex depPATTERN=|kdelibs| ex2: |kdelibs>2.3.6|. 120221 jemimah
do
+ depPATTERN="`echo -n "$depPATTERN" | sed -e 's%|%\&%g' -e 's%^\&%|%' -e 's%\&$%|%'`"
#110722 separate out any versioning... (see also findmissingpkgs.sh)
xdepPATTERN="`echo -n "$depPATTERN" | sed -e 's%&.*%|%'`" #ex: changes |kdelibs>2.3.6| to |kdelibs|
depVERSIONING="`echo -n "$depPATTERN" | grep -o '&.*' | tr -d '|'`" #ex: >2.3.6
One line added to convert the format into what dependencies.sh expects.
Code: Select all
@@ -148,8 +149,12 @@
do
DBFILE="`basename $ONEREPODB`" #ex: Packages-slackware-12.2-official
#find database entry(s) for this package...
- DB_ENTRY="`cat $ONEREPODB | grep "$depPATTERN"`"
- if [ "$DB_ENTRY" != "" ];then
+ FOUND_IN_DB=""
+ DB_ENTRIES="`cat $ONEREPODB | grep "$xdepPATTERN"`"
+ while [ -n "$DB_ENTRIES" ] ;
+ do
+ DB_ENTRY="`echo -n "$DB_ENTRIES" | head -n 1`"
+ DB_ENTRIES="`echo -n "$DB_ENTRIES" | tail -n +2`"
DB_version="`echo -n "$DB_ENTRY" | cut -f 3 -d '|'`"
if [ "$depVERSIONING" ];then #110722
#110822 support chained operators...
Another new variable, $FOUND_IN_DB, keeps track of whether or not a good package was found in this repo's database.
Code: Select all
@@ -164,7 +169,7 @@
done
if [ "$condFLG" = "good" ];then
echo "$DB_ENTRY" >> /tmp/petget_missing_dbentries-${DBFILE}-2
- break
+ FOUND_IN_DB="good"
fi
#if vercmp ${DB_version} ${VERTEST_OP} ${VERTEST_VAL};then
# echo "$DB_ENTRY" >> /tmp/petget_missing_dbentries-${DBFILE}-2
@@ -172,8 +177,11 @@
#fi
else
echo "$DB_ENTRY" >> /tmp/petget_missing_dbentries-${DBFILE}-2
- break
+ FOUND_IN_DB="good"
fi
+ done # DB_ENTRIES loop
+ if [ "$FOUND_IN_DB" = "good" ];then
+ break ;
fi
done
done
- - - - - - - - - - - - - - - - - - - -
I hasten to add that I am not very familiar with the scripts in petget. Although my changes seem to be working okay in my preliminary testing, I am hoping that someone more familiar with the code (Hi Barry!) will look this over to ensure that it makes sense and doesn't add more bugs than it squashes.
EDIT, 2012-Sep-07: In fact, there was at least one bug in my code. By using petget_missingpkgs_patterns_with_versioning, I neglected any dependencies of the dependencies (and any dependencies of the dependencies of the dependencies, etc.) that were added to petget_missingpkgs_patterns by the code earlier in dependencies.sh. Thankfully, Barry noticed that and "re-implemented" the fix with code that works properly. (See Barry's blog: PPM: deps versioning re-fixed)
So the attached files are now obsolete. The latest dependencies.sh can be found in Woof2 fossil. [END OF EDIT]
Also, anyone testing this needs to be aware that just plugging my modified dependencies.sh into an old Puppy will probably not work. One also needs to get the current versions of the other files in /usr/local/petget/ from Woof2 fossil.