Author |
Message |
Iguleder

Joined: 11 Aug 2009 Posts: 2031 Location: Israel, somewhere in the beautiful desert
|
Posted: Thu 15 May 2014, 17:27 Post subject:
|
|
Now it's in good shape - I fixed another memory leak and added support for querying the package database. I did extensive testing in the past two weeks and it works very well.
Now, I want to add a package description field (for display in graphical frontends).
puppyluvr - I want the name to be generic and not associated with a single distro. As a community distro, we're very poor and I don't want to deter other distros from using this code, since that's the best way to lower the maintenance burden. When others use your code, they develop the motivation and man power to fix bugs and add more features.
EDIT: I just submitted committed many efficiency improvements. packdude is way faster now
_________________ My homepage
My GitHub profile
|
Back to top
|
|
 |
slavvo67
Joined: 12 Oct 2012 Posts: 1617 Location: The other Mr. 305
|
Posted: Tue 20 May 2014, 15:07 Post subject:
|
|
Can I suggest implementing this as a pet, first? The shock of replacing PPM, regardless of how bad it is might be a lot to handle.
If possible, it might be better to allow for pet or sfs install and allow users to provide feedback first.
Best,
Slavvo67
|
Back to top
|
|
 |
Iguleder

Joined: 11 Aug 2009 Posts: 2031 Location: Israel, somewhere in the beautiful desert
|
Posted: Fri 23 May 2014, 05:26 Post subject:
|
|
Backwards compatibility is easy to do. It's not that hard to unpack a PET package using C code. It's just a .tar.gz archive with the MD5 appended.
However, I see no reason to do this, because this makes the package manager design more complicated. Also, our PET repositories are really small - it's much easier to convert them to another format. For all the PETs hanging around in the forums, we could use a temporary, "pirate" PET installation tool which extracts packages to / and doesn't integrate with the new package manager.
I just added XZ support to packdude, so packages are tiny. I also added some efficiency improvements, so package download and installation is insanely fast now.
I decided to freeze the code and keep the current code base "stable". More features means more bugs and memory leaks, while at the moment, packdude already does everything I want it to. I might do a packdude 2.0, if I change my mind.
_________________ My homepage
My GitHub profile
|
Back to top
|
|
 |
mavrothal

Joined: 24 Aug 2009 Posts: 3089
|
Posted: Fri 23 May 2014, 16:08 Post subject:
|
|
Iguleder wrote: | Backwards compatibility is easy to do. It's not that hard to unpack a PET package using C code. It's just a .tar.gz archive with the MD5 appended.
However, I see no reason to do this, because this makes the package manager design more complicated. Also, our PET repositories are really small - it's much easier to convert them to another format. For all the PETs hanging around in the forums, we could use a temporary, "pirate" PET installation tool which extracts packages to / and doesn't integrate with the new package manager. |
The problem with a "pirate" tool is that you also need a tool to record them and uninstall them, that looks like a second package manager.
If the "unofficial" pets are allowed, would be better if the package manager is aware of them and handle them properly. Maybe by including the pet conversion script/program that will be used to convert the pet repos, as a first step in the process.
_________________ == Here is how to solve your Linux problems fast ==
|
Back to top
|
|
 |
stemsee
Joined: 27 Jun 2013 Posts: 2547 Location: In The Way
|
Posted: Fri 23 May 2014, 16:25 Post subject:
packdude Subject description: try out |
|
When can we get it?
ht
Today i built Arch and trusty, but there is something wrong with my kernel (i hope it is only that!) I will rebuild a new kernel and insert it tonight. I would like to include packdude.
|
Back to top
|
|
 |
Iguleder

Joined: 11 Aug 2009 Posts: 2031 Location: Israel, somewhere in the beautiful desert
|
Posted: Fri 23 May 2014, 16:34 Post subject:
|
|
Everything (including documentation) is in GitHub - see the first post.
You'll have to set up a repository before you can use packdude.
_________________ My homepage
My GitHub profile
|
Back to top
|
|
 |
stemsee
Joined: 27 Jun 2013 Posts: 2547 Location: In The Way
|
Posted: Fri 23 May 2014, 16:50 Post subject:
repo Subject description: local |
|
How would I do it locally? adding a dvd, or directory such as /var/apt/cache or woof-ce local-repositories, for testing. So that if i wanted to use some packages from wheezy in a precise build i could use packdude to manage local repos as there is only one pet repo but several .deb repos. Or not?
|
Back to top
|
|
 |
Iguleder

Joined: 11 Aug 2009 Posts: 2031 Location: Israel, somewhere in the beautiful desert
|
Posted: Fri 23 May 2014, 17:13 Post subject:
|
|
Nope.
packdude has its own package format and repositories format. You'll have to convert Debian/Ubuntu repositories to packdude's format first.
_________________ My homepage
My GitHub profile
|
Back to top
|
|
 |
technosaurus

Joined: 18 May 2008 Posts: 4872 Location: Blue Springs, MO
|
Posted: Fri 23 May 2014, 23:09 Post subject:
|
|
The new pkg mgr is cool. I love that its expandable.
I just wanted to note that ppm could be 20x faster if the scripts used:
IFS="|"
while read field1 field2 ... fieldN: do
instead of the hacky hodge podge combo of read awk, grep, sed cut
... and others
shell does fine at text processing if its done right, unfortunately most scripters don't have a good understanding of the powerful built-ins.
to do the same in C, strsep or strtok (both are hellacious) are often used or fscanf with a special string that must account for versions with different number of fields. While it is still faster, the code can grow pretty complex ... and complexity has a cost, which is a perfectly good reason to use an actual database like sqlite or lmdb.
_________________ Check out my github repositories. I may eventually get around to updating my blogspot.
|
Back to top
|
|
 |
technosaurus

Joined: 18 May 2008 Posts: 4872 Location: Blue Springs, MO
|
Posted: Fri 23 May 2014, 23:32 Post subject:
|
|
mavrothal wrote: | In addition if you host binary packages, you should also host the sources (according to GPL) | No, it is perfectly legit to make the source available via snail mail on 8 inch floppies at the expense of any requester who cannot trust that they are the same as upstream. There was a whole debate on this with damn small linux ~10 years or so ago.
_________________ Check out my github repositories. I may eventually get around to updating my blogspot.
|
Back to top
|
|
 |
mavrothal

Joined: 24 Aug 2009 Posts: 3089
|
Posted: Sat 24 May 2014, 02:08 Post subject:
|
|
technosaurus wrote: |
I just wanted to note that ppm could be 20x faster if the scripts used:
IFS="|"
while read field1 field2 ... fieldN: do
instead of the hacky hodge podge combo of read awk, grep, sed cut
... and others
|
20x? wow!
PPM has about 700 of those. About 50 in tandem!
However, a lot of these are for writing in tmp for latter use...
Maybe you could provide a working example for one/any of the PPM scripts
(if you do, maybe in another thread - not to highjack this one)
_________________ == Here is how to solve your Linux problems fast ==
|
Back to top
|
|
 |
Iguleder

Joined: 11 Aug 2009 Posts: 2031 Location: Israel, somewhere in the beautiful desert
|
Posted: Sat 24 May 2014, 04:29 Post subject:
|
|
It's fine - go ahead and hijack this thread. It has only one purpose - improve package management in Puppy.
_________________ My homepage
My GitHub profile
|
Back to top
|
|
 |
technosaurus

Joined: 18 May 2008 Posts: 4872 Location: Blue Springs, MO
|
Posted: Sat 24 May 2014, 07:53 Post subject:
|
|
I could probably help put together a script to convert pet .packages databases or pet.specs files directly into PDDB (pack dude database) with ash and sqlite3. Its just a matter of translating the columns into their respective database fields and writing an "INSERT into ...". The biggest thing that would slow that conversion process down would be trying to do each record one at a time. Sqlite can be pretty quick, but not if you open a database write 1 record and close it 1000 times ... better to open it write 1000 records and close it (this is why fastcgi is so much faster than cgi too)
_________________ Check out my github repositories. I may eventually get around to updating my blogspot.
|
Back to top
|
|
 |
Iguleder

Joined: 11 Aug 2009 Posts: 2031 Location: Israel, somewhere in the beautiful desert
|
Posted: Sat 24 May 2014, 08:25 Post subject:
|
|
Sounds like a good idea, I'll give it a go.
It's even easier - packdude's repository creation tool (repodude) accepts a CSV file very similar to Puppy's package lists. Conversion should be extremely easy.
EDIT: I have a working prototype here it's Puppy's "common" repository, converted to packdude's format. Build the latest packdude and use -u to force the use of this repository.
EDIT 2: it works! I was able to install a52dec. packdude's repositories are limited to one version of each package, so I had to filter only the most recent version of each package.
EDIT 3: Puppy's dependencies are a mess. PPM doesn't really handle dependencies, so some package specify non-existing ones, although they work just fine. For example - "urxvt" as a dependency of a gtkdialog-based package, while the right package name is "rxvt-unicode".
EDIT 4: I'm experimenting with taking things two steps forward. I'm trying to generate a self-hosting toolchain using the last GPLv2 versions of GCC and Binutils, plus the latest musl. I think it would be extremely cool to be able to install a very thin, Aboriginal-style distro to a directory using packdude, then build the entire distro (e.g all packages) inside a chroot environment.
_________________ My homepage
My GitHub profile
|
Back to top
|
|
 |
technosaurus

Joined: 18 May 2008 Posts: 4872 Location: Blue Springs, MO
|
Posted: Sat 24 May 2014, 12:34 Post subject:
|
|
're: self hosting
I have been wanting to do the same with llvm, but I forked binutils, gcc, mupdf gpl2 versions @github.com/technosaurus just in case.
_________________ Check out my github repositories. I may eventually get around to updating my blogspot.
|
Back to top
|
|
 |
|