Pdebthing DEB downloader and packager 005

A home for all kinds of Puppy related projects
Message
Author
User avatar
Iguleder
Posts: 2026
Joined: Tue 11 Aug 2009, 09:36
Location: Israel, somewhere in the beautiful desert
Contact:

Pdebthing DEB downloader and packager 005

#1 Post by Iguleder »

Hello, puppians! Meet Pdebthing, my latest and greatest gift to the Puppy world. :D

What is it?

Pdebthing is a tool for downloading Ubuntu or Debian packages and all their dependencies. It can also extract the downloaded packages so you can make SFSs.

The bottom line: Pdebthing offers fully automated and easy SFS creation for Ubuntu or Debian-based puppies.

What can I do with it?

You can use Pdebthing to download DEB packages with all, but ALL their dependencies.

Pdebthing shines when dealing with big and heavy packages like KDE, Amarok or all GNOME stuff. Pdebthing is really powerful with handling dependencies and can definitely aid you when dealing with such packages.

Also, Pdebthing can extract all the packages it downloads to form one big package. Later, you can use the it for SFS creation. The possibilities are endless.

What are the requirements to run Pdebthing?.

- Run Lupu or any other Ubuntu-built Puppy :wink:
- You must have the devx installed.

How do I use it?

First of all, Pdebthing is not for average and non-techy users. You need to be familiar with Puppy and the console in order to be able to use Pdebthing.

Pdebthing has two configuration files: config/preferences and config/packages.

The former contains all preferences. Here's my configuration file (Pdebthing comes with it by default):

Code: Select all

# the Debian version the packages are built for
distroVersion="squeeze"

# the repositories to support: the the Trinity (the KDE 3.x fork) Debian repo, the USA Debian mirror and Debian Multimedia
repositories="http://mirror.its.uidaho.edu/pub/trinity/trinity/debian http://ftp.us.debian.org/debian http://debian-multimedia.org" 

# the repository sections to support
sections="main contrib non-free"

# the package architecture (i386/amd64)
arch="i386"

# download all packages (true) or just those that don't come with Puppy (false, the default)?
# (download all packages, including those that ship with Puppy, or only those that don't?)
downloadInstalled="false"
This file contains only 4 parameters:
- A list of Ubuntu or Debian repositories, such as http://mirrors.kernel.org/ubuntu (the official Ubuntu mirror). Remove repositories you don't need.
- The version of the Ubuntu or Debian the packages are compiled for. Can be “hardy", “jaunty", “karmic", “lucid" or “maverick", "lenny", "squeeze" and so on. Use “lucid" if you want to make packages for Lupu and "squeeze" if you make packages for dpup.
- The sections of the repository to support - in Ubuntu you have "main", "multiverse" , "restricted" and "universe", but Debian has "main", "contrib" and "nonfree". Leave all unless you know what you're doing.
- The architecture the packages are compiled for - i386 or amd64. Leave the default of i386 unless you use Fatdog.
- A setting called "downloadInstalled", can be either "true" or "false": if set to "false", packages that come with your Puppy won't be downloaded. Ideal if you want to make SFSs. Bear in mind that the detection of packages that came with your Puppy works only with Lupu or dpup (since their packages use their Ubuntu or Debian names) and packages that went into the devx are treated as packages that came with it.

The "config" directory has two sample files, for Debian Squeeze and Ubuntu Maverick Meerkat. Just take one of them and modify it to suit your needs - just make sure you use the right distro version.

EDIT: add "free" to the repo sections if you use Ubuntu. I found out Medibuntu has just "free" and "nonfree" sections, the former isn't listed in the sample configuration file.

Also, Pdebthing needs a list of packages to download; that’s config/packages. Check their names through packages.ubuntu.com, because some package names are different on Ubuntu and Debian (for instance, glibc’s package is named “libc6“ and libmad’s package is named
Attachments
Pdebthing-005.tar
MD5: e9a804a8ab664360433f7d2fe469e4c2
(30 KiB) Downloaded 1674 times
Pdebthing-004.tar
MD5: ae3d27e27b0ddf5a602135ab85e99e4b
(30 KiB) Downloaded 1144 times
Pdebthing-003.tar
MD5: 1f0f4ce396e60398d532ee88b780d576
(20 KiB) Downloaded 1103 times
Pdebthing-002.tar
(20 KiB) Downloaded 1075 times
Pdebthing-001.tar.gz
MD5: 7f2218eba7d296fcc5c1eac1cc3dcc11
(2.93 KiB) Downloaded 1260 times
Last edited by Iguleder on Thu 04 Nov 2010, 10:22, edited 16 times in total.
[url=http://dimakrasner.com/]My homepage[/url]
[url=https://github.com/dimkr]My GitHub profile[/url]

stu90

#2 Post by stu90 »

Hi Iguleder,
This seems an interesting project you have here - I have made a few .pets with 01micko's alien2puppy but probably fall outside of the ' ultra-uber-super-duper Puppy geeks and totally crazy people' camp but i like to give things a go anyway :D

I have downloaded Pdebthing but seem to be running into some problems.

Setup is frugal install of Lucid 5.1 (do i need to use devx)

Preferences:

Code: Select all

# the Ubuntu/Debian/[...] version the packages are built for
distroVersion="lucid"

# the repositories to support (separated by tabs)
repositories="http://mirrors.kernel.org/ubuntu"

# the repository sections to support
sections="main multiverse restricted universe"

# the package architecture (i386/amd64)
arch="i386"
Packages:

Code: Select all

predict
predict-gsat
(these two packages are in the ubuntu repositories in puppy package manager.

However i seem to to be doing something wrong ?

Here is screen shot of the out put in terminal after running 0setup and 1download
Image

User avatar
Iguleder
Posts: 2026
Joined: Tue 11 Aug 2009, 09:36
Location: Israel, somewhere in the beautiful desert
Contact:

#3 Post by Iguleder »

Your Perl is bad. Try the devx, if it doesn't have the Switch.pm module Perl needs, try a DEB from packages.ubuntu.com.

Kinda weird, worked like a charm on my Lupu (with the devx), I tested it.

EDIT: I did "find /initrd/pup* -name Switch.pm" ... I think Perl modules are located in the devx, so I guess it is required for Pdebthing to work.
[url=http://dimakrasner.com/]My homepage[/url]
[url=https://github.com/dimkr]My GitHub profile[/url]

User avatar
paulhomebus
Posts: 120
Joined: Thu 21 Jan 2010, 23:23
Location: New Zealand
Contact:

#4 Post by paulhomebus »

Success..

Thanks Iguleder, I've used this for a creating a few pets for Lucid 5.1 so far....

Fontmatrix, Scribus 1.3.8b, Gimp 2.7.3, & more to come....
You'll find them in the forum... I upload them to MU's dotpups.de generally.

Again thanks for developing such an awesome tool
Paul

stu90

#5 Post by stu90 »

paulhomebus wrote:Success..

Thanks Iguleder, I've used this for a creating a few pets for Lucid 5.1 so far....

Fontmatrix, Scribus 1.3.8b, Gimp 2.7.3, & more to come....
You'll find them in the forum... I upload them to MU's dotpups.de generally.

Again thanks for developing such an awesome tool
Paul
How do you get Gimp 2.7.3, what repositories address do you use?

User avatar
paulhomebus
Posts: 120
Joined: Thu 21 Jan 2010, 23:23
Location: New Zealand
Contact:

#6 Post by paulhomebus »

I used this repository... http://ppa.launchpad.net/matthaeus123/m ... vn/ubuntu/

I just browsed the files there and found the latest...
https://launchpad.net/~matthaeus123/+ar ... /+packages

Hope this helps Stu90!

stu90

#7 Post by stu90 »

paulhomebus wrote:I used this repository... http://ppa.launchpad.net/matthaeus123/m ... vn/ubuntu/

I just browsed the files there and found the latest...
https://launchpad.net/~matthaeus123/+ar ... /+packages

Hope this helps Stu90!
Hi Paul
thanks for the reply however I still seem to be doing something wrong, when i add http://ppa.launchpad.net/matthaeus123/m ... vn/ubuntu/
to DEbthing prefrences file and run ./0setup i get this:


error 404 not found.

And with https://launchpad.net/~matthaeus123/+ar ... /+packages i get:

error cannot verify launchpad certificate.

I must have mypreference file set up wrong or something?

preference file:

Code: Select all

# the Ubuntu/Debian/[...] version the packages are built for
distroVersion="lucid"

# the repositories to support
repositories="http://ppa.launchpad.net/matthaeus123/mrw-gimp-svn/ubuntu/ http://mirrors.kernel.org/ubuntu"

# the repository sections to support
sections="main multiverse restricted universe"

# the package architecture (i386/amd64)
arch="i386"

# download all packages (true) or just those that don't come with Puppy? (false)
downloadInstalled="false"

User avatar
paulhomebus
Posts: 120
Joined: Thu 21 Jan 2010, 23:23
Location: New Zealand
Contact:

#8 Post by paulhomebus »

yep remove the / (backslash) from here...

Code: Select all

repositories="http://ppa.launchpad.net/matthaeus123/mrw-gimp-svn/ubuntu/ <<<!!!! http://mirrors.kernel.org/ubuntu"
And always start with a clean Pdebthing - I don't run it twice... once you have the .deb's just move them out of the repos folder and delete the Pdebthing instance....

I have dpkg -x working on my Lucid, because I installed some apt-get stuff, I assume...
So I just run dpkg -x ******whatever*****.deb ./
on each file in that repo folder... which gives me my install files for the dotpet....

Hope that helped, stu90
Paul

stu90

#9 Post by stu90 »

paulhomebus wrote:yep remove the / (backslash) from here...

Code: Select all

repositories="http://ppa.launchpad.net/matthaeus123/mrw-gimp-svn/ubuntu/ <<<!!!! http://mirrors.kernel.org/ubuntu"
And always start with a clean Pdebthing - I don't run it twice... once you have the .deb's just move them out of the repos folder and delete the Pdebthing instance....

I have dpkg -x working on my Lucid, because I installed some apt-get stuff, I assume...
So I just run dpkg -x ******whatever*****.deb ./
on each file in that repo folder... which gives me my install files for the dotpet....

Hope that helped, stu90
Paul
Hi Paul thanks for the tips worked a treat 8)

Now have gimp 2.7.3, i also made the latest stellarium 0.10.5 (pet & sfs) and transmission 2.04 (pet)


Pdebthing is a great little tool, thanks Iguleder

User avatar
Iguleder
Posts: 2026
Joined: Tue 11 Aug 2009, 09:36
Location: Israel, somewhere in the beautiful desert
Contact:

#10 Post by Iguleder »

I'm really glad to see this tool actually helps somehow :)

I integrated all my package tools into Pdebthing 003 and added some improvements, this time it's a little bit faster and has the new 2preppkg script that extracts all DEBs, strips everything, optimizes all PNG images and gzip archives, separates DEV/DOC/NLS and makes a SFS from the main part. In other words, it turns Pdebthing into a full SFS/PET creation kit for Lupu :)

It also keeps the directories with the main, DEV, DOC and NLS components, so you can modify them manually or create packages from them :D

Now I'm doing some testing on my Core 2 Duo, my netbook's Atom can't handle this ... takes time and kills my battery.

Puppy 4eva!!!!! :lol:

EDIT: uploaded, but I decided to leave the PET/SFS creation for the user. Just go to the "sfs" directory, the steamin' hot package is waiting for you there.
[url=http://dimakrasner.com/]My homepage[/url]
[url=https://github.com/dimkr]My GitHub profile[/url]

reckrhodes
Posts: 116
Joined: Wed 30 May 2007, 08:15

#11 Post by reckrhodes »

Hi Iguleder. A happy computing day!

First of all thank you for this tool. (down loading it now)

I wanted to use the Pdebthing to download necessary files for gbrainy and gcompris.

How i wish you could give me some simple guidelines to use your great tool in downloading and making pets or sfs (openoffice and supertux included).

I know that somewhere these packages are already in some places on the forum but i wanted to show some of my co-teachers someday on how to use this Pdebthing of yours. Good luck on your wonderful projects.

By the way, i am using Lucid Puppy Version 5.1. I am so happy with this Puppy. Thank so much to Barry K., playdayz, 01micko, WhoDo and and the rest of the devs and friendly people around here.

I am just lurking around, reading and learning, once again thank you.

User avatar
Iguleder
Posts: 2026
Joined: Tue 11 Aug 2009, 09:36
Location: Israel, somewhere in the beautiful desert
Contact:

#12 Post by Iguleder »

I'll rewrite this topic with a detailed HowTo just for you :)

EDIT: done. :wink:
[url=http://dimakrasner.com/]My homepage[/url]
[url=https://github.com/dimkr]My GitHub profile[/url]

reckrhodes
Posts: 116
Joined: Wed 30 May 2007, 08:15

#13 Post by reckrhodes »

hi Iguleder, thank you so much for the quick guidelines.

I've got a little error here.

these are what i have done so far

- change the content of /config/packages. i am editing it to supertux
- and preferences to

repositories="http://mirrors.kernel.org/ubuntu http://ppa.launchpad.net/matthaeus123/m ... svn/ubuntu"

- key in the following commands. ./0setup, ./1download, ./2preppkg
- i've got stuck to ./2preppkg. it gives error like this

Stripping everything
../strippkg: line 58: optipng: not found
../strippkg: line 58: advpng: not found
../strippkg: line 58: optipng: not found
../strippkg: line 58: advpng: not found
../strippkg: line 58: advdef: not found

- i checked the content of failed.txt. It has these texts
perlapi-5.10.0
upstart-job

I cannot continue with the command dir2sfs or dir2pet because there are no directories that named supertux(s) when i checked the /sfs/stuff directory.

Thank you and I am glad that you take time to response in my previous post.

User avatar
paulhomebus
Posts: 120
Joined: Thu 21 Jan 2010, 23:23
Location: New Zealand
Contact:

#14 Post by paulhomebus »

Sounds like you missed downloading the .deb packages for Optipng and advancecomp.....

Check Iguleder's first post... those packages are seperate from Pdebthing 004
- If you want 2preppkg to be able to strip and optimize packages, get AdvanceCOMP and OptiPNG.

User avatar
Iguleder
Posts: 2026
Joined: Tue 11 Aug 2009, 09:36
Location: Israel, somewhere in the beautiful desert
Contact:

#15 Post by Iguleder »

reckrhodes wrote:- i've got stuck to ./2preppkg. it gives error like this

Stripping everything
../strippkg: line 58: optipng: not found
../strippkg: line 58: advpng: not found
../strippkg: line 58: optipng: not found
../strippkg: line 58: advpng: not found
../strippkg: line 58: advdef: not found
Get OptiPNG and AdvanceCOMP as I said.
reckrhodes wrote:- i checked the content of failed.txt. It has these texts
perlapi-5.10.0
upstart-job
That's normal, I always see those packages as failed when I mess with KDE and I see no problems with my packages.
reckrhodes wrote:I cannot continue with the command dir2sfs or dir2pet because there are no directories that named supertux(s) when i checked the /sfs/stuff directory.
You're supposed to see a directory named "sfs" with a directory named "stuff" under it. The latter is your package. The "sfs" directory should contain stuff_DEV, stuff_DOC and stuff_NLS (in most cases) and a file named included-packages.txt.

By the way: all paths are relative, the "sfs" directory is under the Pdebthing directory, not in /.
[url=http://dimakrasner.com/]My homepage[/url]
[url=https://github.com/dimkr]My GitHub profile[/url]

User avatar
playdayz
Posts: 3799
Joined: Fri 25 Apr 2008, 18:57

#16 Post by playdayz »

You can use Pdebthing to download DEB packages with all, but ALL their dependencies.
This worries me just a bit. We do not want to duplicate libraries and other dependencies that are already present in Lucid Puppy. Probably you have already accounted for that.
Very often in my experience, Ubuntu will report dependencies that are extreme. For instance, it reports Midori as needing 74MB of dependencies. However, someone built (maybe me I can't remember) a Midori with Ubuntu binaries that "only" needs 39MB. Then the shared Midori that technosaurus built in Puppy only needs 17MB--and that is the same version of midori and close to the same webkit.

Most of the time in building packages from Ubuntu they need some manual tweaking and adjustment. I don't want to sound discouraging though--the more good packages for Lucid puppy the better! So, make them and post them for testing and let's go forward. Thanks.

reckrhodes
Posts: 116
Joined: Wed 30 May 2007, 08:15

#17 Post by reckrhodes »

hi Iguleder,

So sorry for being late in my response. I am now very happy to report that i finally got to create a supertux.pet and sfs.

For now, i am trying to create sfs or pets of the following program: gbrainy, gcompris and openoffice. I am very happy and impress with the Pdebthing.

Thank you so much for your help.

User avatar
Iguleder
Posts: 2026
Joined: Tue 11 Aug 2009, 09:36
Location: Israel, somewhere in the beautiful desert
Contact:

#18 Post by Iguleder »

playdayz wrote:This worries me just a bit. We do not want to duplicate libraries and other dependencies that are already present in Lucid Puppy.
I'll answer with a quote.
- A setting called "downloadInstalled", can be either "true" or "false": if set to "false", packages that come with your Puppy won't be downloaded.
No duplicates. I guess you haven't read the whole thing.
playdayz wrote:Very often in my experience, Ubuntu will report dependencies that are extreme.
That's because Ubuntu (like any other major desktop distro) turns everything on and compiles packages with all optional features. There's nothing you can do about this, just compile your own package with your choice of features and dependencies.

And it's your choice - either a non-functioning Ubuntu package with missing dependencies or a bigger but working package, that's something you can achieve with Pdebthing.
playdayz wrote:Most of the time in building packages from Ubuntu they need some manual tweaking and adjustment..
In this case, you're wrong. It takes time to get all the dependencies of an Ubuntu package and it's sort of impossible if you do that manually. The manual tweaks I do are editing of .desktop files, fonts and configuration files in extreme cases. Not a big problem.
playdayz wrote:I don't want to sound discouraging though
Oh, too bad.
[url=http://dimakrasner.com/]My homepage[/url]
[url=https://github.com/dimkr]My GitHub profile[/url]

stu90

#19 Post by stu90 »

If anyone is interested i have made a GUI for Pdebthing - I didn't know how to add option for folder name change (can use right click Rename) so you only need to use the terminal for making the .pet dir2pet or alien2puppy.
Last edited by stu90 on Thu 02 Sep 2010, 12:19, edited 1 time in total.

User avatar
Iguleder
Posts: 2026
Joined: Tue 11 Aug 2009, 09:36
Location: Israel, somewhere in the beautiful desert
Contact:

#20 Post by Iguleder »

Cool stuff, stu90! :)

I'll change Pdebthing's behavior and make it rename all "stuff" directories after the first package mentioned in config/packages, I think it makes sense to assume the most important package goes first. For instance, if you want Amarok, you won't call the package "xine-lib", because that's just a dependency, you want Amarok and not xine. Then you'll be able to make a package creation button, dir2pet or dir2sfs :wink:

Also, I can call dir2sfs and dir2pet from a fourth and final script that does the actual packaging :D

Additionally, I have a plan to fix all rear-corner case issues. For example, there's no check whether the extraction of the extracted package lists worked, because it makes sense to assume that if the download succeeded (that's something it does verify), the archive is valid. I'll add all those checks, error message and exit codes. Once we have that, I'll write a wrapper script that calls all four scripts.

This way you get verbosity and things are simpler. It also makes GUIzation of Pdebthing simpler and more friendly.

Ummm ... that was the ambitious me again, nevermind.
[url=http://dimakrasner.com/]My homepage[/url]
[url=https://github.com/dimkr]My GitHub profile[/url]

Post Reply