Wget gets error using Ubuntu file search page.

Using applications, configuring, problems
Post Reply
Message
Author
User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

Wget gets error using Ubuntu file search page.

#1 Post by sunburnt »

The Ubuntu site has a search page for packages and files. http://packages.ubuntu.com/

I need to find and download the packages that needed files are in, especially libraries.

This is the resulting page URL from searching for a library:
http://packages.ubuntu.com/search?searc ... &arch=i386

The wget command line:

Code: Select all

wget -P /mnt/sdb3/AppPkg/tools/0_dnld http://packages.ubuntu.com/search?searchon=contents&keywords=libfusion-1.2.so&mode=filename&suite=lucid&arch=i386
Using the Ubuntu page manually finds the file, but wget downloads an error page.
The error:

Code: Select all

  <h1 class="documentFirstHeading">Error</h1><!-- messages.tmpl --><div class="perror">
 <p>keyword not valid or missing</p>
I`m a little surprised it does not work, after all it`s the same URL.
Ubuntu it seems is being tricky with their site access.

tecnosaurus gave me a repository search page, but it fails to find the file also.
I tried many combinations of the values in the search page`s URL, none worked.

### This is critical to building packages automatically. ###

I can`t think of another way to find the package containing a given file.
It would be great if Ubuntu had a download file listing "file to package".
Even with a "package to file" list I could write a script to make one.

Once the package is known, the mirrors can be easily navigated for download.

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#2 Post by sunburnt »

I found exactly what I need in a parent directory of one of Ubuntu`s mirrors.
It`s a "contense" file that`s 257 MB in size. But it does file to package.

But including this huge file in an app. is kinda out of the question.
I still really need a good online search engine to do this.
Either that or the app. will download the 257 MB file to the user`s PC.


# UPDATE: I was able to grep the file down to libs. only, now it`s 2.1 MB.!
So it` 235 KB gzipped, not bad at all if I do say so myself...

Not all lib. files start with lib, and not all have .so in them ( but most do ).

User avatar
SFR
Posts: 1800
Joined: Wed 26 Oct 2011, 21:52

Re: Wget gets error using Ubuntu file search page.

#3 Post by SFR »

sunburnt wrote:The wget command line:

Code: Select all

wget -P /mnt/sdb3/AppPkg/tools/0_dnld http://packages.ubuntu.com/search?searchon=contents&keywords=libfusion-1.2.so&mode=filename&suite=lucid&arch=i386
Using the Ubuntu page manually finds the file, but wget downloads an error page.
The error:

Code: Select all

  <h1 class="documentFirstHeading">Error</h1><!-- messages.tmpl --><div class="perror">
 <p>keyword not valid or missing</p>
Hey Sunburnt.

Try wrapping the URL in quotes.

I guess the problem is that the URL string contains &, so without quotes the URL is probably truncated right before the first occurence of &:
wget -P /mnt/sdb3/AppPkg/tools/0_dnld http://packages.ubuntu.com/search?searchon=contents
and the process is sent to background...

Greetings!
[color=red][size=75][O]bdurate [R]ules [D]estroy [E]nthusiastic [R]ebels => [C]reative [H]umans [A]lways [O]pen [S]ource[/size][/color]
[b][color=green]Omnia mea mecum porto.[/color][/b]

User avatar
RSH
Posts: 2397
Joined: Mon 05 Sep 2011, 14:21
Location: Germany

#4 Post by RSH »

Hi.

Using wget in console and url without the double quotes downloads the error page.

Using wget in console and url with double quotes downloads the page.

Image

Seems to be the &.

I just did saw the new post by SFR and did know he has something like a solution. 8)
[b][url=http://lazy-puppy.weebly.com]LazY Puppy[/url][/b]
[b][url=http://rshs-dna.weebly.com]RSH's DNA[/url][/b]
[url=http://murga-linux.com/puppy/viewtopic.php?t=91422][b]SARA B.[/b][/url]

User avatar
sunburnt
Posts: 5090
Joined: Wed 08 Jun 2005, 23:11
Location: Arizona, U.S.A.

#5 Post by sunburnt »

Hey thanks guys!

I prefer the idea of using active web site pages. That way it`s always up to date.
A static page has creeping errors from web site changes and has to be updated.

This should be the last item for the AppPkg Builder to be able to do local packages.
Now to just tie it all together, I`ve got most of that done, lots of testing to do.

Post Reply