Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Sat 19 Apr 2014, 20:51
All times are UTC - 4
 Forum index » Off-Topic Area » Programming
How To: Search Unbuntu repository in Bash?
Post new topic   Reply to topic View previous topic :: View next topic
Page 1 of 1 [12 Posts]  
Author Message
sunburnt


Joined: 08 Jun 2005
Posts: 4982
Location: Arizona, U.S.A.

PostPosted: Sat 13 Oct 2012, 21:18    Post subject:  How To: Search Unbuntu repository in Bash?  

I looked at wget and didn`t see any find or search type features.
It`s very possible I missed understanding some of the options.

I tried this to see if I could get a dir. listing I could parse:
Code:
wget -r -A deb --spider http://ubuntu.mirrors.pair.com/archive/pool/universe/s/*


I need to find and verify a files existence ( like a library file ).
And to get a dir. listing to do "grep" on is good for unknown file names.

Search an Ubuntu mirror like:
Code:
http://ubuntu.mirrors.pair.com/archive/pool/universe/


technosaurus gave me a web page file that you can manually do a search.
But I need my app. to be able to automatically find and download files.
Back to top
View user's profile Send private message 
amigo

Joined: 02 Apr 2007
Posts: 2169

PostPosted: Sun 14 Oct 2012, 03:33    Post subject:  

http is not really searchable. You have to use wget to download an html page of the site and then parse entries out of that -a very tedious thing, indeed.
If you already know the name of the file, then you can simply use the exit status of the wget command to tell whether the download was successful. But if you need to dig through possible/unknown subdirectories, then you need to do as above suggested. ftp is also not really searchable, but there a couple of ways to get an 'ls' type listing of the site. Some sites accessed using http can also be accessed using ftp, but mileage varies greatly.
Back to top
View user's profile Send private message 
jpeps

Joined: 31 May 2008
Posts: 3219

PostPosted: Sun 14 Oct 2012, 05:08    Post subject:  

You can grab an index, and search it offline:

ls-lR.gz

http://archive.ubuntu.com/ubuntu/
Back to top
View user's profile Send private message 
sunburnt


Joined: 08 Jun 2005
Posts: 4982
Location: Arizona, U.S.A.

PostPosted: Mon 15 Oct 2012, 13:27    Post subject:  

Hey jpeps; I looked at that list and wow, what a mess! I have several other lists, one just files and one with a full description.
I can`t find where I got them, they don`t make it easy. Ultimately I need the download url for wget, which the full one gives.

Hi amigo; Parsing a download mirror`s index page for a listing isn`t so bad really. But the main site`s pages are a maze.
Ubuntu has libs in specific dirs., the apps. have specific libs in with them. This makes it a little troublesome to find the libs.


It would be really nice if sites had text files of their pages, but it`s obvious that Ubuntu wants only apt-get to auto. download.


### Qs: Is Slackware`s download site straight forward and sanely organized? And does it have all the files that Ubuntu has?
.
Back to top
View user's profile Send private message 
jpeps

Joined: 31 May 2008
Posts: 3219

PostPosted: Mon 15 Oct 2012, 17:25    Post subject:  

sunburnt wrote:
Hey jpeps; I looked at that list and wow, what a mess! I have several other lists, one just files and one with a full description.

everything is there; parse it.
Back to top
View user's profile Send private message 
sunburnt


Joined: 08 Jun 2005
Posts: 4982
Location: Arizona, U.S.A.

PostPosted: Mon 15 Oct 2012, 22:07    Post subject:  

jpeps; Apparently the Murga Puppy site won`t take attachments now...

I tried several times to post the file I spoke of but no luck.
Looking at the upload limits, it says 0 MB allowed! Laughing
My gzipped file is 6.8 MB, so it may be too big, I don`t know.
Back to top
View user's profile Send private message 
jpeps

Joined: 31 May 2008
Posts: 3219

PostPosted: Mon 15 Oct 2012, 22:15    Post subject:  

It's the entire database, so it's huge...unzipped 126M.
Back to top
View user's profile Send private message 
sunburnt


Joined: 08 Jun 2005
Posts: 4982
Location: Arizona, U.S.A.

PostPosted: Mon 15 Oct 2012, 22:49    Post subject:  

Here`s the Geany package`s info.:
Code:
Package: geany
Priority: optional
Section: universe/devel
Installed-Size: 6492
Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>
Original-Maintainer: Damián Viano <des@debian.org>
Architecture: i386
Version: 0.18-1
Depends: libatk1.0-0 (>= 1.20.0), libc6 (>= 2.4), libcairo2 (>= 1.2.4), libfontconfig1 (>= 2.4.0), libfreetype6 (>= 2.2.1), libgcc1 (>= 1:4.1.1), libglib2.0-0 (>= 2.18.0), libgtk2.0-0 (>= 2.17.7), libpango1.0-0 (>= 1.14.0), libstdc++6 (>= 4.1.1)
Suggests: libvte9, doc-base
Filename: pool/universe/g/geany/geany_0.18-1_i386.deb
Size: 2366426
MD5sum: 7bbd2f5f369e7cb4c1b335e2ef2139ce
SHA1: 796906f02afad38ac89043642cc4c9f114b89515
SHA256: 58010219836361d9d9f50afb53d9174c0896a3c0cdcf6efcbbe786e9d3e76c9d
Description: A fast and lightweight IDE
 Geany is a small and lightweight integrated development environment.
 It was developed to provide a small and fast IDE, which has only a
 few dependencies from other packages. It is using only the GTK2 toolkit
 and therefore you need only the GTK2 runtime libraries to run Geany.
 .
 The basic features of Geany are:
  - syntax highlighting
  - code completion
  - auto completion of constructs like if, for and while, XML and HTML
  - call tips
  - folding
  - many supported filetypes like C, Java, PHP, HTML, Python, Perl, Pascal
  - symbol lists
  - embedded terminal emulation
 .
Homepage: http://www.geany.org
Bugs: https://bugs.launchpad.net/ubuntu/+filebug
Origin: Ubuntu

It has the description which I need for the GUI, and the other data.
As you can see it`s a breeze to parse the data.

Unfortunately I cannot find where I got this file from!!!
.
Back to top
View user's profile Send private message 
jpeps

Joined: 31 May 2008
Posts: 3219

PostPosted: Tue 16 Oct 2012, 01:12    Post subject:  

Yes, but it doesn't tell you what's available. You can use "man geany" for a description of geany. Here's the database search for geany, 2012:

1936302 Mar 19 2012 geany-common_0.21.dfsg-1ubuntu4_all.deb
11739 Mar 19 2012 geany_0.21.dfsg-1ubuntu4.debian.tar.gz
2236 Mar 19 2012 geany_0.21.dfsg-1ubuntu4.dsc
1011790 Mar 19 2012 geany_0.21.dfsg-1ubuntu4_amd64.deb
858418 Mar 19 2012 geany_0.21.dfsg-1ubuntu4_armel.deb
856420 Mar 19 2012 geany_0.21.dfsg-1ubuntu4_armhf.deb
1016568 Mar 19 2012 geany_0.21.dfsg-1ubuntu4_i386.deb
994392 Mar 19 2012 geany_0.21.dfsg-1ubuntu4_powerpc.deb
4176730 Jan 5 2012 geany_0.21.dfsg.orig.tar.gz

If I search for an entry, then I get all the specifics on a download page:

http://pkgs.org/ubuntu-12.04/ubuntu-universe-i386/geany-common_0.21.dfsg-1ubuntu4_all.deb.html

Quote:

geany-common_0.21.dfsg-1ubuntu4_all.deb Download

geany-common - fast and lightweight IDE -- common files

Geany is a small and lightweight integrated development environment. It was developed to provide a small and fast IDE, which has only a few dependencies from other packages. It is using only the GTK2 toolkit and therefore you need only the GTK2 runtime libraries to run Geany.

The basic features of Geany are:

- syntax highlighting

- code completion

- auto completion of constructs like if, for and while, XML and HTML

- call tips

- folding

- many supported filetypes like C, Java, PHP, HTML, Python, Perl, Pascal

- symbol lists

- embedded terminal emulation

This package contains arch independent files.
...

Distribution: Ubuntu 12.04.1 LTS
Repository: Ubuntu Universe i386
Package name: geany-common
Package version: 0.21.dfsg-1ubuntu4
Binary package: geany-common_0.21.dfsg-1ubuntu4_all.deb
Source package: geany
Download size: 1,85 MB
Back to top
View user's profile Send private message 
sunburnt


Joined: 08 Jun 2005
Posts: 4982
Location: Arizona, U.S.A.

PostPosted: Tue 16 Oct 2012, 01:51    Post subject:  

jpeps; Interesting, but this is run from my GUI, there`s no download page.
Or I should say... The text file`s easier to parse than html as amigo mentioned.

The only difference is it shows other architectures. But the only one interesting is amd64.
All you need to do is substitute amd64 ( or any other ) for i386 and viola, a different arch.

Also if you haven`t downloaded the package, then there`s no man page to read, right?

There are about a dozen or two ways to do this that I`ve seen so far. Simple is good.

Here`s the Ubu. backend info. parser script I wrote today:
Code:
#!/bin/sh
##########   File Package Info.         Terry Becker   Oct. 15  2012

####   Usage:  . pkg-info (Pkg. Name)


[ "$1" ]|| exit

#=====================================================   Ubuntu Section.
[ "$2" ]|| Mirror=http://mirrors.us.kernel.org/ubuntu/
FileList=${0%/*}/Ubuntu_Universe-Pkgs.txt
                                                                              # find pkg.
StN=`grep -n -m1 "^Package: $1$" $FileList`
[ "$StN" = '' ]&& echo -e "\n###  ERROR:  Did Not Find Package:  $1\n" && exit
StN=${StN/:*}
Info=`sed $StN,$((StN+100))'!d' $FileList`

EndN=`echo "$Info" |grep -n -m1 '^Bugs:'`
EndN=$((${EndN/:*}-2))
Info=`echo "$Info" |sed 1,$EndN'!d'`
                                                                              # parse info.
PathPkg=`echo "$Info" |grep '^Filename:'`
PathPkg=${PathPkg#*: }
MD5sum=`echo "$Info" |grep '^MD5sum:'`
MD5sum=${MD5sum#*: }
Version=`echo "$Info" |grep '^Version:'`
Version=${Version#*: }
Arch=`echo "$Info" |grep '^Architecture:'`
Arch=${Arch#*: }
Depends=`echo "$Info" |grep '^Depends:'`
Depends=${Depends#*: }
PkgSize=`echo "$Info" |grep '^Size:'`
PkgSize=${PkgSize#*: }
InstSize=`echo "$Info" |grep '^Installed-Size:'`
InstSize=${InstSize#*: }
Desc=`echo "$Info" |sed -n '/Description/,$p'`
#=====================================================
Back to top
View user's profile Send private message 
RSH


Joined: 05 Sep 2011
Posts: 2036
Location: Germany

PostPosted: Wed 17 Oct 2012, 10:42    Post subject:  

Hi.

I don't know if this will help. The attached .tar.gz contains all directories for http://archive.ubuntu.com and http:://ubuntu.mirrors.pair.com

I have extracted this from the ls-lR.gz file.

RSH
Ubuntu-Archives-AllDirs.tar.gz
Description 
gz

 Download 
Filename  Ubuntu-Archives-AllDirs.tar.gz 
Filesize  731.07 KB 
Downloaded  360 Time(s) 

_________________
LazY Puppy
RSH's DNA
SARA B.
Back to top
View user's profile Send private message 
sunburnt


Joined: 08 Jun 2005
Posts: 4982
Location: Arizona, U.S.A.

PostPosted: Wed 17 Oct 2012, 22:34    Post subject:  

Hi RSH; I just posted after you in the SFS files thread too!

I`ll look at your attachment, the file I have is smaller than ls-lR by 100 MB!
This makes it much easier to include with my "package build and dnld." GUI.
It has most all the info. I need, but further along I may need to use ls-lR.

Again, ultimately what I need is the download URL for a given app`s exec.
Then my script runs "ldd" to get the execs. missing libs., and here the pkg.
list will give the dnld. URL and my script dnlds. them and checks ldd again.
Using ldd solves the duplication of libs. for any given host Linux O.S.

So for truly more portable AppPkg`s, use a minimal O.S. like Tiny Core.
This way most of the needed libs. would be included in the AppPkg.
I may make an option to include all libs., or allow selection from a list.
Back to top
View user's profile Send private message 
Display posts from previous:   Sort by:   
Page 1 of 1 [12 Posts]  
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » Off-Topic Area » Programming
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.0758s ][ Queries: 13 (0.0056s) ][ GZIP on ]