Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Sun 21 Dec 2014, 14:49
All times are UTC - 4
 Forum index » Advanced Topics » Cutting edge
Petch-0.3.pet -- CLI Puppy repository search and install
Moderators: Flash, Ian, JohnMurga
Post_new_topic   Reply_to_topic View_previous_topic :: View_next_topic
Page 2 of 5 Posts_count   Goto page: Previous 1, 2, 3, 4, 5 Next
Author Message
Caneri

Joined: 04 Sep 2007
Posts: 1580
Location: Canada

PostPosted: Tue 16 Sep 2008, 12:28    Post_subject:  

Hi Tom,

I wonder if adding puppyfiles.ca to the search in Petch would be helpful as well.

MU mirrors his dotpups.de there and if he is down then puppyfiles.ca is available. There are instructions on how to use puppyfiles.ca on the front page there from MU.

I'm not sure if there is a package list for puppyflies.ca though, but there must be as PSI can use it...Maybe Mark can answer that one.

Best,
Eric

_________________
Be not afraid to grow slowly, only be afraid of standing still.
Chinese Proverb

Back to top
View user's profile Send_private_message 
MU


Joined: 24 Aug 2005
Posts: 13644
Location: Karlsruhe, Germany

PostPosted: Tue 16 Sep 2008, 13:37    Post_subject:  

these folders have dotpups.txt, also in small .bz2 format:
http://puppyfiles.ca/dotpupsde/dotpups/
http://puppyfiles.ca/dotpupsde/puppy3/dotpups/

I just see, this one has none yet, I will add it this week:
http://puppyfiles.ca/dotpupsde/puppy4/dotpups/

Mark

_________________
my recommended links
Back to top
View user's profile Send_private_message Visit_website 
tombh


Joined: 12 Jan 2007
Posts: 422
Location: Bristol, UK

PostPosted: Thu 18 Sep 2008, 07:45    Post_subject:  

@ndujoe1: Didn't see yer post there before, thanks!

@Eric: Where should Petch live on .ca then? Puppyfiles.ca is already included, that's what I really mean when I say dotpups.de as I assumed that the former is a mirror of the latter.

@MU: Thanks, I already had http://puppyfiles.ca/dotpupsde/dotpups/ but not
http://puppyfiles.ca/dotpupsde/puppy3/dotpups/ so I added it.

_________________
Colour-Memories Database
Back to top
View user's profile Send_private_message Visit_website 
Pizzasgood


Joined: 04 May 2005
Posts: 6270
Location: Knoxville, TN, USA

PostPosted: Thu 18 Sep 2008, 12:27    Post_subject:  

Cool. This would be even better if we set up that attachment list, because then it could grab stuff from the forum too. What would be ideal in that respect is just a single .php page that ran an sql query for all .pet, .pup, etc. files (so we don't get things like conky.jpg screenshots).

I happen to have a (disabled) phpBB forum on my site, so I'll see about throwing something like that together this weekend. If I can get something more or less working on my end I can email it to John and see if he'll upload it to the server.

_________________
Between depriving a man of one hour from his life and depriving him of his life there exists only a difference of degree. --Muad'Dib

Back to top
View user's profile Send_private_message Visit_website 
cb88


Joined: 28 Jan 2007
Posts: 1169
Location: USA

PostPosted: Thu 18 Sep 2008, 13:00    Post_subject:  

thanks tombh this is a great development
_________________
Taking Puppy Linux to the limit of perfection. meanwhile try "puppy pfix=duct_tape" kernel parem eater.
X86: Sager NP6110 3630QM 16GB ram, Tyan Thunder 2 2x 300Mhz
Sun: SS2 , LX , SS5 , SS10 , SS20 ,Ultra 1, Ultra 10 , T2000
Mac: Platinum Plus, SE/30
Back to top
View user's profile Send_private_message Visit_website 
Lobster
Official Crustacean


Joined: 04 May 2005
Posts: 15122
Location: Paradox Realm

PostPosted: Thu 18 Sep 2008, 19:26    Post_subject:  

Works OK in Muppy
Should be in Puppy - is it too late for 4.1? Crying or Very sad

Certainly should be in 4.2 Cool

Brilliant!
This could be called from the package manager . . .

Code:
# petch xara
searching ...

       SOURCE        | DIRECTORY      | FILENAME (size)
[ 1] - puppylinux.ca | tpp            | Xara-XtremeV0.7r1785.pet (27447892)
Please type the number of the package you would like to fetch:


Tom before I forget
Don't think I can meet up in central London this weekend
but will see how it goes . . .
Hope you have a great trip

_________________
Puppy WIKI
Back to top
View user's profile Send_private_message Visit_website 
tombh


Joined: 12 Jan 2007
Posts: 422
Location: Bristol, UK

PostPosted: Fri 19 Sep 2008, 07:06    Post_subject:  

@Pizzasgood: Yes it would be even better wouldn't it. I imagine phpBB stores all its attachments in a sub-folder somewhere -- so if worse comes to worse I guess at the least we could just have a PHP script (without any SQL calls) make a daily list of, say all .pets, below a certain directory.

So for anyone that's interested I've attached the script that MU wrote to do this on dotpups.de and how Caneri now does it on puppylinux.ca too.

@cb88: Thanks.

@Lobster: I don't know if Barry will want it in his Puppy because it doesn't just search the official ibiblio repository. But he's very welcome if he does though!
No worries about meeting up Smile
cronpets.php.tar.gz
Description 
gz

 Download 
Filename  cronpets.php.tar.gz 
Filesize  1.02 KB 
Downloaded  414 Time(s) 

_________________
Colour-Memories Database
Back to top
View user's profile Send_private_message Visit_website 
HairyWill


Joined: 26 May 2006
Posts: 2949
Location: Southampton, UK

PostPosted: Sat 20 Sep 2008, 07:50    Post_subject:  

tom
how about getting petch to look for updated packages

for each installed package search for versions with a higher version number

If you call petch on an already installed package it could report the current installed version number.

_________________
Will
contribute: community website, screenshots, puplets, wiki, rss
Back to top
View user's profile Send_private_message 
Pizzasgood


Joined: 04 May 2005
Posts: 6270
Location: Knoxville, TN, USA

PostPosted: Mon 22 Sep 2008, 20:18    Post_subject:  

I looked at that file and dug up the one it creates at puppylinux.ca. Is there any particular reason for the second path entry for each item?


The forums attachments are all stored in the files/ subdirectory, but there's an index.php file there to block a file list from being generated by the server. So we could use something like what you posted, except that it wouldn't filter out packages that are inside a PM. I doubt that anybody would put something confidential inside a .pet package, but you never know. A more likely concern is if they put something inside a .tar.gz file, which I currently have this set to grab.

So I went ahead and wrote something that runs a couple simple queries so it can filter out attachments. Just making something to output a list is too simple, so I added a second mode to it. You can visit it with no options and have a more detailed list, with the file creation time, the attachment description, a link to the post it was posted in, the username of the person who uploaded it, and a link to that user's profile. That list also shows the package's original name, before the forum adjusts things to avoid overwriting existing packages.

The basic mode is accessed by providing the ?view=basic option in the URL. Then it outputs plain-text (set the header to text/plain so the browser treats it like a .txt file) formatted just like the dotpups.txt file on Caneri's server.

One thing to note is that the forum does rename a package after you upload it if the name would conflict with an existing package. It stores the original name, and under normal circumstances I suppose it copies the file to a temporary directory and renames it to that before letting you download it. Nether modes of this script bother with that. The fancy view will display the original name in the link, but the downloaded file uses the actual name the forum named it. The basic view only shows the actual name, since AFAIK I can't have it show the real name without messing up the format (unless this is the function of that duplicate path, which is the main reason I asked about it).


Right now it will grab any file with the following extensions, except those that are inside a PM: 'pet', 'pup', 'zip', 'tar', 'gz', 'rar', 'rpm', 'deb', 'tgz', 'tbz', 'bz2'


I don't have a demo up yet, because it turned out I needed to install the modifications that this forum is using, and I was too lazy to mess around with doing that remotely with my website. I do have a local server running that I tested it on, so I can attach some screenshots along with the actual script. I'll try to rig up a demo later tonight, or maybe tomorrow after I get done with my errands and homework.




Incidentally, the entry in the screenshot that says "anon" and "N.A." is an attached file that isn't associated with any post or PM. They happened accidentally when I was uploading something but didn't finish the post. Apparently the forum doesn't associate them with anything until the post/PM has been submitted, so they were stranded with no data in the DB to link them to a user. In the admin panel they turn up as belonging to "Guest" and can be manually deleted. There are also some buttons to "clean up" the forum, so I imagine one of them can probably deal with such items.
package_list.php.gz
Description  The php script
gz

 Download 
Filename  package_list.php.gz 
Filesize  1.58 KB 
Downloaded  352 Time(s) 
fancy.png
Description  The fancy view
png

 Download 
Filename  fancy.png 
Filesize  64.06 KB 
Downloaded  383 Time(s) 
basic.png
 Description   The basic view
 Filesize   30.14 KB
 Viewed   411 Time(s)

basic.png


_________________
Between depriving a man of one hour from his life and depriving him of his life there exists only a difference of degree. --Muad'Dib

Back to top
View user's profile Send_private_message Visit_website 
lluamco

Joined: 16 Mar 2007
Posts: 207
Location: Banyoles, Spain

PostPosted: Tue 23 Sep 2008, 06:11    Post_subject:  

Hi Tom.
Thank you for your nice and useful program! I have detected one problem, though. Pets located at dotpups.de do not appear. It seems that the source dotpups.de is not recognized by the search engine. Or am I misunderstanding something?
Thanks
Back to top
View user's profile Send_private_message 
tombh


Joined: 12 Jan 2007
Posts: 422
Location: Bristol, UK

PostPosted: Tue 23 Sep 2008, 06:38    Post_subject:  

@HairyWill: What a neat idea! I'll look into it Smile

@Pizzasgood: Umm, I can't see any reason for there being a second path entry for each item, that's just the way MU's original script is. At the moment though the search script takes the SOURCE field from the first path entry and the DIRECTORY field from the actual URL of the text file that all the entries are contained within (e.g. http://127.0.0.1/puppytest/package_list.php?view=basic). Therefore, I don't think the second path entry is actual used at all!

Well it looks great Smile

Maybe one thing to consider could be saving the output as a stand alone file, if the file was accessed a lot then it would incur less CPU load if PHP/SQL calls weren't being run all the time -- obviously then you'd need a cron job to update that text file every so often. But hopefully we can set it up so that Petch never calls your script directly, but calls a copy of it stored on puppylinux.org.

@lluamco: Thanks Smile Can you give some examples?

_________________
Colour-Memories Database
Back to top
View user's profile Send_private_message Visit_website 
lluamco

Joined: 16 Mar 2007
Posts: 207
Location: Banyoles, Spain

PostPosted: Tue 23 Sep 2008, 07:23    Post_subject:  

Hi again, Tom.
My observation started when using petch hplip
The output is

SOURCE | DIRECTORY | FILENAME (size)
[ 1] - puppylinux.ca | members | hplip_for_puppy_217-1.7.4a-i486.pet (46729952)
[ 2] - puppylinux.ca | members | hplip_for_puppy-1.7.4a.pet (32093140)
Please type the number of the package you would like to fetch:

only 2 very old packages appear, and none of the new ones located at

http://dotpups.de/puppy4/dotpups/Printing/hplip_for_dingo/
and its mirror
http://puppyfiles.ca/dotpupsde/puppy4/dotpups/Printing/hplip_for_dingo/
appear. However, afterwards I realized that MU is working on this particular puppy4/dotpups directory, so it should be good waiting... (thanks Canery!)

But then I tried also to petch cups, and I obtained this output:

SOURCE | DIRECTORY | FILENAME (size)
[ 1] - puppylinux.ca | puppyfiles | cups-1.1.23-1.pet (1606782)
[ 2] - puppylinux.ca | puppyfiles | cups-1.2.11.pet (1557475)
[ 3] - puppylinux.ca | puppyfiles | cups_espgs_gutenprint-1.1.23-8.15-5.0.0-2.pet (5118201)
[ 4] - puppylinux.ca | puppyfiles | cups_pdf-0.8.pet (8105)
[ 5] - puppylinux.ca | puppyfiles | cups_espgs_gutenprint-1.1.23-8.15-5.0.0-2.pet (1543728)
[ 6] - puppylinux.ca | puppyfiles | cups_espgs_gutenprint-1.1.23-8.15-5.0.0.pet (5118557)
[ 7] - puppylinux.ca | puppyfiles | cups_espgs_gutenprint_DEV-1.1.23-8.15-5.0.0.pet (1112408)
[ 8] - puppylinux.ca | puppyfiles | cups_pdf-0.2.pet (2215)
[ 9] - puppylinux.ca | puppyfiles | cups_pdf-0.3.pet (2388)
[10] - puppylinux.ca | puppyfiles | cups-1.1.23.pet (1175167)
[11] - puppylinux.ca | puppyfiles | cups-1.3.3.pet (1701989)
[12] - puppylinux.ca | puppyfiles | cups_pdf-0.8.pet (8173)
[13] - puppylinux.ca | puppyfiles | epdfview-0.1.6-4cups1123.pet (66464)
[14] - puppylinux.ca | puppyfiles | ghostscript-8.15.2-4cups1123.pet (1916649)
[15] - puppylinux.ca | puppyfiles | gtk+-2.12.1-4cups1123.pet (1498926)
[16] - puppylinux.ca | puppyfiles | gutenprint-5.0.1-4cups1123.pet (1658231)
[17] - puppylinux.ca | puppyfiles | libgnomecups-0.2.2-4cups1123.pet (19549)
[18] - puppylinux.ca | puppyfiles | libgnomecups-0.2.2.pet (19581)
[19] - puppylinux.ca | puppyfiles | libgnomeprint-2.18.2-4cups1123.pet (220590)
[20] - ibiblio | pet_packages-2 | cups_espgs_gutenprint-1.1.23-8.15-5.0.0-2.pet (1.5M)
[21] - ibiblio | pet_packages-2 | cups_espgs_gutenprint-1.1.23-8.15-5.0.0.pet (4.9M)
[22] - ibiblio | pet_packages-2 | cups_espgs_gutenprint_DEV-1.1.23-8.15-5.0.0.pet (1.1M)
[23] - ibiblio | pet_packages-2 | cups_pdf-0.2.pet (2.2K)
[24] - ibiblio | pet_packages-2 | cups_pdf-0.3.pet (2.3K)
[25] - ibiblio | pet_packages-4 | cups-1.1.23.pet (1.1M)
[26] - ibiblio | pet_packages-4 | cups-1.3.3.pet (1.6M)
[27] - ibiblio | pet_packages-4 | cups_pdf-0.8.pet (8.0K)
[28] - ibiblio | pet_packages-4 | epdfview-0.1.6-4cups1123.pet (65K)
[29] - ibiblio | pet_packages-4 | ghostscript-8.15.2-4cups1123.pet (1.8M)
[30] - ibiblio | pet_packages-4 | gtk+-2.12.1-4cups1123.pet (1.4M)
[31] - ibiblio | pet_packages-4 | gutenprint-5.0.1-4cups1123.pet (1.6M)
[32] - ibiblio | pet_packages-4 | libgnomecups-0.2.2-4cups1123.pet (19K)
[33] - ibiblio | pet_packages-4 | libgnomecups-0.2.2.pet (19K)
[34] - ibiblio | pet_packages-4 | libgnomeprint-2.18.2-4cups1123.pet (215K)
[35] - ibiblio | pet_packages-3 | cups-1.1.23-1.pet (1.5M)
[36] - ibiblio | pet_packages-3 | cups-1.2.11.pet (1.5M)
[37] - ibiblio | pet_packages-3 | cups_espgs_gutenprint-1.1.23-8.15-5.0.0-2.pet (4.9M)
[38] - ibiblio | pet_packages-3 | cups_pdf-0.8.pet (7.9K)
[39] - puppylinux.ca | tpp | cups_addon.pet (13406385)
[40] - puppylinux.ca | tpp | libpoppler-libcups-patch-1.00.pet (744693)

where NONE of the sources is dotpups.de. That is very strange, since there are cups packages in MU's repository.

Still another very minor detail. If you invoke petch without arguments, its output is
Usage: fetch [package to search for]
fecth should be petch, right?

Thanks again,
Lluis
Back to top
View user's profile Send_private_message 
Caneri

Joined: 04 Sep 2007
Posts: 1580
Location: Canada

PostPosted: Tue 23 Sep 2008, 09:21    Post_subject:  

Hi Lluis,

puppyfiles.ca here in Canada is the mirror for MU's dotpups.de in Germany and should be exactly the same as MU's main server.

When you see this
[ 1] - puppylinux.ca | puppyfiles | cups-1.1.23-1.pet (1606782)
this file is exactly the same on dotpups.de.

I'll let MU speak on that to be sure.

Petch is evolving so some directories/package lists will be updated in the near future...but tombh,MU will be the ones to respond to this.

Best,
Eric

_________________
Be not afraid to grow slowly, only be afraid of standing still.
Chinese Proverb

Back to top
View user's profile Send_private_message 
lluamco

Joined: 16 Mar 2007
Posts: 207
Location: Banyoles, Spain

PostPosted: Tue 23 Sep 2008, 11:40    Post_subject:  

Thanks Eric,
I was erroneously expecting to read dotpups.de as Source, my mistake.
Thanks again for clarifying and sorry for the noise...
Back to top
View user's profile Send_private_message 
Caneri

Joined: 04 Sep 2007
Posts: 1580
Location: Canada

PostPosted: Tue 23 Sep 2008, 14:01    Post_subject:  

Hi lluamco,

Questions are not noise but helpful.

Now I see that trying to pull a specific pet form puppyfiles.ca/dotpups.de is not working for me also.

My above post is wrong..the phrase [ 1] - puppylinux.ca | puppyfiles | cups-1.1.23-1.pet (1606782) is coming from puppylinux.ca/puppyfiles..sorry...hey I'll figure this out yet..Wink

Best,
Eric

PS I'll keep my mouth shut and listen for awhile..then I may be of use.

_________________
Be not afraid to grow slowly, only be afraid of standing still.
Chinese Proverb

Back to top
View user's profile Send_private_message 
Display_posts:   Sort by:   
Page 2 of 5 Posts_count   Goto page: Previous 1, 2, 3, 4, 5 Next
Post_new_topic   Reply_to_topic View_previous_topic :: View_next_topic
 Forum index » Advanced Topics » Cutting edge
Jump to:  

Rules_post_cannot
Rules_reply_cannot
Rules_edit_cannot
Rules_delete_cannot
Rules_vote_cannot
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.1425s ][ Queries: 13 (0.0435s) ][ GZIP on ]