Normal Linux commands to Locate your system files, INSTANTLY

[gcmartin]

I understand. The reason for the repeats you've seen is so that WOOFCE/WOOFQ developers who would have the commands a part of the Linux builds could "choose" which they felt most at ease with; namely updatedb or slocate. Personally, I side with your choice of slocate as a single versus the combination updatedb-locate. But, in the past, Lighthouse has made available over the years, the latter.

Either approach could/would be a cron job.

Thanks, though for the clarity. Good idea.

[musher0]

Hello, file db lovers!

Here's a little script that will update your slocate database at every
you-decide-which-interval!:)

Please take a minute to read the comments in the script.
It'll all become clear how to use it. Thanks.

Besides the attached *.pet will do the default set-up for you
(with an update every 45 minutes).

Code: Select all

#!/bin/ash
# ash is used here because of its +/- 50 % greater speed than bash.
# Filename: /root/my-applications/bin/auto-updatedb-slocate.sh,
# with link in /root/Startup.
# Purpose: update your slocate database every x hour. or x minutes.
# Thanks to member Médor for having found the "while" time loop below.
# musher0, 12 Febr. 2015
####
sleep 1m # With this script, the slocate database will be updated
# exactly one minute after you began your Puppy session. After that
# it will be updated according to the interval you specify below, at
# at lines 19 to 23.

while [ 1 ]; do
	slocate -e "/proc,/dev/,/tmp,/initrd" -f "NFS,iso9660" -l0 -u

# Uncomment the one appropriate for your use.

# sleep 20m # every 20 minutes 
# sleep 30m # every 30 minutes
sleep 45m # every 45 minutes
# sleep 1h # every hour
# sleep 2h # every 2 hours
done

# From : auto_clean_mem, by Médor, 20150208. Source :
# http://murga-linux.com/puppy/viewtopic.php?p=826916&sort=lastpost#826916
# Retrieved 09.02.2015 13:43:49.
#

Enjoy!

musher0

[slavvo67]

I decided to take Retrotechguy's item a bit further. I simply call it drive index creator (in menu under "Business") or via terminal typing "index1".

1st you mount all the drives you wish to index and choose option 1. This creates the index.

Options 2 and 2a let you search the index file you created.

Option 3 lets you copy any of the sub-searches from 2. (root/chuck1)

Most in this thread can probably do better but I had fun making it.

Musher0 - Where is the actual Slocate program? Is it a pet somewhere?

Best,

Slavvo67

[amigo]

updatedb/slocate is not a matter of choosing one or the other -the two go together and must be used together. slocate *consults* the database which must first be created by updatedb and must be kept up-to-date using updatedb.

The only way one can get really quick results on a large filesystem is by indexing the contents -then the results can be retrieved nearly instantly. Without an index/db, then a tool like 'find' is the only way to get the information -and it will take about the same time to run as using updatedb.

I, like other here, then to use find -after cd-ing into the directory where I want to look. If I don't remember at all where something is, then that means starting the search from the next level up -or the next...

[gcmartin]

Installed this PET built by @StemSee for Emsee24

1. Opened a terminal, got this

   Code: Select all

   sh-4.3# slocate
   slocate: warning: Could not find the group: slocate in the /etc/group file.
   slocate: fatal error: This is a result of the group missing or a corrupted group file.
   

2. Updated system's group file

   Code: Select all

   sh-4.3# echo "slocate::114:root,spot,fido" >> /etc/group

3. Group file updated for command use in terminal, got this

   Code: Select all

   sh-4.3# slocate
   slocate: fatal error: Could not find user database '/var/lib/slocate/slocate.db':  No such file or directory

4. Ask slocate to update/create the database of files PUPPY can see (per recommendation from @Musher0)

   Code: Select all

   sh-4.3# slocate -u -e /dev,/initrd,/lost+found,/proc
   

5. NOW, EVERY SEARCH FOR FILESYSTEM FILE IS INSTANTANEOUS!

   Code: Select all

   sh-4.3# slocate *.log
       o    
       o
       o   
       o    
   

Request for a PET update

• Could this PET be updated adding to the group file when its installed and could it also run an initial "slocate -u" to create the initial database for users. By doing so, no user would pose a support question to the forum on this command when the system database is absent.

We can be hopeful that PUPs build by development via WOOFCE/WOOFQ, doing the following steps, for all new PUPs

• have slocate -u run at time of build or at initial pfix=ram of any PUP starts

Hope this dialogue is clear and accurate for community understanding and use.

[stemsee]

the pet already has all that code in /.pinstall.sh

But it is my first pinstall script maybe it should not be hidden just /pinstall.sh

For updating /etc/group file you should choose a number tht isn;t already used! 114 was just an educated guess.

[stemsee]

Try this

Edit: It Works!

[musher0]

(...)
Musher0 - Where is the actual Slocate program? Is it a pet somewhere?

Best,

Slavvo67

Hi, Slavvo67.

You can download the slackware package I used from the second URL listed at:
http://murga-linux.com/puppy/viewtopic. ... ost#826647
which is on the 1st page of the current thread.

As I mentioned in that post, I felt no need to create a pet package, at least on this
Puppy slacko, because my slacko-6.0b recognized the archive format and installed it
directly and instantly simply by clicking on the slocate.tgz archive.

Alll Puppies can install tgz packages directly, no? The capacity is built-in, in all
Puppies, like for *.deb archives. (I didn't do any extensive testing, but I think so.)

BFN.

musher0

[musher0]

updatedb/slocate is not a matter of choosing one or the other -the two go together and must be used together. slocate *consults* the database which must first be created by updatedb and must be kept up-to-date using updatedb.

The only way one can get really quick results on a large filesystem is by indexing the contents -then the results can be retrieved nearly instantly. Without an index/db, then a tool like 'find' is the only way to get the information -and it will take about the same time to run as using updatedb.

I, like other here, then to use find -after cd-ing into the directory where I want to look. If I don't remember at all where something is, then that means starting the search from the next level up -or the next...

Amigo,

As I kept explaining to gcmartin for a little while, the peculiarity of the slocate
package is that the updatedb is built-in. With slocate you don't need a separate
updatedb utility. The update routine is within the slocate program.

Please read my previsous posts before more confusion sets in?
I mean "read" as in "actually reading" ? Thanks.

musher0

[amigo]

Yes, Musher, of course the slocate package also contains the program updatedb. What I was trying to point out is that this:
"SEARCH FOR FILESYSTEM FILE IS INSTANTANEOUS"
is not accurate. slocate does not search the 'filesystem' -it searches the database created by using 'updatedb' or 'slocate -u'

If your run either of those commands and then add a file or files to the filesystem, these new items will not show up in the slocate 'search' since they are not in the database. slocate is only useful and accurate when the database is kept up-to-date.

[musher0]

Yes, Musher, of course the slocate package also contains the program updatedb. What I was trying to point out is that this:
"SEARCH FOR FILESYSTEM FILE IS INSTANTANEOUS"
is not accurate. slocate does not search the 'filesystem' -it searches the database created by using 'updatedb' or 'slocate -u'

If your run either of those commands and then add a file or files to the filesystem, these new items will not show up in the slocate 'search' since they are not in the database. slocate is only useful and accurate when the database is kept up-to-date.

True. Sorry for the misunderstanding.

[gcmartin]

Our biggest item is how does the command(s) get into PUPs terminal commandset as a standard item. PETs are useful in setup but, if its in the PUP's commandset, then users or distro developers can implement as they see proper need and use.

Not sure how to approach this, now that the community, here, has shown its benefit in use.

This is NOT a special program developed. This is a LINUX command, missing in PUPs

[musher0]

Hi, everyone.

Indeed, it's blazingly fast. Another feature, for me anyway, is that it is accessible
from the initial console. That may sound like nothing, but yesterday, slocate
helped me find my way around on the initial console. (I was wondering where I
had put a new wm I wanted to test.)

In any case, the slocate utility is an all-around plus for PuppyLinux. If it had been
in Puppy when I started, It would have saved me hours of feeling and groping
around the system.

It's worth having slocate in every Puppy and explaining its use to newbies. It'll
save them countless time and frustrations.

BFN.

musher0

[RetroTechGuy]

I decided to take Retrotechguy's item a bit further. I simply call it drive index creator (in menu under "Business") or via terminal typing "index1".

I'm still digesting what these tools do...

I have a lot of USB drives (HDD and flash), and usually don't leave stuff plugged in.

I'm thinking a combination of the indexxing tool here, and a better front-end to access the stored databases. It appears that this and slocate both create a single index file in /root/.

Since any given drive might be plugged in, in a random order, it would make sense for each drive to have its own database, and have those collated by the graphical or command line tool when plugged in (the tool can store the list of the file tree, and identify which mount location when it reports the search results).

[smokey01]

By default it creates a database called slocate.db in /var/lib/slocate

By having drives mounted does not necessarily mean they will be indexed, well not here anyway. The only drive/s that seems to get indexed are / and /mnt/home

How do you make it index all drives?

[musher0]

By default it creates a database called slocate.db in /var/lib/slocate

By having drives mounted does not necessarily mean they will be indexed, well not here anyway. The only drive/s that seems to get indexed are / and /mnt/home

How do you make it index all drives?

Hi, Smokey.

Look in the readme. It explains a way to create secondary db's. You probably can
create a file db for each thumb-drive and have it reside on it. (Or not!)

BFN.

musher0

[Uten]

I used to use locate when I started out on linux. But when I changed to a livecd distro and as my number of removable disks grew I simply did not find locate to be the right tool.

Using find and grep is my choice these days. I do something like this

Code: Select all

time nice find /mnt/hda1/ > /mnt/hda1/locate.db
time nice find /mnt/hdd2/ > /mnt/hdd2/locate.db
time nice find /root/ > /root/locate.db

Building the locate.db files obviously takes some time. The time spent depends on the disk IO speed, but this is also true when you use updatedb. I use nice in front of find to be able to do other work.


And I search with something like this:

Code: Select all

export locatedbs="/mnt/hda1/locate.db /mnt/hdd2/locate.db /root/locate.db"
grep hurt.*[.]mp3$ $locatedbs

It takes about 1sek to search 1.000.000 entries in the locate.db files on my pentium 4 (hmm, when the locate.db are in /tmp, on slow disks it will take a bit longer).


To update the locate.db files we can use find -cnewer and such, but usually I don't bother

Sometimes I wrap it all up in two nice scripts, depending on how much I think I will use it before a reinstall.

)

[slavvo67]

Enhanced Drive Index Creator. I jumped to Drive Index Creator - 3.

Again, just for kicks but I improved to be able to search and then search the sub. I'll probably go one more round and just let the user choose which directory to move the files to.

Best,

Slavvo67

[smokey01]

Hi musher0,

I'm not interested in indexing usb flash drives but I would like to index all of my main drives and partitions into one large.db file. This way I can find files even if the drives are not mounted. I guess drives/partitions would have to be mounted to index them though. It would be easy enough to write a script to mount each drive/partition, index it then unmount it at each reboot. It probably could be re-indexed more often but that would do me.

My problem is I have lots of drives/partitions and often have problems finding files. Having to search through many different .db files would be a pain or less help than seems possible.

I read the read.me but I failed to fully understand it.

Care to share some code to address my requirement.

Thanks

By default it creates a database called slocate.db in /var/lib/slocate

By having drives mounted does not necessarily mean they will be indexed, well not here anyway. The only drive/s that seems to get indexed are / and /mnt/home

How do you make it index all drives?

Hi, Smokey.

Look in the readme. It explains a way to create secondary db's. You probably can
create a file db for each thumb-drive and have it reside on it. (Or not!)

BFN.

musher0

[smokey01]

What is the latest source available for slocate?

The latest I could find was 3.1 with a patch to make it 3.1-1 but slocate --version still reports it at 3.1.

I'm running it of FD700RC and it's verrrrry fast.