Normal Linux commands to Locate your system files, INSTANTLY

[slavvo67]

I think Find works just fine and even if my scripts take 10+ minutes to find all the files that I'm trying to extract, the customizations (as mentioned earlier in this thread) make it suitable for most needs.

It works fairly well for finding items by mime type (I do get false positives here), files changed or modified between certain dates and specific or all file extensions.

Always nice to try new things, though. Speed is always nice to have.

Best,

Slavvo67

[nic007]

If you don't want databases lying around and you like guis then searchmonkey I found to be as fast as anything I tried unless you go to gtk1 land .... content search and regular expressions too... invaluable for compiling I find to look for those 'lost' functions.

No bias...I did not write it...just tidied up the interface because I liked it

Indeed I turn off those indexing daemons in windows as they are a drag...and after all I am looking for stuff in unknown places like kernel sources, devx and app sources...NON of which would be in a recent database..... seems odd to search for stuff you already know about...but thats just me.

Mike

Indexing is turned off on my computer. "Everything" is lightning quick. When you start it the first time the database will obviously be built first (which doesn't take long). On my 80GB drive which is full (mostly mp3's), the data base is 300kb big. You don't notice the updates to the data base (too quick). I only start the programme when I need it so it does not run in the background. Makes no difference to the speed though.

[mikeb]

I suppose I only search for things now and then.... and filename searches seem fast anyway...a few seconds on a 1TB external drive. ...only content takes a while.

What is everyone losing?

mike

[nic007]

I have quite a large music collection. The songs of a particular artists may be all over the show in seperate directories/sub-directories. Very easy to find and play directly with Everything. Get results whilst typing so most of the time you already have what you are looking for after typing only a few characters. The search string is also very "wide" so if searching for a song like "under the moon of love" eg. entering moo lov will give instantaneous results (in fact the search results change as you are typing).

[mikeb]

How do we get a petition going? Smile

thanks for the chuckle

Je ne suis pas intelligent, je suis agile.
Il m'aide à éviter les dagues.

mike

[musher0]

How do we get a petition going? Smile

thanks for the chuckle

Je ne suis pas intelligent, je suis agile.
Il m'aide à éviter les dagues.

mike

[musher0]

Greetings to all members of the Puppy Brotherhood!

Here's a tid-bit to update the slocate database without breaking a sweat:

Code: Select all

#!/bin/sh
# $MBINS/SLocate/Slocate-updt.sh
# Purpose : update slocate's' files database.
# musher0, 2015-02-09
####
slocate -e /dev,/initrd,/lost+found,/proc./tmp -f vfat,ntfs,iso9660 -u

It excludes folders and files in /dev, /initrd, /lost+found, /proc. and /tmp, as well as
file systems vfat, ntfs, and iso9660, which makes the slocate database smaller, and
faster to search.

BFN.

musher0

[RetroTechGuy]

Commands which should be added back into the system's terminal commands: "updatedb" and "locatedb".

Linux commands whose benefits is explained here.

Over the years many of us have found our "find" command to be taking longer and longer. This is due to the very fact that more and more data is on our HDD/USB/SSD/etc storage media than ever before.

Many of us home users have music, family pictures, videos, documents, etc where our PUPs have been useful in our housing this content.

Its no wonder that locating data whose name and location we've forgot, is taking longer and longer.

Linux, for many years has addressed this, but, in PUPs beginnings, this was not considered.

Today, with so much data in the home we need every element of assistance to quickly find data, as is possible.

This is a formal request asking developers of WOOFCE/WOOFQ/WOOF/etc the PUP builder system to please add updatedblocatedb so that these can benefit users in finding files quickly in the system.

PUPPY developer consideration requested.

I do the "retro" style solution of periodically just using find to create an index file... Then just grep for the lost file (recent files I probably remember where I put them).

Code: Select all

find > listing.txt
grep -i lost.file listing.txt

[slavvo67]

Retrotechguy:

This is a nice little trick. I tried it on a mounted USB and it worked quite quickly. I'm wondering how it will do on a much larger drive so that'll be my next run.

Sometimes, simple and effective is good.

Thanks,

Slavvo67

[gcmartin]

Thanks @Retrotechguy:

Yes, over the years on 32bit PUPs except for1,I have on occasion resorted to this. It does work. When I have resorted is after, I have run "ls" or "find" back to back looking for something and having to wait what seems like eons for an answer to the terminal (every one of us has done this, we know how we hate waiting on results). The beauty is that this feature would be in every WOOFCE/WOOFQ buit PUP, OOTB, and no one would have to install or craft something on any PUP.

I only hope the Linux system commands are included in future PUPs to make instantaneous, file search results, as is found in other Linux distros and Lighthouse64 where these commands are present in the base system. Instantaneous as a part of system's normal behavior has advantages as its included in Linux's help system (both "man" command and "info" command)

The system doing the work has no noticeable impact on either system speed and there is almost no RAM impact that would impact applications ability.

Here's hoping the distro developers can weigh the advantage of putting this back into the Linux arsenal for our 2015 forward progress with all this data many of us are using.

AGain, thanks for that alternative, while we wait for development to also understand and consider the benefit of the Linux commands.

[RetroTechGuy]

Retrotechguy:

This is a nice little trick. I tried it on a mounted USB and it worked quite quickly. I'm wondering how it will do on a much larger drive so that'll be my next run.

Sometimes, simple and effective is good.

Thanks,

Slavvo67

I "cd" to the drive, so the index resides on the drive and travels with it.

On big drives, it does take some time, but the search will be faster than using a live search every time you want to find a different file... Drives have just gotten so darn big that it's hard to keep track...

And if you want, you can copy all of the listings to a single folder, and search all of the files at once (give them a filename that makes sense, so when it reports a file you know which file it was in, and thus which drive it is on).

[RetroTechGuy]

Thanks @Retrotechguy:

Yes, over the years on 32bit PUPs except for1,I have on occasion resorted to this. It does work. When I have resorted is after, I have run "ls" or "find" back to back looking for something and having to wait what seems like eons for an answer to the terminal

I'm not sure that "find" is as fast as ls (which requires "ls -r"), but it stores the entire pathname in the output, whereas "ls -r" doesn't (unless there is some parameter to make ls do so).

This gives a listing much as a MS Dos prompt "dir /r /b" which gives a "bare" listing, including full path.

With our big modern drives, unless the system has some chron job or widget running in the background to store all file changes in each system, the scan time is going to be a killer...

Hmmm... I wonder how much horsepower a memory resident task would consume, to watch for new files (and deletion of files)?... Such a tool would probably be your solution.

It could create a folder ".index" or some such, to "hide" the file(s) so they don't clutter the appearance of the drive.

[musher0]

(...)
With our big modern drives, unless the system has some chron job or widget running in the background to store all file changes in each system, the scan time is going to be a killer...

Hmmm... I wonder how much horsepower a memory resident task would consume, to watch for new files (and deletion of files)?... Such a tool would probably be your solution.

It could create a folder ".index" or some such, to "hide" the file(s) so they don't clutter the appearance of the drive.

Hi, RetroTechGuy.

Actually, as I said to mikeb on page 1, updating this slocate database takes a
looong second, two seconds at the most. So you may feel a "hic-cup"
on your system for a second ot two while the slocate db is being updated, but
that's it.

Now speaking of "automatic", I've borrowed a CLI sub-routine from French member
Médor for another little project (thanks are due to him). You can set it so this
"snapshot" is taken at the interval you want: ½ an hour, 1 hour, 2 hours... and it
is launched from the /root/Startup folder and runs in the background.

Takes very little resources, because it's not watching all the time, rather it takes
snapshots at intervals of x time only.

If you're interested, tell me. I should be able to adapt it to the slocate update
function in an hour or so.

BFN.

musher0

[gcmartin]

It appears some may not have known of this feature of Linux; namely "updatedb" command.

It is not necessary for anyone to do any alternate scripts or methods. Linux writers has done all the heavy lifting long before now. @Musher0, in an earlier post, expresses a linux command which uses the benefit via a terminal command; "slocate ..."

This is the reason for this to be added back into Puppy's version of Linux's commands. We are not taxing the system or writing any specialized scripts. Its merely a Linux command that is ONLY included by couple of PUPPY developers, today.

[musher0]

It appears some may not have known of this feature of Linux; namely "updatedb" command.

It is not necessary for anyone to do any alternate scripts or methods. Linux writers has done all the heavy lifting long before now. @Musher0, in an earlier post, expresses a linux command which uses the benefit via a terminal command; "slocate ..."

Yoo-hoo, gc!?

As I said before the updatedb program is incorporated in slocate.

Fading echos :
... the updatedb program is incorporated in slocate
....... the updatedb program is incorporated in slocate.
.......... the updatedb program is incorporated in slocate.
..............the updatedb program is incorporated in slocate.

That's why I chose it over the other two available choices.
You don't need a separate updatedb program with slocate, it's
incorporated / included / absorbed / digested into slocate as a symlink.

Capiche?

But to follow through on RetroTechGuy's suggestion, we'll still need a tiny script to
activate the slocate db fundtion at every x interval.

BFN

musher0

[gcmartin]

I understand. The reason for the repeats you've seen is so that WOOFCE/WOOFQ developers who would have the commands a part of the Linux builds could "choose" which they felt most at ease with; namely updatedb or slocate. Personally, I side with your choice of slocate as a single versus the combination updatedb-locate. But, in the past, Lighthouse has made available over the years, the latter.

Either approach could/would be a cron job.

Thanks, though for the clarity. Good idea.

[musher0]

Hello, file db lovers!

Here's a little script that will update your slocate database at every
you-decide-which-interval!:)

Please take a minute to read the comments in the script.
It'll all become clear how to use it. Thanks.

Besides the attached *.pet will do the default set-up for you
(with an update every 45 minutes).

Code: Select all

#!/bin/ash
# ash is used here because of its +/- 50 % greater speed than bash.
# Filename: /root/my-applications/bin/auto-updatedb-slocate.sh,
# with link in /root/Startup.
# Purpose: update your slocate database every x hour. or x minutes.
# Thanks to member Médor for having found the "while" time loop below.
# musher0, 12 Febr. 2015
####
sleep 1m # With this script, the slocate database will be updated
# exactly one minute after you began your Puppy session. After that
# it will be updated according to the interval you specify below, at
# at lines 19 to 23.

while [ 1 ]; do
	slocate -e "/proc,/dev/,/tmp,/initrd" -f "NFS,iso9660" -l0 -u

# Uncomment the one appropriate for your use.

# sleep 20m # every 20 minutes 
# sleep 30m # every 30 minutes
sleep 45m # every 45 minutes
# sleep 1h # every hour
# sleep 2h # every 2 hours
done

# From : auto_clean_mem, by Médor, 20150208. Source :
# http://murga-linux.com/puppy/viewtopic.php?p=826916&sort=lastpost#826916
# Retrieved 09.02.2015 13:43:49.
#

Enjoy!

musher0

[slavvo67]

I decided to take Retrotechguy's item a bit further. I simply call it drive index creator (in menu under "Business") or via terminal typing "index1".

1st you mount all the drives you wish to index and choose option 1. This creates the index.

Options 2 and 2a let you search the index file you created.

Option 3 lets you copy any of the sub-searches from 2. (root/chuck1)

Most in this thread can probably do better but I had fun making it.

Musher0 - Where is the actual Slocate program? Is it a pet somewhere?

Best,

Slavvo67

[amigo]

updatedb/slocate is not a matter of choosing one or the other -the two go together and must be used together. slocate *consults* the database which must first be created by updatedb and must be kept up-to-date using updatedb.

The only way one can get really quick results on a large filesystem is by indexing the contents -then the results can be retrieved nearly instantly. Without an index/db, then a tool like 'find' is the only way to get the information -and it will take about the same time to run as using updatedb.

I, like other here, then to use find -after cd-ing into the directory where I want to look. If I don't remember at all where something is, then that means starting the search from the next level up -or the next...

[gcmartin]

Installed this PET built by @StemSee for Emsee24

1. Opened a terminal, got this

   Code: Select all

   sh-4.3# slocate
   slocate: warning: Could not find the group: slocate in the /etc/group file.
   slocate: fatal error: This is a result of the group missing or a corrupted group file.
   

2. Updated system's group file

   Code: Select all

   sh-4.3# echo "slocate::114:root,spot,fido" >> /etc/group

3. Group file updated for command use in terminal, got this

   Code: Select all

   sh-4.3# slocate
   slocate: fatal error: Could not find user database '/var/lib/slocate/slocate.db':  No such file or directory

4. Ask slocate to update/create the database of files PUPPY can see (per recommendation from @Musher0)

   Code: Select all

   sh-4.3# slocate -u -e /dev,/initrd,/lost+found,/proc
   

5. NOW, EVERY SEARCH FOR FILESYSTEM FILE IS INSTANTANEOUS!

   Code: Select all

   sh-4.3# slocate *.log
       o    
       o
       o   
       o    
   

Request for a PET update

• Could this PET be updated adding to the group file when its installed and could it also run an initial "slocate -u" to create the initial database for users. By doing so, no user would pose a support question to the forum on this command when the system database is absent.

We can be hopeful that PUPs build by development via WOOFCE/WOOFQ, doing the following steps, for all new PUPs

• have slocate -u run at time of build or at initial pfix=ram of any PUP starts

Hope this dialogue is clear and accurate for community understanding and use.