Normal Linux commands to Locate your system files, INSTANTLY

[musher0]

How do we get a petition going? Smile

thanks for the chuckle

Je ne suis pas intelligent, je suis agile.
Il m'aide à éviter les dagues.

mike

[musher0]

Greetings to all members of the Puppy Brotherhood!

Here's a tid-bit to update the slocate database without breaking a sweat:

Code: Select all

#!/bin/sh
# $MBINS/SLocate/Slocate-updt.sh
# Purpose : update slocate's' files database.
# musher0, 2015-02-09
####
slocate -e /dev,/initrd,/lost+found,/proc./tmp -f vfat,ntfs,iso9660 -u

It excludes folders and files in /dev, /initrd, /lost+found, /proc. and /tmp, as well as
file systems vfat, ntfs, and iso9660, which makes the slocate database smaller, and
faster to search.

BFN.

musher0

[RetroTechGuy]

Commands which should be added back into the system's terminal commands: "updatedb" and "locatedb".

Linux commands whose benefits is explained here.

Over the years many of us have found our "find" command to be taking longer and longer. This is due to the very fact that more and more data is on our HDD/USB/SSD/etc storage media than ever before.

Many of us home users have music, family pictures, videos, documents, etc where our PUPs have been useful in our housing this content.

Its no wonder that locating data whose name and location we've forgot, is taking longer and longer.

Linux, for many years has addressed this, but, in PUPs beginnings, this was not considered.

Today, with so much data in the home we need every element of assistance to quickly find data, as is possible.

This is a formal request asking developers of WOOFCE/WOOFQ/WOOF/etc the PUP builder system to please add updatedblocatedb so that these can benefit users in finding files quickly in the system.

PUPPY developer consideration requested.

I do the "retro" style solution of periodically just using find to create an index file... Then just grep for the lost file (recent files I probably remember where I put them).

Code: Select all

find > listing.txt
grep -i lost.file listing.txt

[slavvo67]

Retrotechguy:

This is a nice little trick. I tried it on a mounted USB and it worked quite quickly. I'm wondering how it will do on a much larger drive so that'll be my next run.

Sometimes, simple and effective is good.

Thanks,

Slavvo67

[gcmartin]

Thanks @Retrotechguy:

Yes, over the years on 32bit PUPs except for1,I have on occasion resorted to this. It does work. When I have resorted is after, I have run "ls" or "find" back to back looking for something and having to wait what seems like eons for an answer to the terminal (every one of us has done this, we know how we hate waiting on results). The beauty is that this feature would be in every WOOFCE/WOOFQ buit PUP, OOTB, and no one would have to install or craft something on any PUP.

I only hope the Linux system commands are included in future PUPs to make instantaneous, file search results, as is found in other Linux distros and Lighthouse64 where these commands are present in the base system. Instantaneous as a part of system's normal behavior has advantages as its included in Linux's help system (both "man" command and "info" command)

The system doing the work has no noticeable impact on either system speed and there is almost no RAM impact that would impact applications ability.

Here's hoping the distro developers can weigh the advantage of putting this back into the Linux arsenal for our 2015 forward progress with all this data many of us are using.

AGain, thanks for that alternative, while we wait for development to also understand and consider the benefit of the Linux commands.

[RetroTechGuy]

Retrotechguy:

This is a nice little trick. I tried it on a mounted USB and it worked quite quickly. I'm wondering how it will do on a much larger drive so that'll be my next run.

Sometimes, simple and effective is good.

Thanks,

Slavvo67

I "cd" to the drive, so the index resides on the drive and travels with it.

On big drives, it does take some time, but the search will be faster than using a live search every time you want to find a different file... Drives have just gotten so darn big that it's hard to keep track...

And if you want, you can copy all of the listings to a single folder, and search all of the files at once (give them a filename that makes sense, so when it reports a file you know which file it was in, and thus which drive it is on).

[RetroTechGuy]

Thanks @Retrotechguy:

Yes, over the years on 32bit PUPs except for1,I have on occasion resorted to this. It does work. When I have resorted is after, I have run "ls" or "find" back to back looking for something and having to wait what seems like eons for an answer to the terminal

I'm not sure that "find" is as fast as ls (which requires "ls -r"), but it stores the entire pathname in the output, whereas "ls -r" doesn't (unless there is some parameter to make ls do so).

This gives a listing much as a MS Dos prompt "dir /r /b" which gives a "bare" listing, including full path.

With our big modern drives, unless the system has some chron job or widget running in the background to store all file changes in each system, the scan time is going to be a killer...

Hmmm... I wonder how much horsepower a memory resident task would consume, to watch for new files (and deletion of files)?... Such a tool would probably be your solution.

It could create a folder ".index" or some such, to "hide" the file(s) so they don't clutter the appearance of the drive.

[musher0]

(...)
With our big modern drives, unless the system has some chron job or widget running in the background to store all file changes in each system, the scan time is going to be a killer...

Hmmm... I wonder how much horsepower a memory resident task would consume, to watch for new files (and deletion of files)?... Such a tool would probably be your solution.

It could create a folder ".index" or some such, to "hide" the file(s) so they don't clutter the appearance of the drive.

Hi, RetroTechGuy.

Actually, as I said to mikeb on page 1, updating this slocate database takes a
looong second, two seconds at the most. So you may feel a "hic-cup"
on your system for a second ot two while the slocate db is being updated, but
that's it.

Now speaking of "automatic", I've borrowed a CLI sub-routine from French member
Médor for another little project (thanks are due to him). You can set it so this
"snapshot" is taken at the interval you want: ½ an hour, 1 hour, 2 hours... and it
is launched from the /root/Startup folder and runs in the background.

Takes very little resources, because it's not watching all the time, rather it takes
snapshots at intervals of x time only.

If you're interested, tell me. I should be able to adapt it to the slocate update
function in an hour or so.

BFN.

musher0

[gcmartin]

It appears some may not have known of this feature of Linux; namely "updatedb" command.

It is not necessary for anyone to do any alternate scripts or methods. Linux writers has done all the heavy lifting long before now. @Musher0, in an earlier post, expresses a linux command which uses the benefit via a terminal command; "slocate ..."

This is the reason for this to be added back into Puppy's version of Linux's commands. We are not taxing the system or writing any specialized scripts. Its merely a Linux command that is ONLY included by couple of PUPPY developers, today.

[musher0]

It appears some may not have known of this feature of Linux; namely "updatedb" command.

It is not necessary for anyone to do any alternate scripts or methods. Linux writers has done all the heavy lifting long before now. @Musher0, in an earlier post, expresses a linux command which uses the benefit via a terminal command; "slocate ..."

Yoo-hoo, gc!?

As I said before the updatedb program is incorporated in slocate.

Fading echos :
... the updatedb program is incorporated in slocate
....... the updatedb program is incorporated in slocate.
.......... the updatedb program is incorporated in slocate.
..............the updatedb program is incorporated in slocate.

That's why I chose it over the other two available choices.
You don't need a separate updatedb program with slocate, it's
incorporated / included / absorbed / digested into slocate as a symlink.

Capiche?

But to follow through on RetroTechGuy's suggestion, we'll still need a tiny script to
activate the slocate db fundtion at every x interval.

BFN

musher0

[gcmartin]

I understand. The reason for the repeats you've seen is so that WOOFCE/WOOFQ developers who would have the commands a part of the Linux builds could "choose" which they felt most at ease with; namely updatedb or slocate. Personally, I side with your choice of slocate as a single versus the combination updatedb-locate. But, in the past, Lighthouse has made available over the years, the latter.

Either approach could/would be a cron job.

Thanks, though for the clarity. Good idea.

[musher0]

Hello, file db lovers!

Here's a little script that will update your slocate database at every
you-decide-which-interval!:)

Please take a minute to read the comments in the script.
It'll all become clear how to use it. Thanks.

Besides the attached *.pet will do the default set-up for you
(with an update every 45 minutes).

Code: Select all

#!/bin/ash
# ash is used here because of its +/- 50 % greater speed than bash.
# Filename: /root/my-applications/bin/auto-updatedb-slocate.sh,
# with link in /root/Startup.
# Purpose: update your slocate database every x hour. or x minutes.
# Thanks to member Médor for having found the "while" time loop below.
# musher0, 12 Febr. 2015
####
sleep 1m # With this script, the slocate database will be updated
# exactly one minute after you began your Puppy session. After that
# it will be updated according to the interval you specify below, at
# at lines 19 to 23.

while [ 1 ]; do
	slocate -e "/proc,/dev/,/tmp,/initrd" -f "NFS,iso9660" -l0 -u

# Uncomment the one appropriate for your use.

# sleep 20m # every 20 minutes 
# sleep 30m # every 30 minutes
sleep 45m # every 45 minutes
# sleep 1h # every hour
# sleep 2h # every 2 hours
done

# From : auto_clean_mem, by Médor, 20150208. Source :
# http://murga-linux.com/puppy/viewtopic.php?p=826916&sort=lastpost#826916
# Retrieved 09.02.2015 13:43:49.
#

Enjoy!

musher0

[slavvo67]

I decided to take Retrotechguy's item a bit further. I simply call it drive index creator (in menu under "Business") or via terminal typing "index1".

1st you mount all the drives you wish to index and choose option 1. This creates the index.

Options 2 and 2a let you search the index file you created.

Option 3 lets you copy any of the sub-searches from 2. (root/chuck1)

Most in this thread can probably do better but I had fun making it.

Musher0 - Where is the actual Slocate program? Is it a pet somewhere?

Best,

Slavvo67

[amigo]

updatedb/slocate is not a matter of choosing one or the other -the two go together and must be used together. slocate *consults* the database which must first be created by updatedb and must be kept up-to-date using updatedb.

The only way one can get really quick results on a large filesystem is by indexing the contents -then the results can be retrieved nearly instantly. Without an index/db, then a tool like 'find' is the only way to get the information -and it will take about the same time to run as using updatedb.

I, like other here, then to use find -after cd-ing into the directory where I want to look. If I don't remember at all where something is, then that means starting the search from the next level up -or the next...

[gcmartin]

Installed this PET built by @StemSee for Emsee24

1. Opened a terminal, got this

   Code: Select all

   sh-4.3# slocate
   slocate: warning: Could not find the group: slocate in the /etc/group file.
   slocate: fatal error: This is a result of the group missing or a corrupted group file.
   

2. Updated system's group file

   Code: Select all

   sh-4.3# echo "slocate::114:root,spot,fido" >> /etc/group

3. Group file updated for command use in terminal, got this

   Code: Select all

   sh-4.3# slocate
   slocate: fatal error: Could not find user database '/var/lib/slocate/slocate.db':  No such file or directory

4. Ask slocate to update/create the database of files PUPPY can see (per recommendation from @Musher0)

   Code: Select all

   sh-4.3# slocate -u -e /dev,/initrd,/lost+found,/proc
   

5. NOW, EVERY SEARCH FOR FILESYSTEM FILE IS INSTANTANEOUS!

   Code: Select all

   sh-4.3# slocate *.log
       o    
       o
       o   
       o    
   

Request for a PET update

• Could this PET be updated adding to the group file when its installed and could it also run an initial "slocate -u" to create the initial database for users. By doing so, no user would pose a support question to the forum on this command when the system database is absent.

We can be hopeful that PUPs build by development via WOOFCE/WOOFQ, doing the following steps, for all new PUPs

• have slocate -u run at time of build or at initial pfix=ram of any PUP starts

Hope this dialogue is clear and accurate for community understanding and use.

[stemsee]

the pet already has all that code in /.pinstall.sh

But it is my first pinstall script maybe it should not be hidden just /pinstall.sh

For updating /etc/group file you should choose a number tht isn;t already used! 114 was just an educated guess.

[stemsee]

Try this

Edit: It Works!

[musher0]

(...)
Musher0 - Where is the actual Slocate program? Is it a pet somewhere?

Best,

Slavvo67

Hi, Slavvo67.

You can download the slackware package I used from the second URL listed at:
http://murga-linux.com/puppy/viewtopic. ... ost#826647
which is on the 1st page of the current thread.

As I mentioned in that post, I felt no need to create a pet package, at least on this
Puppy slacko, because my slacko-6.0b recognized the archive format and installed it
directly and instantly simply by clicking on the slocate.tgz archive.

Alll Puppies can install tgz packages directly, no? The capacity is built-in, in all
Puppies, like for *.deb archives. (I didn't do any extensive testing, but I think so.)

BFN.

musher0

[musher0]

updatedb/slocate is not a matter of choosing one or the other -the two go together and must be used together. slocate *consults* the database which must first be created by updatedb and must be kept up-to-date using updatedb.

The only way one can get really quick results on a large filesystem is by indexing the contents -then the results can be retrieved nearly instantly. Without an index/db, then a tool like 'find' is the only way to get the information -and it will take about the same time to run as using updatedb.

I, like other here, then to use find -after cd-ing into the directory where I want to look. If I don't remember at all where something is, then that means starting the search from the next level up -or the next...

Amigo,

As I kept explaining to gcmartin for a little while, the peculiarity of the slocate
package is that the updatedb is built-in. With slocate you don't need a separate
updatedb utility. The update routine is within the slocate program.

Please read my previsous posts before more confusion sets in?
I mean "read" as in "actually reading" ? Thanks.

musher0

[amigo]

Yes, Musher, of course the slocate package also contains the program updatedb. What I was trying to point out is that this:
"SEARCH FOR FILESYSTEM FILE IS INSTANTANEOUS"
is not accurate. slocate does not search the 'filesystem' -it searches the database created by using 'updatedb' or 'slocate -u'

If your run either of those commands and then add a file or files to the filesystem, these new items will not show up in the slocate 'search' since they are not in the database. slocate is only useful and accurate when the database is kept up-to-date.