How to auto-skip duplicate files when merging? [SOLVED]

Filemanagers, partitioning tools, etc.
Post Reply
Message
Author
User avatar
Sky Aisling
Posts: 1368
Joined: Sat 27 Jun 2009, 23:02
Location: Port Townsend, WA. USA

How to auto-skip duplicate files when merging? [SOLVED]

#1 Post by Sky Aisling »

Hello, :)

The goal is to merge two directories of photos into one directory of photos and eliminate the duplicates.
There will be many duplicates during the merge.
Is there a batch merge program in Puppy that will auto-skip writing over the duplicate file?

I have found rmlint here.
http://murga-linux.com/puppy/viewtopic.php?t=95233
Anyone have experience with this program?
Suggestions for other .pets?

Thank you in advance for your response.
Last edited by Sky Aisling on Wed 28 Feb 2018, 17:01, edited 1 time in total.

User avatar
don570
Posts: 5528
Joined: Wed 10 Mar 2010, 19:58
Location: Ontario

#2 Post by don570 »

The following does the opposite i.e it will overwrite

cd into folder of images.....

Code: Select all

 cp -a -f --remove-destination * /path/destination/folder/ 
 sync

musher0
Posts: 14629
Joined: Mon 05 Jan 2009, 00:54
Location: Gatineau (Qc), Canada

#3 Post by musher0 »

HI Sky Aisling.

I am not all that familiar with rsync, but I think it can do that. Maybe have a look
at the man or docs, or find a matching topic at the stackoverflow forum or similar.

Rsync skips identical files that already exist in the destination dir., which makes
it very fast as a back-up tool. E.g. the following command copies only new files
in the ramdisk to its back-up dir. :

Code: Select all

rsync -avz . /mnt/home/ramdisk
. means that you have to be in the actual ramdisk when you issue the command

-a means "preserve file attributes"

-v means verbose, i.e. "display the filenames as the files are copied" -- (I find that
I need this parameter, it is not just a whim; otherwise, I will not know which files
have been copied)

-z means "compress the flow of the copy at the origin and decompress it at the
destination" (for greater speed).

~~~~~~~~~~

But as I said, I don't know rsync all that well. Please explore it in more detail.

IHTH
Last edited by musher0 on Thu 01 Mar 2018, 07:23, edited 1 time in total.
musher0
~~~~~~~~~~
"You want it darker? We kill the flame." (L. Cohen)

User avatar
Sky Aisling
Posts: 1368
Joined: Sat 27 Jun 2009, 23:02
Location: Port Townsend, WA. USA

How to auto-skip duplicate files during a directory merge?

#4 Post by Sky Aisling »

don570 and musher0

Thank you! Got it!

musher0
Posts: 14629
Joined: Mon 05 Jan 2009, 00:54
Location: Gatineau (Qc), Canada

#5 Post by musher0 »

My pleasure!
musher0
~~~~~~~~~~
"You want it darker? We kill the flame." (L. Cohen)

Post Reply