Moving Directories in Linux w/ Rsync

[s243a]

If you try to move files using rox by dragging from one folder to another and it fails for some reason; for example:
1. I/O error (Maybe you have a flaky USB connection or drive)
2. Too Much Ram being used
3. Insufficient space in the destination drive.
4. Insufficient permissions
5. Power outage
6. etc.

You could end up with a mess where some of your files were transfered and others wern't. Worse yet ROX is horrible at merging directors and I'm not sure if it identifies when a file is only partially transfered.

The solution that I'm trying as we speak uses rsync to do this task instead of Rox but we'll do a few tricks to make it more GUI-Like.

I've created three scripts:

The first script is used to save the path to the source directory that we want to move files from. This file -- located in the /tmp directory -- will be read later to get the path.


/usr/sbin/s243a_Fm

Code: Select all

#!/bin/sh
OLD_IFS=IFS
IFS=""
WD="/tmp/s243a"
#https://stackoverflow.com/questions/793858/how-to-mkdir-only-if-a-dir-does-not-already-exist
[ -d $WD ] || mkdir -p $WD 
echo $(readlink -f "$1") > "$WD/fm.txt"
IFS="$OLD_IFS"

To call this script you can either navigate to the directory in ROX and either pres the tilda key "~" to get a console or alternatively make the script clickable by rightclicking on the director, selecting "open with", then selecting "customize"

This will open the following directory in Rox

/root/.config/rox.sourceforge.net/OpenWith

Now symbolically link this script to the directory by draging the script into this directory and selecting "link relative"

Next we create a similar script for the destination directory:
/usr/sbin/s243a_To

Code: Select all

#!/bin/sh
OLD_IFS=IFS
IFS=""
WD="/tmp/s243a"
#https://stackoverflow.com/questions/793858/how-to-mkdir-only-if-a-dir-does-not-already-exist
[ -d $WD ] || mkdir -p $WD 
echo $(readlink -f "$1") > "$WD/to.txt"
IFS="$OLD_IFS"

and also make it clickable.

Finally, having saved the source path and destinations path to a file we can use rsync to move the files between these two locations. The script that we will use to do this will look like:

Code: Select all

#!/bin/sh
OLD_IFS=IFS
IFS=""
WD="/tmp/s243a"
TO="`cat $WD/to.txt`"
FM="`cat $WD/fm.txt`" 
#
# https://unix.stackexchange.com/questions/78375/move-files-and-delete-directories-with-rsync
#
rsync -avE --size-only --remove-source-files "$FM" "$TO" && \
find "$FM"  -depth -type d  -empty -exec rmdir "{}" \;
IFS="$OLD_IFS"

We can also make this script clickable.

However, it is better to run it in the console so that you can watch the progress:

From any path in the console simply type

s243a_Rsync

and then the transfer will begin. Some notes on the options.

The -a is the archive option. It will preserver information about the file, such as timestamps, ownership and permissions. It will also do recursion.

I added the E option because I'm not sure if the -a option preserves execution permissions.

I added --size-only because I was thinking of the case where someone like me tried using rox to move a directory and it and failed. My theory was that if the size was different then the transfer wasn't completed and in this case we always wanted to take the original file if the size differed.

The --remove-source-files option removes files after they are synced but will not delete directories. The following line after rsync (i.e. the find command)

Code: Select all

find "$FM"  -depth -type d  -empty -exec rmdir "{}" \;

is only executed if rsync succeeds. This command will delete only empty directories from the source folder.

I suppose that one might want to do this anyway regardless of whether or not rsync succeeds but only after they are sure that all the files are moved. Perhaps we could create a fourth script to delete empty directories and also make it clickable.

I would like to add more options to this scipt and also add a gui.

I'd probably try GTK dialog for the GUI. It would work as follows:

When you type

Code: Select all

s243a_Rsync

A dialog opens up with the source and destination paths shown as previously set by the other two scripts. By each path there would be a button that you could press to select a different path. There would also be a checkboxes so that you could change some of the rsync options. For instance maybe I want to always skip a file if it exists in the destination folder. I might do this if I transfered some of the files to another folder and was trying to merge them back in but I didn't know if all of these files in this other folder were complete.

Another thing that we might want to try is a delete as we go option.

We could do this as follows:

With the verbose option selected in rsync it outputs to the console the name of each file that it moves (or syncs).

We could pipe this output into a script which watches these names and waits for a directory name change. Once a directory change is detected it would then empty the directory.

[musher0]

Interesting lesson about rsync. Thanks.

[slavvo67]

I didn't use your examples but I did successfully move a full WoofQ with your Rsync idea. The cp command just didn't bring everything over properly (links, etc.).

Thanks,

Slavvo67

[Flash]

Forum member don570 made a .pet called Copy-fast that uses rsync. I use Copy-fast all the time to move directories containing audio books to my mp3 player from flash drives.
Here's a topic about Copy-fast in the forum.