Help with Robocopy script to move *EXTRA files and directories

[FozzieBear]

I have been successfully using RoboCopy to backup various shares on my Windows 10 (Ver 22H2 Build 19045.4046) "Home" Server to a directory on an external eSATA drive. This directory is the OneDrive root folder. As a result any file or folder copied to this directory gets backed up to OneDrive.

(Note: I am aware some people do not consider OneDrive a "secure backup" however there is a local copy on the eSata drive and I also backup a second copy to another PC which also backs up to separate OneDrive account)

The problem with the way I have configured Robocopy is that if I change the source file name or location (edit a photo and rename it or move to its final destination folder) I end up with duplicate copies of the same file on OneDrive which is using up space.

This is shown in the log file as "*EXTRA Dir" or "*EXTRA File" in the destination folder. I don't want to MIRROR the source and destination in case I accidentally delete a file in the working directory, as that will also delete the backup copy.

What I am looking for is to add another line in the script which MOVES any EXTRA Files or EXTRA Directories, identified by Robocopy directory comparison, to another directory. I can then periodically review these and delete them if they are duplicates.

Below is my Robocopy script which copies four shares:

Robocopy "D:\Documents" "E:\Onedrive\Documents" *.* /E /COPYALL /XO /MT[:n] /LOG:"C:\Users\Admin\Desktop\documents.txt" /Z /R:3 /W:10

Robocopy "D:\Videos" "E:\Onedrive\Videos" *.* /E /COPYALL /XO /MT[:n] /LOG:"C:\Users\Admin\Desktop\videos.txt" /Z /R:3 /W:10

Robocopy "D:\Photos" "E:\Onedrive\Photos" *.* /E /COPYALL /XO /MT[:n] /LOG:"C:\Users\Admin\Desktop\photos.txt" /Z /R:3 /W:10

Robocopy "D:\Downloaded Photos" "E:\Downloaded Photos" *.* /E /COPYALL /XO /MT[:n] /LOG:"C:\Users\Admin\Desktop\Downloaded Photos.txt" /Z /R:3 /W:10

and an extract from one of the logs in this case Photos script:

*EXTRA Dir -1 E:\Onedrive\Photos\2006 04 29 Spring Ball 2006\
*EXTRA Dir -1 E:\Onedrive\Photos\2006 11 03 Jacky & Jimmy Golden Wedding\
*EXTRA Dir -1 E:\Onedrive\Photos\2007 04 08 Neil & Janies Engagment Party\
*EXTRA Dir -1 E:\Onedrive\Photos\2008 01 04 Ashleys Wedding\
*EXTRA Dir -1 E:\Onedrive\Photos\2011 06 04 Ryans Birthday Party\
*EXTRA Dir -1 E:\Onedrive\Photos\2011 07 23 Ryan watering flowers\
*EXTRA Dir -1 E:\Onedrive\Photos\2011 12 24 A414 Christmas Eve\
*EXTRA Dir -1 E:\Onedrive\Photos\2012 01 12 Ryan on Merry Go Round\
*EXTRA Dir -1 E:\Onedrive\Photos\2012 01 22 Ryan at Nannys in her T-Shirt\
*EXTRA Dir -1 E:\Onedrive\Photos\2012 02 14 Boys Aeroplane shelf\
*EXTRA Dir -1 E:\Onedrive\Photos\2012 03 24 Ryan on Grandads Lap\
*EXTRA Dir -1 E:\Onedrive\Photos\2012 03 30 Ryan & Hadleigh on Mummys Lap\
*EXTRA Dir -1 E:\Onedrive\Photos\2012 04 01 Hadleigh on Changing Mat\
*EXTRA Dir -1 E:\Onedrive\Photos\2012 04 06 Hadleigh asleep on Mummys knees\
*EXTRA File 615807 E:\Onedrive\Photos\01 August 2007.pdf
*EXTRA File 404986 E:\Onedrive\Photos\090794 (2).jpg
*EXTRA File 386182 E:\Onedrive\Photos\090794 (3).jpg
*EXTRA File 413042 E:\Onedrive\Photos\090794.jpg
*EXTRA File 2.6 m E:\Onedrive\Photos\2005 10 09 Friends Ball (1).jpg
*EXTRA File 2.3 m E:\Onedrive\Photos\2005 10 09 Friends Ball (2).jpg
*EXTRA File 815091 E:\Onedrive\Photos\2007 Hollwood Tower of Terror.jpg
*EXTRA File 9.9 m E:\Onedrive\Photos\2016 05 08_Sardinia Holiday_0187.JPG
*EXTRA File 38.5 m E:\Onedrive\Photos\2016 05 08_Sardinia Holiday_0188.MTS
*EXTRA File 1.5 m E:\Onedrive\Photos\2017 03 10 Judy, Lorraine, Angela (1).JPG
*EXTRA File 1.5 m E:\Onedrive\Photos\2017 03 10 Judy, Lorraine, Angela (2).JPG
*EXTRA File 1.9 m E:\Onedrive\Photos\2017-11-09 Sandra birthday (1).JPG
*EXTRA File 956567 E:\Onedrive\Photos\2017-11-09 Sandra birthday (2).JPG
*EXTRA File 1.9 m E:\Onedrive\Photos\2017-11-09 Sandra birthday (3).JPG
*EXTRA File 1.9 m E:\Onedrive\Photos\2017-11-28 Kealys birthday meal (1).JPG
*EXTRA File 1.9 m E:\Onedrive\Photos\2017-11-28 Kealys birthday meal (2).JPG
*EXTRA File 1.9 m E:\Onedrive\Photos\2017-11-28 Kealys birthday meal (3).JPG
*EXTRA File 1.9 m E:\Onedrive\Photos\2017-11-28 Kealys birthday meal (4).JPG
*EXTRA File 1.9 m E:\Onedrive\Photos\2017-11-28 Kealys birthday meal (5).JPG
*EXTRA File 1.9 m E:\Onedrive\Photos\2017-11-28 Kealys birthday meal (7).JPG
*EXTRA File 2.4 m E:\Onedrive\Photos\2017-12-03 Hadleigh school choir.JPG
*EXTRA File 1.4 m E:\Onedrive\Photos\2017-12-12 Hadleigh school play (1).JPG
*EXTRA File 1.3 m E:\Onedrive\Photos\2017-12-12 Hadleigh school play (2).JPG
*EXTRA File 2.2 m E:\Onedrive\Photos\2017-12-16 Christmas cake decorating at Nannys (1).JPG
*EXTRA File 2.3 m E:\Onedrive\Photos\2017-12-16 Christmas cake decorating at Nannys (2).JPG
*EXTRA File 3.3 m E:\Onedrive\Photos\2017-Oct half term with boys (1).JPG
*EXTRA File 2.8 m E:\Onedrive\Photos\2017-Oct half term with boys (2).JPG
*EXTRA File 3.8 m E:\Onedrive\Photos\2017-Oct half term with boys (3).JPG
*EXTRA File 1.9 m E:\Onedrive\Photos\2017-Oct half term with boys (4).JPG

So to clarify I want to add another script for each individual backup i.e. Documents, Pictures, Videos etc that MOVES all directories and files identified as EXTRA from the destination directory to another directory.
I am not fluent in command line or scripting as was never taught it but I have done a lot of reading and testing. I can follow and interpret instructions

As an aside when a switch in Robocopy "deletes" a file or folder is it permanently deleted or does it go into the recycle bin? The reason I ask is perhaps I could use occasionally run /Purge and then manually move the deleted files from the recycle bin to another location?
Any suggestions very much appreciated

[Simon Sheppard]

The problem with the approach you are describing is that I don't think it will result in a complete backup.

You are focussing on preserving copies of renamed or deleted files but there are lots of other situations when you may need an older copy of a file. e.g. if you edit a file and in the process corrupt it, you will no longer have the older copy to revert to.

Sometimes it takes months to discover you have some corrupted files.

A common use of ROBOCOPY is to make an on-disk copy of all the files and then use a backup solution to keep versions of those files, that could be as complex as multiple off-site tapes or if you have the disc space, as simple as renaming the folder with the day's date.

The 3-2-1 backup rule is:

• You create three copies of your data – the original data on your primary device and at least two copies.
• You use two different storage devices.
• You keep one of the backup copies off-site.

The thing to avoid is having all of the backup copies perfectly in sync, if that's the case then any corruption will be everywhere.

[FozzieBear]

Many thanks Simon,
I believe I am using the 3-2-1 backup rule. I have the original copy on the share (this is protected by Drivebender the equivalent of Raid 10) this is then copied by RoboCopy to an external eSATA hard drive. It is then backed up to Onedrive. (In addition I am also setting up another onsite backup using a seperate low power PC to copy direct from the source and thence to another Onedrive account). I know things go wrong but think that is pretty sound.

My problem is that the backup grows in size as there are duplicates of files. This occurs when my wife or I edit a file e.g. rename a photo to a meaningful title ( IMG2365 to Grandchildren on Sledge) and then move that photo from its current location to its final folder ("Downloaded Photos" to "2019 Grandchildren in Garden"). If I renamed all files immediately they are downloaded to the server then the problem would not occur however Robocopy runs nightly and will back up the original file and folder (Downloaded Photos/IMG2365) and then later the renamed file and folder (2019 Grandchildren in Garden\Grandchildren on Sledge). Robocopy flags the initial files and folders as *Extras.
I am trying to find out if a Robocopy or other script can move these "*EXTRA" files to another directory.
I am not overly worried about corrupted files. Word is fairly good at recovering documents and the only time I had corrupted files was a number of images where a hard drive died but hopefully I have planned against drive failure with DriveBender.

If Robocopy /Purge moves files to the recycle bin I could manually run a /Purge every few months and then move those files to another directory for a period until I delete them permanently. However I believe Robocopy erases the file permanently?

Perhaps another backup software XCopy, RichCopy might have more functionality to do what I need although I am sure a script could accomplish this but I dont have the skills to create one.

John