SS64 Forum

nder

Member

Registered: 26 Oct 2020

Posts: 3

moving specific file types from many subfolder to 1 major folder

hi guys i have backup files that has been created over the years and in each backup there is similar files and newer files and now i have stopped backing up to my PC i would like to compile all the files to to a major folder WITHOUT any sub sub folders and overwrite/delete duplicate files.
i have over 200k files with about 80gb size which are duplicates and new ones. i have tried robocopy but it copied the whole directory into the new folder (luckily deleted the old one) 
can anyone help me. its taking very long to transfer using the search command.

basically i have created a specific new major folder according to the type (pictures/audio/videos)
i need all the files( specific files; jpg/png/jpeg etc) from the backup folders which has multiple level to be transferred to this major folders and any duplicate should be deleted

bluesxman

Member

From: UK

Registered: 29 Dec 2006

Posts: 1,129

Re: moving specific file types from many subfolder to 1 major folder

Whilst you might not care about duplicates, you will still need to decide how you want to handle them (IE which copy to keep/delete).

This should move files from the source to target but takes a "safe" route by trying to create a unique name for duplicates, based on updated date (**UNTESTED**):

set "SOURCE=X:\Source\Dir"
set "TARGET=X:\Target\Dir"

cd /d "%SOURCE%" || exit /b 1

mkdir "%TARGET%" || exit /b 1

for /f "usebackq tokens=* delims=" %%a in (`dir /b/s/a:-d *.jpg *.png *.jpeg`) do (
  if not exist "%TARGET%\%%~nxa" (
    move "%%~a" "%TARGET%\%%~nxa"
  ) ELSE (
    call :safe_move "%%~a"
  )
)

goto :EOF

:safe_move

set STAMP="%~t1"
REM might need to strip additional non-file-safe characters here
set STAMP="%STAMP: =_%"
set STAMP="%STAMP::=_%"
set STAMP="%STAMP:/=_%"

set TFILE="%TARGET%\%~n1_%STAMP%%~x1"

if not exist "%TFILE%" (
  move "%~1" "%TFILE%"
) ELSE (
  echo:Clashing safe target: "%~1" to "%TFILE%" >&2
)

goto :EOF

If you really don't care which duplicate should be kept, you could just skip the call :safe_move [...] part.

NB -- this could operate slowly if your source and target locations are on different drives.

Last edited by bluesxman (29 Oct 2020 12:52)

---

cmd | *sh | ruby | chef

Simon Sheppard

Admin

Registered: 27 Aug 2005

Posts: 1,130

Website

Re: moving specific file types from many subfolder to 1 major folder

It's not clear to me from the question whether the file structure is just from the backup routine or from a standard hierarchical file system

if you have
\cookery\book1.xls
\finances\book1.xls

you probably don't want to just overwrite and keep the most recent.

if you have
\backup1\finances\book1.xls
\backup2\finances\book1.xls

Then overwriting might be a good approach.

nder

Member

Registered: 26 Oct 2020

Posts: 3

Re: moving specific file types from many subfolder to 1 major folder

Hello there thank you so much will try this soon. tired of using the search and moving the files into mega folder

Whilst you might not care about duplicates, you will still need to decide how you want to handle them (IE which copy to keep/delete).

This should move files from the source to target but takes a "safe" route by trying to create a unique name for duplicates, based on updated date (**UNTESTED**):

set "SOURCE=X:\Source\Dir"
set "TARGET=X:\Target\Dir"

cd /d "%SOURCE%" || exit /b 1

mkdir "%TARGET%" || exit /b 1

for /f "usebackq tokens=* delims=" %%a in (`dir /b/s/a:-d *.jpg *.png *.jpeg`) do (
  if not exist "%TARGET%\%%~nxa" (
    move "%%~a" "%TARGET%\%%~nxa"
  ) ELSE (
    call :safe_move "%%~a"
  )
)

goto :EOF

:safe_move

set STAMP="%~t1"
REM might need to strip additional non-file-safe characters here
set STAMP="%STAMP: =_%"
set STAMP="%STAMP::=_%"
set STAMP="%STAMP:/=_%"

set TFILE="%TARGET%\%~n1_%STAMP%%~x1"

if not exist "%TFILE%" (
  move "%~1" "%TFILE%"
) ELSE (
  echo:Clashing safe target: "%~1" to "%TFILE%" >&2
)

goto :EOF

If you really don't care which duplicate should be kept, you could just skip the call :safe_move [...] part.

NB -- this could operate slowly if your source and target locations are on different drives.

nder

Member

Registered: 26 Oct 2020

Posts: 3

Re: moving specific file types from many subfolder to 1 major folder

i was using samsung smart phone and every time i backed up to my PC it created a new back up folder but the content was usually the same except for the new files. so literally the backup was not overwriting and updating the backup folder. and at the end i would have duplicates (which i want to avoid)

It's not clear to me from the question whether the file structure is just from the backup routine or from a standard hierarchical file system

if you have
\cookery\book1.xls
\finances\book1.xls

you probably don't want to just overwrite and keep the most recent.

if you have
\backup1\finances\book1.xls
\backup2\finances\book1.xls

Then overwriting might be a good approach.

Simon Sheppard

Admin

Registered: 27 Aug 2005

Posts: 1,130

Website

Re: moving specific file types from many subfolder to 1 major folder

So why can't you just use the most recent backup folder and ignore the rest?

If you combine older backups and new ones you will be restoring files that were once deleted and for any which have been renamed, you will end up with both copies.