You are not logged in.

#1 16 Jun 2018 13:21

JohnP
New Member
Registered: 16 Jun 2018
Posts: 2

Parse xml files and create symbolic links with results

Hello,

The goal is to create symbolic links using 2 xml databases. Both DDBB have the same structure, only two fields "file name" and "checksum".
As an example real.xml points to the actual files, the ones stored in HD(s) and softl.xml stores the paths, where the links and their paths have to be created, and their checksums to compare to the real.xml.

The script would have to get the hash from each line in softl.xml and compare it to the hash in real.xml, then create the path, if needed, and a symbolic link using softl.xml path related to the found hash.

real.xml

<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<db>
  <files>
    <file name="g:\real\abandoned_places-wallpaper-2560x1440.jpg" checksum="770319cee2c2ddfa5c47651c460e443677918433" />
    <file name="g:\real\autumn_37-wallpaper-2560x1440.jpg" checksum="2594cb5313fe01f577136d7bf9d42ebea5536b0e" />
    <file name="f:\real\bg 45\autumn_blossom-wallpaper    2560x1440.jpg" checksum="9af4911807b56713031871979e8710b07d5a732b" />
    <file name="f:\real\bg 45\blue_texture-wallpaper-2560x1440.jpg" checksum="5860f080c21002a6b7d55fe290d9faf598d1e8a5" />
    <file name="f:\real\bg2\armenia_tanahat_hayk_photography_abandoned-wallpaper-2560x1440.jpg" checksum="7e00714771da30e5f6f0929af94b8cccceca8b1f" />
    <file name="f:\real\bg2\back_in time-wallpaper-2560x1440.jpg" checksum="0fa63e654c98c80cca7a888bf6963dbc3318fb7e" />
    <file name="f:\real\bg2\CINEBENCHR15.038.zip" checksum="29c23f58c628b63df5099ff44378b501a8fa3b9e" />
    <file name="f:\real\ChecksumVerifier.exe" checksum="f7463fdd181aca1830d0ceba3674e396fa15a61d" />
  </files>
</db>

softl.xml

<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<db>
  <files>
    <file name="l:\softl\ChecksumVerifier.exe" checksum="f7463fdd181aca1830d0ceba3674e396fa15a61d" />
    <file name="l:\softl\file\here\abandoned_places-wallpaper-2560x1440.jpg" checksum="770319cee2c2ddfa5c47651c460e443677918433" />
    <file name="l:\softl\old\yeste rday\back in time.jpg" checksum="0fa63e654c98c80cca7a888bf6963dbc3318fb7e" />
    <file name="l:\softl\old\yeste rday\back no time\autumn_blossom-wallpaper.jpg" checksum="9af4911807b56713031871979e8710b07d5a732b" />
    <file name="D:\To be removed\old\yesterday images\back in time.jpg" checksum="0fa63e654c98c80cca7a888bf6963dbc3318fb7e" />
    <file name="l:\softl\old\yesterday images\back no time\autumn_blossom-wallpaper.jpg" checksum="9af4911807b56713031871979e8710b07d5a732b" />
    <file name="l:\softl\old\yesterday images\abandoned_places-wallpaper-2560x1440.jpg" checksum="770319cee2c2ddfa5c47651c460e443677918433" />
  </files>
</db>

xml ddbb are created using Nic Jansma github.com/nicjansma/checksum-verifier, tho it's not mandatory

Hope I made myself clear, any help is much appreciated.
Thanks for reading!
JP

Offline

#2 17 Jun 2018 08:05

Pyprohly
Member
Registered: 26 Nov 2014
Posts: 37

Re: Parse xml files and create symbolic links with results

A batch script wouldn’t be appropriate for this kind of task so I’m going to give you this PowerShell script instead.

It uses New-Item to create the symbolic link. One quirk with New-Item is that the symbolic link must point to a valid path for the link to be created. If this is an issue for you, you can instead use cmd.exe to create the link by uncommenting and commenting out the relevant parts.

#requires -RunAsAdmin

$InformationPreference = 'Continue'
$DebugPreference = 'Continue'

[xml]$source = Get-Content -LiteralPath real.xml
[xml]$target = Get-Content -LiteralPath softl.xml

foreach ($sourceEntry in $source.db.files.file) {
	if ($links = $target.db.files.file | where checksum -eq $sourceEntry.checksum) {
		foreach ($targetEntry in $links) {
			$path = Split-Path $targetEntry.name
			$name = Split-Path -Leaf $targetEntry.name
			if ($path | Push-Location -PassThru -ErrorAction SilentlyContinue) {
				$item = New-Item -Name $name -Type SymbolicLink -Value $sourceEntry.name
				if ($item) {
					Write-Debug "Created $item <==> $($sourceEntry.name)"
				}

				<#
				& cmd.exe /c "mklink ""$name"" ""$($sourceEntry.name)""" >'' 2>&1
				if ($LASTEXITCODE) {
					Write-Error "Could not create link $name"
				} else {
					Write-Debug "Created $item <==> $($sourceEntry.name)"
				}
				#>

				Pop-Location
			} else {
				Write-Warning "The path of $path ($name) could not be accessed"
			}
		}
	} else {
		Write-Information "No checksum match for $($sourceEntry.name)"
	}
}

Last edited by Pyprohly (18 Jun 2018 05:11)

Offline

#3 17 Jun 2018 11:36

JohnP
New Member
Registered: 16 Jun 2018
Posts: 2

Re: Parse xml files and create symbolic links with results

Hello Pyprohly,

Thanks a lot, it works great!, if I may I'd like to iron out a pair of minor issues:

1- from the output I presume the script does check all real.xml entries against softl.xml thus resulting in an ouput containing tons of lines when using real.xml in real life (this is the main database and will grow in time to +400k checksums). While softl.xml(s) are just files ranging from 1 to 100 entries.

Can it be reversed?, i mean that the output does warn about checksums in softl.xml not found inside real.xml. So then I'll know real.xml must be updated or a file(checksum) is completely missing.

No checksum match for f:\real\autumn_37-wallpaper-2560x1440.jpg
No checksum match for f:\real\bg 45\blue_texture-wallpaper-2560x1440.jpg
No checksum match for f:\real\bg2\armenia_tanahat_hayk_photography_abandoned-wallpaper-2560x1440.jpg
...and so on

2- about creating paths using cmd I still get the same output "the path could no be accessed", perhaps is something related to my system because the same happens when I try to make the symbolic link manually. Only after i created the path the softlink is able to made.
So I probably misunderstood you or I'm doing something wrong.

Screenshot_-_17_6_2018_12_29_19.png

Thanks again for your help!
JP

Offline

#4 18 Jun 2018 05:02

Pyprohly
Member
Registered: 26 Nov 2014
Posts: 37

Re: Parse xml files and create symbolic links with results

Making softl.xml the source file would certainly make it faster if real.xml is larger. Needs testing though; I wasn’t expect such large files and I haven’t worked with large files lately. If it takes too long then the xml files may need to be unravelled into a more efficient hashmap-like data structure.

The folder path still needs to exist for cmd.exe to create the symbolic link, of course. The difference between `mklink` and `New-Item` that I was trying to get at is that `New-Item` checks that the symbolic link data buffer points to a valid path before creating the link and will raise an error if it does not, whereas `mklink` will accept any arbitrary string whether or not it points to a valid location. This means you can’t pre-emptively create links and add the files later with `New-Item`.

Nice blood-shot command line btw. Way to get the eyes going.

Offline

Board footer

Powered by