Update to 2023, double directories in database

[Triplefin]

Hi there,

I upgraded from the 2021 to 2023 version of imatch yesterday.
Initially it worked as expected. However, when imatch opened it displayed a directory on one drive twice.

One of the two (let's call it "dirA") showed the logo to be updated or reimported. It contained 16 out of the 17 subdirectories with two of them also showed the logo to be updated.

The other double directory (Let's call it "dirB") didn't have any indication for a need to be updated. But it contained only 3 of the 17 subdirectories on the harddrive.

Both dirA and dirB in the database have the exact same name and are located on the same harddrive.
When double checking the harddrive there was only one directory as expected with all subdirectories.

I run database diagnosis (no error), started to update dirA and hoped that imatch would recognize the two directories in the database as the one present.

Then imatch spent several hours importing the 14 subdirs missing in dirB although I asked to update dirA. Dir B was complete then, dirA was left unchanged.

All newly imported image files in dirB didn't have any of the numerous categories I assigned in imatch 2021.
The files in dirA still had the previously assigned categories though.

I then moved (drag and drop) one subdirectory from dirA to dirB wondering what would happen as they are actually the identical files on the harddrive. To my surprise imatch moved the files in the database. The subdirectory in dirB then had every file twice (same filename, same content, one with and one without the categories assigned).
Next I ran database diagnosis and updated the subdirectory in dirB. It removed the double entries but unfortunately removed those with categories assigned .

I then tried to move one subdirectory from dirB to dirA. That caused imatch to almost complete the moving but hang at the very end. I waited one hour and forced imatch to close then.
Upon restart it took quite long to load the database. The directory I tried to move was gone (also on the harddrive). I found it in the deleted items folder and could retrieve it from there.

So far I assume that dirB is the database entry corresponding to the real files on the harddrive.


What can I do to get a database that includes my previously assigned categories and does not have double directories/files? Any hint is appreciated a lot.

Wolfgang

[Mario]

As always, it would be helpful to have the IMatch log file (log file) and the IMatch diagnosis log file here to see if IMatch reported anything unusual. IMatch 2023 runs a database diagnosis automatically when a database is opened for the first time.
And a screen shot of what you see. It's hard to imagine what you see otherwise.

Upgrading to a new IMatch version upgrades the database. But no new folders are created or existing folders split into duplicates or anything.

A common cause for users getting duplicate folders or even entire drives into their database is moving / renaming the folder out side of IMatch (e.g. in Windows Explorer) and then adding it again to the database. Or, moving images to a new hard disk and adding the files again. In both cases using the Relocate command would have avoided the problem (Off-line Folders and Files)

I understand that you have a two folders A and B in your database, both pointing at the same physical folder on your hard disk, let's call it C. C has 17 sub-folders.
A shows the blue refresh icon, meaning that IMatch has detected that this folder or some of it's sub-folders have changed (different "last modified" time stamp).

A shows 3 of these 17 sub-folders and B shows 16.

I have no idea how such a thing can happen. Except for a mix of wrong relocating and then moving files around in the database. Which you did not do.

The subdirectory in dirB then had every file twice (same filename, same content, one with and one without the categories assigned).

This situation happens if the user "folds" an off-line folder and a real folder into one folder using the wrong relocate.
A folder shows as off-line (yellow off-line icon) because it was moved or renamed outside of IMatch.
Instead of relocating the folder to tell IMatch that has has a new name or location, the user adds the folder to the database again. And then, for whatever reason, relocates the old folder to the new folder.
This is the only known scenario for duplicating folder and file records in an IMatch database.

What to do:

1. Restore the database backup
2. Open it in IMatch 2023 and let the migration run and the diagnosis.
3. Make a screen shot of the Media & Folders View so we can see the duplicate folders
4. ZIP and attach the diagnosis log file to your reply (The Diagnosis Log File)

If you think that B is the "correct" folder and correctly linked to the physical C folder (does it show as on-line (no yellow off-line indicator) then relocate each sub-folder under A (say, A\test to B\test) using the Relocate command available in the context menu of the folder.

This will not move files around on your hard disk or anything. It will just tell IMatch that the folders are now elsewhere.
After relocating every folder from A to B you can use the "Remove Folder from Database" command on folder A.

[Triplefin]

Hi Mario,

thanks for the swift reply and your hints!

I stopped moving files or folders outside IMatch a few years ago after experiencing the issues you described . I also guess I'm reasonably familiar with relocation and moving within IMatch.

So I will follow your advise, redo the migration and report the result here.

Wolfgang

[Triplefin]

Hi Mario,

new database migration is completed. Similar result as the previous migration. See attached files (all zipped into 2023_upgrade.zip):

1 shows the main directories after migration. There are two directories "Reisen23" with identical Name. The bigger on has the "update me" logo. Underneath is one directory each called 2304_Costa_Rica.

2 shows the subdirectories after migration. Under the 2304_Costa_Rica directories you see the subdirectories. It should be 16 in total (I wrote 17 in my previous post which is incorrect, sorry).
One 2304_Costa_Rica with 16 subdirs (2 to be updated) and the other one with two subdirs (again my mistake in counting as I previoulsy said it's 3).

3 just for information the result of the first databse diagnosis

4 the log file from diagnosis

What would you recommend to do?

Wolfgang

[Mario]

The diagnosis log shows:

H:\Reisen_23\ (DA90-55A2 'Volume')
H:\Reisen_23\ (DA90-55A2 'REISEN3')

So the two Reisen_23 folders were added from two differently named volumes: 'Volume' and 'REISEN3'.
This may cause the problem.

1. Make a backup copy of the freshly migrated database.

2. Check the volume name of the H: disk as it is now (Windows Explorer, right click on the drive and select Properties).

3. Then Relocate the "H:\Reisen_23\" folder with the wrong volume name.
Just relocate it from " H:\Reisen_23\ " to " H:\Reisen_23\ " (same name) so IMatch picks up the new volume name.

4. Wait until IMatch is finished, then close and re-open the database.

The 500,000 warnings logged were caused by a new check I've added in the 2023.1.10 to find files with GPS coordinates but not GPS flags. As we have learned, there are many cameras which produce empty GPS records with valid version numbers but no coordinates, causing many files to get the GPS globe but having no coordinates. One must love the Japanese software vendors.

This problem has been resolved in IMatch 2023.1.12 which now correctly detects if a file has a valid GPS record but no coordinates and removes the globe again.

[Triplefin]

Hi Mario,

the issue is solved now 

I removed the the directory with only 2 subdirectories from the database so that only the one large directory tree was remaining.
Next I updated one of the two subdirectories that was marked to be updated. I hoped that it would simply update within the displayed directory structure. However, instead it added again the second directory tree in IMatch, having only one subdirectory (the one I tried to update).
The large directory tree was left unchanged, still with two subdirectories tagged to be updated.

Finding that obviously for unknown reasons the large displayed directory tree is not maintained by IMatch but instead a second one is always created/updated I then tried to relocate all subdirectories into the newly created directory tree. And it worked!

As a result all subdirectories were relocated to the new directory.
The old directory tree was removed from the database.
Now I have only one directory tree displayed in IMatch which corresponds to the file structure on the harddisk and including my categories. Running diagnosis and pack and go both worked fine.

I am not sure if the whole mess was due to anything I did wrong or anything that was not ok in my old IMatch 2021 database. However, it works fine for now.
Thank you for your advise!

Wolfgang

[Triplefin]

Hi Mario,

Thanks for checking the log file. The reason you found makes sense.

I initally forgot to change the label of a new harddisk I added to my PC. So it was the standard Windows "Volume".
IMatch didn't allow me to change the name of the volume (it allows only to change names of directories and files). Hence I changed it to Reisen3 in Windows. And that created the mess I saw when migrating the database.

The IMatch refusal to update the directories that were added to IMatch before the volume name change is only logical.
Your advise is what finally worked in my trial and error actions last night

Again thank you for solving the issue!

Wolfgang