Duplicates in database

[cytochrome]

I have a problem similar to that of Leifel https://www.photools.com/community/index.php?topic=8035.msg56404#msg56404 ... but worse.
My main photo disk crashed, I had several backup disks that I collected and restored all images on a new HD. In Imatch the files were all marked as off-line. I don't know what I missed but after hitting relocate which took several hours, everything is fine but I have now two thumbs for each image. I setup a data-driven category based on {File.Added|format:YYYY-MM-DD} set on yesterday it collects all files including the doubles: they appear to have been added all at the same date!!

The only visible difference is the numbering shown on the thumb they increase with each photo and the two sibblings are separated by a fixed offset 86429 here.
In the history panel the first file of each couple has a +/- long history, the second file has nothing or updated.
Is there a way to setup a filter on this?
Some days ago I bought IM 2018, but have not installed it yet. Will it make any difference if I install it now? or make the mess worse?

[cytochrome]

As a follow up: i launched a database diagnostic which hangs on Folders since half an hour. I will abort and try "Repair the database". Is this a good idea in my case?

[cytochrome]

More news on this: Since the database diagnostic did hang I aborted and closed Imatch.
I installed IM 2018 but did not open it. I opened again IM2017 et voila: Everything was back: all folders marekd as of-line. The tree on my F: disk is F:\Images\folders.
I relocated  a first folder say "Andalousie": Imatch created a folder as F:\F(7979-A1D0)\Andalousie with all the files and metadata.

I am now relocating folder by folder, they are big chunks like \D80, \D7K, \GX7 ect with many subfolders. Every thing is update very fast: some seconds for 4-5 thousand files.

I suppose that at the end  F:\F(7979-A1D0) will resolve to F:\Images ??
Francis

[Mario]

I don't know what I missed but after hitting relocate which took several hours, everything is fine but I have now two thumbs for each image.

The only know way to create this is:

1. Instead of relocating the folders (drives), they are added to the database again.
2. Now the user uses relocate and basically 'folds' the old (off-line) and new (pointing at the correct folder) together.

The second copy (the file link that was added to the database instead of using relocate) has no category assignments, Attributes etc.

To fix this, you need to find and select the files which were added to your database accidentally. Then use the "remove from database" command available in the File Window context menu to remove these files from the database (not from the disk). This will take a long time for 90,000 images so be prepared.

A

If you have a backup of the database (you should do DAILY backups), just restore it. Then do the Relocate again, do not use "Rescan".

B If you have no backup (shame!) you need to find the duplicate images.
Depending on how your db is setup, search for images without manually assigned categories. Or for images without attributes.

Another way would be to determine the OID (internal number) of the first newly added file. You can create a sort profile for OID and then sort the File Window by that (even for the entire database). Display the OID in a custom layout (there is a File Window layout attribute for that) . This way you can sort the File Window by OID, scroll down until you reach the first accidentally added file and then select from there to the end of the File Window. Then run "Remove from Database".

[cytochrome]

Mario, thanks for the answer, but I stay with my report:
I reconstructed on a NEW disk my photo files tree from different backups, then opened IM 2017: All folders and files were marked with the yellow triangle: file not on-line: do relocate.
I relocated not file by file but by big ensembles all D80 files, all D7000 files etc. And got two exactly similar thumbs for each and every file.
Anyway I had a 2 month old Packngo, used it and all is OK. 

Now I have two questions:
- IM(2018) uses G:\Temp_PacknGo_Out\IMatch_20180619\25-10-14_IMatch Database.imd5. I would prefer to have the database in its normal place. Can I simply move it to the old place?
- in Media & Folders the tree looks lile Database\F:\F(7979-A1D0)\all my photo folders.. while it should be Database\F:\Images\all my photo folders as it was before my disk crash.

Is there something to do?
Francis

[Mario]

1. You can move the database file to any folder you like, when IMatch is not running.

2.

looks lile Database\F:\F(7979-A1D0)\all my photo folders..

Usually you will see something like

F: [drive node]
  |-- top-level folder 1
  |-- top-level folder 2
  ...

Do you see a media node with the serial number as well? A gray box with the media label and serial number next to it?
This only shows when IMatch has more than one media from the same drive in the database.

First try to press <F5> to refresh the Media & Folders tree to refresh it. Or close and re-open the database. This may be a left over of the massive relocate.