Can you pause Rescanning?

[Damit]

I want to harness face regions and annotations that exists in a backup of files. I want to replace the current files with those in the backup, and then load the Face Annotation and Regions using the "Recreate face annotations from XMP data" command. Then I want to save all metadata that is stored in the iMatch database, as they have been updated since the backup, along with the updated face information for those files.  The only thing I want from the backup is the face regions and annotations, everything else is as it should be in iMatch.

My concern is that if iMatch rescans the files it will overwrite the existing metadata in iMatch (some of which has been repaired) with the outdated metadata in the backup files, so that when I write metadata, the new information would be replaced by the old in the backup. By pausing the rescanning, I believe I can avoid this from happening because only the face regions and annotations would be read into iMatch when I manually ask it to recreate the data and the other metadata would not be loaded into iMatch, because the folder was never rescanned. Can I pause rescanning or is there another way to accomplish my goal?

[Mario]

Mhm, that's quite a task. Did you lose the original files or something?

I have never tested such a scenario (replace file indexed in database with older file but keep metadata read from now replaced file but import face regions from the freshly replaced file). Very unusual.

I would restore the files in a way that the timestamps (last modified) don't change and match the timestamps of the files I replace. I think Windows Robocopy has an option for this. I don't know how you plan to restore the files. If you restore via a backup software and it does not offer this feature, restore into a temp folder and then use RoboCopy to replace the actual files.

In IMatch, I would turn off indexing under Edit > Preferences > Background Processing. This should prevent IMatch from automatically re-ingesting files when Windows reports changes in the folders or changes done to files. Never tested this particular scenario.

[Damit]

Something disastrous happened and I do not know how it did, but all the face annotation were wiped out and replaced. Thousands of hours of work gone! I think Lightroom somehow did this, but I am not sure. It should have not done any face detection on folders already processed by it.

I am now engaging in a strict backup protocol as I thought the pack and go was a full backup of the database, but I have learned, the hard way, that it is not. I have read your primer on backup and am making a copy of all the folders and registry, as instructed, so that I can recover if anything happens again. 

The backups have the face annotations, but the current files have some work I did on the files and database before I realized this happened.

Thank you for the advice.  I will follow your instructions after all the backups are completed and hopefully I can execute this recovery.

[Damit]

In IMatch, I would turn off indexing under Edit > Preferences > Background Processing. This should prevent IMatch from automatically re-ingesting files when Windows reports changes in the folders or changes done to files. Never tested this particular scenario.

Unfortunately, it does not.  I set the options as instructed but I match immediately detected the changed files. I tired just doing a regular copy first, while creating a secondary copy of the original files.  I will now try to decipher the robocopy commands to see if I can get that to work.

[Mario]

I thought the pack and go was a full backup of the database

It is.

Pack & Go is not a backup replacement, but it can backup any number of databases, IMatch settings, presets etc.
I changed IMatch to prompt the user after learning that many users don't backup their data at all!

Pack & Go does not backup your files or any non-IMatch data, though.
It's main purpose is to transfer databases and settings between computers.

I think Lightroom somehow did this,

Lr does not automatically import metadata changed by other applications.
You have to explicitly ask it.
If you did face recognition in IMatch but did not re-import the data in Lr afterwards and then modified metadata in Lr and asked it to write back, it will wipe all changes done in IMatch or other software.
This is an Adobe product, after all. They don't design for cooperation with 3rd party products.

[Damit]

It is.

If it is, then the face recognition data should be there.  I made multiple copies of it. I guess I mis-read your manual. So I should be able to recover the information.  I am going to have to attempt this.

The only problem is that I made a bunch of changes to the Thesaurus since I assumed I lost everything and I will lose those if I go back, but I guess losing 20-30 hours is not as bad as losing thousands.

The changes were made in lightroom and then updated in iMatch, not the other way around.  I still don't understand how it happened because I was doing face recognition in LR and updating the files in iMatch for months and everything worked fine. This happened when I returned to the project after a bit of a lay-off and noticed that all the face regions and annotations were missing or improperly associated. It is as if I asked imatch or LR to re-run face matching again, but I did not.

[Mario]

If it is, then the face recognition data should be there.

IMatch stores face regions together with other metadata when you write-back.
If you open an image with the ExifTool Command Processor and run the "List Metadata" preset, you can see the face data when you search for region.

The only problem is that I made a bunch of changes to the Thesaurus since I assumed I lost everything and I will lose those if I go back, but I guess losing 20-30 hours is not as bad as losing thousands.

The thesaurus is stored in the database, not in images. I thought you wanted to restore backups of images, not go back to an older database. Anyway, you can just export the thesaurus and import it into the other database: Exporting Thesaurus Data

IMatch sees face regions written by Lr as soon as you tell Lr to write the XMP data.
And vice-versa. When Lr tells you that the metadata in a file was modified (by IMatch, in this case) use the option to reload the metadata.

[Damit]

I was trying to restore files, I was just under the misconception that the face region data was stored in the database as well as the files themselves. Well, I went back to several versions of the back ups and all the face regions are gone.  As I believe I understood correctly from your last response, they seem to have been stored in the files and not in the database, so if they have been wiped off the files, no backup would restore them, only perhaps an earlier version of this file, which I do not have.  That is why I will now start an incremental Macrium reflect backup so that I can hopefully go back and resolve these issues if they crop up again. Hopefully I can just find the version that has the information lost and restore that. I was considering using a syncing backup, like freefilesync with versioning, but then you would have to restore each file one by one. 

This was a very painful experience but hopefully one that will provide lessons to prevent a complete loss next time around. Thank you so much for your assistance in trying to help me resolve things.

One thing I would clear up: why did iMatch continue indexing even though instructed not to?  It would be great if we had the ability to command the program to not scan or index the files and for it to reliably do so.  I assume that is the intended use of that command.  Why is it not working?

[rolandgifford]

Can you use the Text Export and Import CSV Files?

Restore a database containing the face recognition data
Export the face recognition tags data
Restore the current database
Import face recognition data

I've never tried it but it may do what you want

[Damit]

It could work, but I am gong back further and the face regions are not there.  I thought the regions and all metadata was saved with the database and, if you wanted, also written in the files themselves but what I inferred from Mario's response is that the regions were stored only in the files. This would make it impossible to restore this info. That is what I understood, but I hope I misunderstood.

[rolandgifford]

I understand that info is always stored in the database, optionally in the files as well

[thrinn]

Well, I went back to several versions of the back ups and all the face regions are gone. 

May I ask how you checked / confirmed that all regions were gone? Did you check the result of {File.Faces.OID} for some photos with people? Or any of the other {File.Faces} variables?
Or did you check e.g. for {File.Persons.Label}? I know that Person assignment is not the same as Face Regions, but they are still connected.
Or did you check in the People view for the number of pictures containing each person?

[thrinn]

I understand that info is always stored in the database, optionally in the files as well

IMatch will store the face regions in the database while the face detection process runs (that is my understand). If you don't write back the info to the files, they will not have the face region info.
But there is also a different process step, namely ingesting metadata from new or changed files. Especially if you manipulate face regions available in the files by another application. It depends on you settings, but I can imagine that it is possible to overwrite the face regions available in the database with information from updated files.

[Mario]

IMatch stores face annotations, and all other annotations, in the database.
When you write back a file, IMatch converts the face annotations into XMP regions and stores them in the image. This makes IMatch face annotations available for other applications. Lr does the same.

When IMatch imports metadata from a file, it looks for XMP face regions and imports them, creating face annotations. If you delete XMP face regions in another application, IMatch will delete existing face annotations in the database when it imports the metadata of the file. Like it wipes and re-imports all other metadata.

A workflow that may cause this problem is not writing back files before processing them in Lr. Or not re-importing the XMP data IMatch has written into the Lr catalog. That's an extra step in Lr.
Otherwise Lr will write XMP data without any face regions, wiping them also from the IMatch database during import.

But backup copies of your database will have the face annotations of course.
As long as the images without face annotations are not re-imported. You can see them in the Viewer or check them via the variables mentioned above.
Triggering a write back, e.g. my marking an XMP tag as modified in the Metadata Panel will recreate the face regions in the file from the face annotations in the database.

[janb83]

I have never tested such a scenario (replace file indexed in database with older file but keep metadata read from now replaced file but import face regions from the freshly replaced file). Very unusual.

This request seems quite similar to these:
https://www.photools.com/community/index.php?topic=6918.0
https://www.photools.com/community/index.php/topic,13637

Maybe the approaches discussed there can help you Damit

[Damit]

This request seems quite similar to these:
https://www.photools.com/community/index.php?topic=6918.0
https://www.photools.com/community/index.php/topic,13637

Maybe the approaches discussed there can help you Damit

Thank you so much janb83!  I will look into this, but it seems my face annotations were lost somewhere earlier along the way OR maybe I was not as fastidious as I remembered in my face tagging.  It seems about 90% of the face tagging is still present but some are definitely missing.  Right now there are a lot of screwed up things going on in my database and these problems may emanate from that, who knows? Hopefully I can get it sorted out with Mario's help so I can finally start the real work of tagging and organizing my photo files once and for all!

[sybersitizen]

I understand that info is always stored in the database, optionally in the files as well

IMatch will store the face regions in the database while the face detection process runs (that is my understand). If you don't write back the info to the files, they will not have the face region info.

Just jumping in late to say that's incorrect. I have face annotations and keywords recorded for a large number of files, and have never written back any data to them. All of it is in the IMatch database. Since I don't use other tools for reading or writing those things, there's no compelling reason (yet) for me to write to the files.

[Mario]

When IMatch does face recognition, it creates IMatch face annotations from faces and stores them in the database. That's all IMatch needs for all face- and people related features.

When you write back, IMatch creates standard XMP face regions from the face annotations and stores them in the image. This allows other applications to read the face regions when they support this. And it makes the user independent from IMatch. The work IMatch has performed is persisted in the image itself, in a standardized way. That's how I like it. I don't use customer retention / lock-in schemes via proprietary metadata or otherwise.

I recommend to perform write-backs frequently. Just so you know that the metadata you have added/modified in IMatch is stored safely and independently from IMatch.

[sybersitizen]

Sure, I'm aware. If/when I move away from IMatch as my only DAM I will of course do the writebacks.