MWG::Regions\RegionInfo being set as pending writeback after a new or re-scan

[mlavicka]

     I'm not sure if this is a bug or feature request.  But I can't figure out how to scan (or re-scan) in photos without getting a pending writeback for MWG::Regions\RegionInfo.  I created all face regions with iMatch and write back all metadata to the file.  Then if I scan it in again, I get the pen icon for this RegionInfo.  This happens if I just rescan in the metadata, but also if scan for first time following a new database creation. I use Metadata Analysts to compare the metadata before and after the rescan, and it shows no difference in the "XMP-mwg-rs" metadata.
     This may or may not be related to a comment Mario made in a different post:
This behavior is by design. When it imports face regions, IMatch no longer "knows" that the file on disk may have XMP face regions. When XMP face regions are found, the existing PersonInImage data is dropped and, at some later time, recreated. Changing this behavior would be quite complex and "expensive".

My settings are:
  Metadata Working Group Compliance = Yes
  Automatic Face Recognition        = Yes        Automatically assign persons to faces = Yes
  Automatically confirm faces       = Never      Keep existing XMP                     = Yes
  Protect unwritten metadata        = No         Protect existing XMP                  = No
  Import XMP Face Regions           = Yes        Import XMP Face Regions in reverse order = No
  Export Face Annotations to XMP    = Yes        Export only confirmed person tags     = Yes

 

[Mario]

Why do you have a need to rescan files manually? Or need to create new databases?
IMatch databases usually last decades (my personal one was created in 2004)?

You have also modified several of the default metadata settings. Why?

[mlavicka]

Four main reasons for rescanning:

1) iMatch detects a folder change - I tend to move groups of files and folders around a lot, from within iMatch.  Occasionally, iMatch thinks the folder has a change by showing the "Folder needs refresh" icon.  I assume that this is just a timing mismatch of how long it takes to do the operation vs iMatch background checking of a folder change outside of iMatch.  So I just rescan the folder to fix it.

2) Undoing Metadata Changes - Sometimes I mess up the changes I wanted to make and just need a reset by rescanning in the previous state of the file from last write.

3) Not being dependent on centralized iMatch database - I store all of my photos in files that are automatically synced to the cloud (OneDrive).  I want to make sure that all data I care about is stored in the photo files, and that I am not dependent on an iMatch database to view it.  So I only use metadata that can be written back to the file and can be read by most software.  This is why I have changed the default settings to make sure the Metadata in the file is the "golden copy".

4) Changes made outside of iMatch - I try to minimize this, but if I make a change or just need to ingest new files with RegionInfo, then I will need to scan them in.

Regarding creating a new iMatch database.  This comes into play when I change platforms.  While my photos are stored locally and synced to the cloud, my iMatch database is exclusively on my local device.  So when I am home, this is my Desktop.  But when I travel, this is my Laptop.  So I don't deal with all the files involved with the "Pack & Go" option.  I just recreate the database on my laptop, and it reads in all the data fresh from the files.  When I return home after a long trip, I just recreate the database on my Desktop.  I never use my Laptop and Desktop at the same time.

The only other data that I need to properly recreate the database is the People data.  For that I export the JSON file and import it before scanning all the files.  As long as the Person Tag matches up, then everything works beautifully.

As a side note, I do a lot of work in photo genealogy where thousands of my photos are early 1900s that have been scanned from old photos or negatives.  I use the face recognition to help identify people in the photos.  I have a complex family tree that I try to keep in sync with iMatch People data.  It would be great if there was a way to maintain the people data in a genealogy program and then read it into iMatch, instead of having to manually maintain it in two places.

[axel.hennig]

It would be great if there was a way to maintain the people data in a genealogy program and then read it into iMatch, instead of having to manually maintain it in two places.

I haven't tested, but I quickly found several tools which are able to convert a *.gedcom-file to a *.json-file. This, together with maybe https://jqlang.org/ might do what you want.

[Mario]

Occasionally, iMatch thinks the folder has a change by showing the "Folder needs refresh" icon.

There are multiple reasons for this:

+ Windows sends IMatch a "folder modified" or "file modified" message for the folder or a file within.
+ IMatch detects that the folder's "last modified" timestamp is newer than record on disk
+ During startup, IMatch scans the folder and finds a file that has a newer "last modified" timestamp than the folder.

If a file is modified outside of IMatch, you undo whatever metadata change you did, I see no reason for IMatch re-importing the XMP face regions and marking one or more tags as modified. There are 15,000 metadata tags ExifTool and IMatch supports, and many with dependencies and relations with/to other tags, mapping between numerical codes and user-facing values and all of that. The metadata engine in IMatch is complex enough.
Sometimes it's just easier and more safe to mark tags IMatch updates as modified, even when the have the "same" user-facing content. The internal representation may have changed, or the write-back is required so ExifTool can synchronize the data in other linked tags.

So I don't deal with all the files involved with the "Pack & Go" option.  I just recreate the database on my laptop, and it reads in all the data fresh from the files.  When I return home after a long trip, I just recreate the database on my Desktop.  I never use my Laptop and Desktop at the same time.

Isn't this a lot more work than just using Pack & Go? Or just copy the database/settings database if the installations are on the same version number?
Your temporary database will miss many of the templates, AutoFill, categories etc. you real database has, and you also have to merge all the stuff back. I'm sure simply copying the database and images from the laptop over is faster. I work that way for many years.

It would be great if there was a way to maintain the people data in a genealogy program and then read it into iMatch, instead of having to manually maintain it in two places.

Yeah. This comes up maybe once a year. Once as a feature request even, when I recall correctly.

GEDCOM is an, eh, "complex" data exchange format. The specification document has over 150 pages!
Which is OK when you are a company specializing in genealogy data or systems, or maybe an AI company hovering up all the data to feed an AI. Or an enthusiast with a lot of time, trying to solve a specific problem.
But for me, adding support for this to IMatch would be a massive amount of work, for maybe a handful of users who will ever need this.

IMatch can import and export people data in JSON format.
Which makes it possible to read, manipulate and re-import people data by hand or using custom scripts or apps.
I think this is good enough for the very small number of users who need "more" from this.

[mlavicka]

Thanks for the info.  I will revisit just copying the database over.  When this issue does occur for me, I will just run:  "Database Tools ==> Clear Processing Queues ==> Clear metadata write-back queue". So I don't have to rewrite everything to disk again.  I hope that won't cause any other problems.

Also, I wouldn't underestimate the amount of people who interested in old photos and need a great DAM.  There are millions of FamilySearch, Ancestry and MyHeritage users that are utilizing the photo capabilities of those web sites all the time.  I will be at a genealogy conference next month where I will be telling others that iMatch is the way to go for their needs.  The Renamer, TimeWiz, and MetaData Editing are must have functionality that iMatch provides in a superior way.  And now with face recognition and AI capabilities you have added, iMatch is perfectly positioned for this area.

Here is a nice graphic for you

[Mario]

Nice chart. Have a good time at your conference. Tell many people how cool IMatch is

As I said, it's all about user numbers. A feature that takes weeks or months to develop, document and test, requires a sufficient number of paying users to support it. If GEDKOM interchange or something along that line would bring in a couple of hundred new users, I will seriously consider it

That being said, IMatch has a very powerful programming interface named IMWS. Via IMWS, virtually all programming languages can access IMatch database contents, read and write. Features like the People View, the Event View and many other features in IMatch are implemented as "apps" in JavaScript and HTML, and use IMWS to communicate with IMatch.

IMWS offers features to add, modify and delete persons, teams and families. These could be used to write a script/app to export/import person data. When there is a real need - or somebody who is not Mario does it.

Adding a feature or app always creates maintenance debt for me. When I add something, I must be willing and able to support it for a long time, update it, adapt it, document it, test it etc. - until it is no longer useful.

[Jingo]

As a book designer who works closely with many Genealogists and Family History clients, I too can attest to the huge impact photo asset management makes to a project. Unfortunately, most of my clients do not understand how to fully utilize a computer let along a complex digital asset management package so there is a big challenge in that and they are just as happy to dump everything into Ancestry rather than trying to tag, identify and keyword via an outside asset like IMatch.

[Mario]

I'm always weary of things that are free, especially when these things collect personal information. I'm European/German and we are a bit more careful with person data and who we share it with.

Google and Facebook don't make their massive amounts of money from giving away free social networks, photo sharing sites, cloud storage, free email or free web browsers.

Looking at the privacy information for Ancestry, for example, https://www.ancestry.com/c/legal/privacystatement#shared-info is basically top-level legalese to cover any use for their customers data they see fit. Just one example, I don't have anything against Ancestry.