How to do face detection in digiKam and import them into IMatch?

[ChicagoGhost]

I tried face detection in IMatch with 1000~2000 photos. Many faces are not detected, especially for photos with many people, in which the faces are small. Adding face annotations to them one by one is really a tedious job. And there are 150K photos waiting for me to do this.
Using the YOLO v3 model in digiKam, a higher percentage of the faces can be detected, saving me a lot of work.

So I would like to use the face detection function in digiKam and then import them into IMatch.

First, is this a good idea? I don't know whether digiKam will mess up IMatch's photo directory, or mess up the metadata IMatch has written to the photos.

Second, I can't make it work. In digiKam, I have checked "Face Tags" in settings-metadata-behavior. I have run the command from menu: Album-Write metadata to files. In IMatch, I can't see the face annotations. I used the command-Add or update folders. I even created a new database from those photos.

[mastodon]

Check whether Digikam saves the face tags in the jpg file. If it does not, just in its database, IMatch can't read it.

[Mario]

See Working with XMP Face Regions in the IMatch help.
Works with Adobe face regions, Apple face regions, Google face regions. If DigiKam stores and maintains XMP face regions, it will also work with DigiKam.

I used the command-Add or update folders.

This is not required. When DigiKam updates XMP metadata in your images or sidecar files, IMatch recognizes this automatically and rescans the file.

Tip: Look at an image in the ExifTool Command Processor. Run the "List Metadata" preset and search for region. If DigiKam has written the face regions as XMP regions with tags, this is how you find them.

[axel.hennig]

Just also wanted to quickly try digiKam and their face recognition tool, but after importing 16 (!) pictures (jpg) and running the face detection / face recognition, digiKam crashed. Not sure if I want to rely on such a software. Maybe it is more stable on Linux or maybe it is just my Computer, but my start was not that promising.

[mastodon]

Digikam is really promising at first, but overhalming interface and crashing.

[ChicagoGhost]

Not sure if I want to rely on such a software.

After a few days of trying, I think it's not a good idea to rely on it. There are many bugs. And, from the function point of view, it is not comparable to IMatch. IMatch is much more powerful. The main reason that some people like Digikam is that it is free.
However, since face recognition has an option to use the YOLO v3 model, the result is pretty good.

[Mario]

But it does not export the face regions to XMP to make them available for other software?

I have never used DigiKam since I have a proper DAM already   so I cannot really provide any insights.

On their help page they write:

Face Tags (including face areas): will store the face tag paths and the rectangles corresponding to the zones around faces. Usually this information is stored in XMP.

which sounds promising!? Maybe just a switch to enable?

an option to use the YOLO v3 model,

I have worked with the Yolo 3 (which is around for a while) and it's predecessors in OpenCV etc.
Results were mixed. Sometimes IMatch was better, sometimes the Yolo model. Maybe your use case is special. Maybe you have not trained IMatch correctly yet?

[ChicagoGhost]

which sounds promising!? Maybe just a switch to enable?

I got it working. Before confirming the faces, Digikam doesn't write the face recognition results to the file. After I confirmed, they were written and I could see them in IMatch.

But, a big problem arises:
It looks like Digikam somehow corrupted most of the files.
1. In IMatch, when using ExifTool Command Processor, it shows an ExifTool error.
2. IMatch says: there was an error while writing back metadata to this file.
3. Those files don't show face annotations in IMatch.

So, I have come to the conclusion that Digikam is not reliable, even just using it as a face recognition tool.
I am going to stay away from it.

[Mario]

It looks like Digikam somehow corrupted most of the files.

Uh-oh, that's bad. I thought DigiKam was using ExifTool to read and write metadata?
And ExifTool is solid, IMatch uses it since IMatch 5 back in 2016.

[axel.hennig]

digiKam uses Exiftool.

[Mario]

Thought so. More info needed on the damage effects and the actual ExifTool error message.
Error messages may be shown for many reasons.
Maybe run the Metadata Analyst in IMatch on one of the problem files.

[ChicagoGhost]

Thought so. More info needed on the damage effects and the actual ExifTool error message.
Error messages may be shown for many reasons.
Maybe run the Metadata Analyst in IMatch on one of the problem files.

I did the following:
1.   Create a new database in IMatch and add the photos folder.
2.   In Digikam, set the above photos folder as a collection folder.
3.   In Digikam, scan the collection for faces.
4.   Open IMatch again. After all the photos have been updated, I can see:
(1)   Some photos don't have face annotations. They do have them in Digikam. Some other photos are OK.
(2)   Some photos (with or without faces) have the wrong time, the time has been changed to today.   Some other photos are OK.
(3)   Using Exiftool Command Processor, some photos (has or lose face annotations, with or without faces) show: [ExifTool] Warning: JPEG format error. Some other photos are OK.
(4)   When writing metadata to file, some photos (have or lose face annotations, with or without faces) give errors.  Other photos are OK.

Code Select Expand

There was an error while writing back metadata to this file. The file may be write-protected, the format unsupported or the file itself may be corrupted. When you re-attempt to update the metadata, check the ExifTool output panel for detailed error information. This icon vanishes when you rescan the file or metadata is updated successfully
The ExifTool output panel shows:

Code Select Expand

-overwrite_original_in_place
-charset
FILENAME=UTF8
-v2
-m
-use
MWG
-charset
ExifTool={PTETCHARSET}
-ex
-tagsfromfile
C:\phototemp-digiKam-new\20161219_140244.jpg
-@
C:\Program Files\photools.com\imatch6\arg_files\exif2xmp.args
--Exif:rating
-@
C:\Program Files\photools.com\imatch6\arg_files\iptc2xmp.args
-@
C:\Program Files\photools.com\imatch6\arg_files\gps2xmp.args
-sep
 
-XMP-xmpMM:InstanceID=xmp.iid:9e64f456-b3ef-4731-a59a-c902bc26f21c
-xmp:InstanceID=xmp.iid:cd67c8ea-25a6-4385-b6ec-73bca7995e78

-XMP:MetadataDate=now
-XMP:ModifyDate=now
C:\phototemp-digiKam-new\20161219_140244.jpg
-execute
-execute9999

Setting new values from C:\phototemp-digiKam-new\20161219_140244.jpg
Writing Photoshop:IPTCDigest
Writing XMP-xmpMM:InstanceID
Writing XMP-xmpMM:InstanceID
Writing XMP-xmp:MetadataDate
Writing XMP-xmp:ModifyDate
======== C:/phototemp-digiKam-new/20161219_140244.jpg
Rewriting C:/phototemp-digiKam-new/20161219_140244.jpg...
  Editing tags in: APP1 APP13 Photoshop XMP
  Creating tags in: APP1 APP13 Photoshop XMP
  Error = Corrupted JPEG image
    0 image files updated
    1 files weren't updated due to errors


----- Runtime: 1 s.
Warning: JPEG format error - C:\phototemp-digiKam-new\20161219_140244.jpg
Warning: No writable tags set from C:\phototemp-digiKam-new\20161219_140244.jpg
Error: Corrupted JPEG image - C:/phototemp-digiKam-new/20161219_140244.jpg

The Metadata Analyst shows three warnings:

Code Select Expand

Warning: [System] File has unwritten metadata (pending write-back).<br/>The metadata loaded from the image and the data in the database may not match.
Warning: [Metadata] Warnings: 'JPEG format error'
Warning: [Detailed Validation] JPEG format error


[Mario]

Hm, this looks indeed like a corrupted JPG image.
I have not seen this before for JPEG images, but ExifTool is very precise and very picky.
It does not modify images where it finds some issue that may cause the metadata write to cause (further) damage.

All these issues don't sound encouraging. Maybe the current version of DigiKam has a bug, I don't know.
They have a GitHub and many contributors helping with the development of the software. I'm just me.

[ChicagoGhost]

I am trying another way:
In Digikam, save all changes to sidecar files only. Then use Exiftool to copy the -RegionInfo part of the xmp files to jpg files. This way, Digikam is not modifying jpg files directly. I am using the -P option to preserve the file modification date/time, since for some photos, there is no DateTimeOriginal. In that case, the file modification date/time is used to sort the files.
Without the -P option, when I open IMatch, photos are updated automatically.
But with the -P option, nothing is happening. I think it's because IMatch uses the file modification date/time to determine whether the file has been changed. First, I used the "Rescan now", but still nothing is happening. Then, I used the "Add or update folders". Then the face annotations are correctly updated. So, is it right that I should use the "Add or update folders"? Or, is there any setting to let IMatch update the files automatically even if the file modification date/time is not changed? Maybe there is some other method to determine that a file's metadata has been changed?

[Mario]

But with the -P option, nothing is happening. I think it's because IMatch uses the file modification date/time to determine whether the file has been changed.

Yes. If the "last modified" file system timestamp of a file has not changed, IMatch does not rescan it.
Use a forced rescan instead (select the file and press Shift+Ctrl+F5 > Force Update). The same command is also available for folders.

Is the IMatch face recognition really so bad for your files that you have to run through all these hoops? This all sounds error-prone to me.

If your files have no metadata time stamps, why don't you add them?
IMatch creates the important "date subject created" and "create date" from the "last modified" as a last resort.
See How IMatch uses Date and Time Information

When you write back once, all your files will have valid time stamps and no clumsy old-fashioned work-around like keeping the "last modified" file system timestamp unchanged even when you have actually modified the file is needed.

This prevents IMatch from recognizing modified files, it will upset your backup software if you use file-base backups etc.
We did things like this a decade ago when Windows Explorer could not read metadata timestamps and "last modified" was used as a fallback.

[ChicagoGhost]

Is the IMatch face recognition really so bad for your files that you have to run through all these hoops? This all sounds error-prone to me.

IMatch face recognition is not bad at all. It is pretty good. But for my files, I think YOLO v3 model does a good job. For example, I have a folder of 58 files. Most of them are family photos of two or more people, full-body shots. So the faces are very small in the photos. There are 129 human-recognizable faces. YOLO v3 model recognized 125 faces. IMatch recognized 74. Instead of marking all the missing faces manually, maybe it's better to go through the extra steps using Digikam and Exiftool to do it automatically.

If your files have no metadata time stamps, why don't you add them?
IMatch creates the important "date subject created" and "create date" from the "last modified" as a last resort.

That's what I was thinking about, but I don't know how to do it automatically (in batch). I mean among the 150K files I have, how can I find and add date/time only to those files without date/time? Actually, for some of the files, the file names tell the time of the shooting, e.g. 20180501-100303.jpg. I see that Exiftool has the function to write tag date/time retrieved from the filename, but I don't know whether it can find those files without date/time automatically. Maybe I should make a data-driven category with the DateTimeOriginal tag and apply changes only to those files in "others".

See How IMatch uses Date and Time Information

I will read this carefully.

[Mario]

If the faces are very small, try using this option: No Faces Found?

150K files I have, how can I find and add date/time only to those files without date/time?

You don't have to.
IMatch automatically creates the timestamps, as explained in the help topic: Pending Write-Back either from existing EXIF, legacy IPTC, XMP, QuickTime and whatnot. Or, as the last resort, from the "last modified" file system timestamp.

Look at the Metadata Panel in the Default layout. There you see the time stamps IMatch has produced for your files.

If IMatch had to use the last modified file system timestamp since nothing else was available, it marks the time stamps as pending write-back (look at the pen in the Metadata Panel) and when you write back, it creates proper XMP time stamps in the image.