Face detection changes file date (and xmp CreateDate)

[desmooloch]

Hi,

i'm very happy about the new imatch 2020 face detection feature. For now it´s a little bit "hard to handle" (checking faces only in viewer is slow and time intensive), but i read that you are working on something better. Very good

Now to my problem: I´m using Imatch 2020.3.6 (64-bit) and came from imatch 2019 (migrated database, my first version was imatch 3 or something like that )
I have a big database with about 40.000 files and i already had face annotations from picasa. I migrated them to imatch 2020 and run the face detection on all my files (only those without annotations). That was all fine and currently i`m confirming persons. And here comes the problem:
Sometimes i confirm a face the file is updated to current date and time  (xmp CreateDate)!
Thats leads to many problems:
- my timeline (Zeitachse in german) is messed up...
- person layout shows wrong age (new CreateDate...)

i have about 13.000 updated files this week and i have 296 files with wrong DateCreated. They are all from "last night" but months or weeks old.

The new CreateDate seems to be written while "writing metadata to file". But i cannot not 100% reproduce it.
- add a new face to a file
- confirm a person
- choose write back metadata
Then file date in windows explorer is updated to current date and time. But xmp CreateDate is not updated. But it was updated last night for 296 files. I'm not 100 % sure how thias happened but it did

So now 2 questions:
- is this a known bug?
- How can i write right xmp metadata back to file? I need something like: read System Creation Date (it´s still there because windows shows the right "creation date) and write it back to xmp.

I attached a history screenshot for a wrong file.
Hopefully anyone can find the error.

Best regards,
desmo

[Mario]

Changing face annotations changes XMP metadata (region, region tag).
Do you make IMatch immediately write back metadata? If so, the probable cause for this "problem" is that the file has no valid date and time information and that IMatch hence falls back to the "last modified" date and time as reported by Windows. See In Your Language

Solution: Make sure you files have valid date and time information by setting a date and time in the Metadata Panel.

You can check the file with the Metadata Analyst to see if and which time stamps it contains.

[desmooloch]

Changing face annotations changes XMP metadata (region, region tag).
Do you make IMatch immediately write back metadata? If so, the probable cause for this "problem" is that the file has no valid date and time information and that IMatch hence falls back to the "last modified" date and time as reported by Windows. See In Your Language

Solution: Make sure you files have valid date and time information by setting a date and time in the Metadata Panel.

You can check the file with the Metadata Analyst to see if and which time stamps it contains.

Thanks for the ultra fast reply
Immediately write back metadata is off. And i'm sure the file had a valid Date and time information. I used the renamer option for every Import from "unsorted" to move the Files to year, Month and Date folders. This worked a while ago.
I will check If i have an old Backup with the original file.
I remember i had some similiar problems with imatch 2019, but Timeline does Not Matter for me cause of my own "timeline" with directory.
What Else could cause this obscure Error?

[Mario]

So you manually write back metadata?
You can see what IMatch is writing (the tags) in the ExifTool output panel.

I don't see how confirming a face (or anything related to face) would modify MD.XMP::xmp\CreateDate\CreateDate or XMP::photoshop\DateCreated\DateCreated.

If you have steps to reproduce this, let me know.

[desmooloch]

So you manually write back metadata?
You can see what IMatch is writing (the tags) in the ExifTool output panel.

I don't see how confirming a face (or anything related to face) would modify MD.XMP::xmp\CreateDate\CreateDate or XMP::photoshop\DateCreated\DateCreated.

If you have steps to reproduce this, let me know.

Yes manually. I collect about 500 metadata Changes and Last step is write back (takes Long...)
I have a Backup from the original file. Xmp createDate and dateCreated is set to 05.12.2019. after doing face annotation it was Changed. That was the only Change i did.
I will try to reproduce it. I also have my old 2019 database Database and imatch 2019.

[desmooloch]

Is there some kind of "file metadata History" in the imatch Database? Or any other spot i can Look what changed?

[Tveloso]

You should double check whether or not your backup files really do have actual values in XMP Create Date and Date Created, using the ECP.  IMatch may show those two dates populated in the Metadata Panel, when they don't actually exist in the file...as discussed in this topic:

https://www.photools.com/community/index.php?topic=9955.msg70401#msg70401

I too have this issue with a number of file...nearly 2K of my files lack usable dates, and IMatch has had to fall back to the FileSystem Dates.  So these files "pop to the top" of File Windows that are sorted descending by capture time.

Most of these files are Scans, others I recognize as files that I edited years ago (and whatever I used to edit them likely discarded all Metadata), and some are Mobile Phone photos received via Text (where again, the Metadata has been stripped).

In my case, the FileSystem Create Date is also wrong, but if in your case that date is correct for the files, you can use a Metadata Template to populate these two tags:

    XMP::xmp\CreateDate\CreateDate
    XMP::photoshop\DateCreated\DateCreated

...from this variable:

  {File.Created|format:YYYY:MM:DD hh:mm:ss}

After the MD Template is created you can "reset" the dates in all your files back to the FileSystem Create Date (but you should test it first)

[desmooloch]

You should double check whether or not your backup files really do have actual values in XMP Create Date and Date Created, using the ECP.  IMatch may show those two dates populated in the Metadata Panel, when they don't actually exist in the file...as discussed in this topic:

https://www.photools.com/community/index.php?topic=9955.msg70401#msg70401

I too have this issue with a number of file...nearly 2K of my files lack usable dates, and IMatch has had to fall back to the FileSystem Dates.  So these files "pop to the top" of File Windows that are sorted descending by capture time.

Most of these files are Scans, others I recognize as files that I edited years ago (and whatever I used to edit them likely discarded all Metadata), and some are Mobile Phone photos received via Text (where again, the Metadata has been stripped).

In my case, the FileSystem Create Date is also wrong, but if in your case that date is correct for the files, you can use a Metadata Template to populate these two tags:

    XMP::xmp\CreateDate\CreateDate
    XMP::photoshop\DateCreated\DateCreated

...from this variable:

  {File.Created|format:YYYY:MM:DD hh:mm:ss}

After the MD Template is created you can "reset" the dates in all your files back to the FileSystem Create Date (but you should test it first)

My Hero!!!!
You are right, my metadata createDate and dateCreated is empty! Thats why my "creation Date" is updated when i write back metadata. Seems Like i have a few of These files. Dunno why this happened, the Pictures are all from the Same Mobilephone. But now i Know whats the Problem and i can fix it with a Template. Thanks tveloso!
One Thing i did not find: is there a Panel where i can See File.created, File.modified etc...?
Second question: can i Scan my Database for Files with Missing xmp Create Data?

@Mario: showing Not existing metadata content in metadatapanel ist very very misleading! The Panel should only Show existing values or if the "content" is coming from File information there should bei a hint for this

[Mario]

I have told you that above and even linked to the corresponding help section.

One Thing i did not find: is there a Panel where i can See File.created, File.modified etc...?

Sure. Use the Metadata Panel. Use the Browser layout to see all metadata tags imported by IMatch.

Second question: can i Scan my Database for Files with Missing xmp Create Data?

Yes. Use the normal features you use to search for metadata. Like the Metadata Filter in the Filter Panel.

@Mario: showing Not existing metadata content in metadatapanel ist very very misleading! The Panel should only Show existing values or if the "content" is coming from File information there should bei a hint for this

I don't think so. These dates are filled during import as explained in the help topic I linked to in my initial reply to your question. Which exactly pointed out the problem.
They are needed to make the entire date and time machinery working in IMatch, from searching to sorting to the timeline.
Hence these two tags are treated special. Refer to the help topic for details.

[desmooloch]

I have told you that above and even linked to the corresponding help section.

Sorry i did not read it properly... First i thought its only relevant when i use direct write for metadata.

One Thing i did not find: is there a Panel where i can See File.created, File.modified etc...?

Sure. Use the Metadata Panel. Use the Browser layout to see all metadata tags imported by IMatch.

Sorry i can´t find it. Exif tool states "[System]        File Creation Date/Time         : 2019:09:01 17:39:08+02:00", but i cannot find it in meta data panel! Am i blind?!

Second question: can i Scan my Database for Files with Missing xmp Create Data?

Yes. Use the normal features you use to search for metadata. Like the Metadata Filter in the Filter Panel.

Thanks!

@Mario: showing Not existing metadata content in metadatapanel ist very very misleading! The Panel should only Show existing values or if the "content" is coming from File information there should bei a hint for this

I don't think so. These dates are filled during import as explained in the help topic I linked to in my initial reply to your question. Which exactly pointed out the problem.
They are needed to make the entire date and time machinery working in IMatch, from searching to sorting to the timeline.
Hence these two tags are treated special. Refer to the help topic for details.

I can understand that there is a fallback if no create Date is found. But in meta data panel imatch shows "xmp create Date" with the current File timestamp. But there is no "xmp create Date" in file. If i run exif tool it get:

Code Select Expand


[ExifTool]      ExifTool Version Number         : 11.92
[System]        File Name                       : IMG_20190816_203911.jpg
[System]        Directory                       : //DISKSTATION/photo/digi bilder/2019/08_August/20190816
[System]        File Size                       : 157 kB
[System]        File Modification Date/Time     : 2020:04:11 02:53:04+02:00
[System]        File Access Date/Time           : 2020:04:11 05:04:28+02:00
[System]        File Creation Date/Time         : 2019:09:01 17:39:08+02:00
[System]        File Permissions                : rw-rw-rw-
[File]          File Type                       : JPEG
[File]          File Type Extension             : jpg
[File]          MIME Type                       : image/jpeg
[JFIF]          JFIF Version                    : 1.01
[JFIF]          Resolution Unit                 : None
[JFIF]          X Resolution                    : 1
[JFIF]          Y Resolution                    : 1
[File]          Exif Byte Order                 : Little-endian (Intel, II)
[IFD0]          Software                        : Picasa
[IFD0]          Modify Date                     : 2020:04:11 02:53:04
[ExifIFD]       Exif Version                    : 0220
[ExifIFD]       Offset Time                     : +02:00
[ExifIFD]       Exif Image Width                : 522
[ExifIFD]       Exif Image Height               : 704
[XMP-x]         XMP Toolkit                     : Image::ExifTool 11.92
[XMP-iptcExt]   Person In Image                 : Jan, Simone
[XMP-dc]        Subject                         : WER, person, Jan, Simone
[XMP-exif]      Exif Version                    : 0220
[XMP-exif]      Exif Image Width                : 522
[XMP-exif]      Exif Image Height               : 704
[XMP-lr]        Hierarchical Subject            : WER|person|Jan, WER|person|Simone
[XMP-mwg-rs]    Region Applied To Dimensions H  : 704.000000
[XMP-mwg-rs]    Region Applied To Dimensions Unit: pixel
[XMP-mwg-rs]    Region Applied To Dimensions W  : 522.000000
[XMP-mwg-rs]    Region Area H                   : 0.539773, 0.479531
[XMP-mwg-rs]    Region Area Unit                : normalized, normalized
[XMP-mwg-rs]    Region Area W                   : 0.605364, 0.553640
[XMP-mwg-rs]    Region Area X                   : 0.590038, 0.393678
[XMP-mwg-rs]    Region Area Y                   : 0.470170, 0.757813
[XMP-mwg-rs]    Region Name                     : Jan, Simone
[XMP-mwg-rs]    Region Type                     : Face, Face
[XMP-tiff]      Software                        : Picasa
[XMP-xmp]       Creator Tool                    : photools.com IMatch 20.3.0.6 (Windows)
[XMP-xmp]       Metadata Date                   : 2020:04:11 02:53:04+02:00
[XMP-xmp]       Modify Date                     : 2020:04:11 02:53:04+02:00
[XMP-xmpMM]     Document ID                     : xmp.did:ef98aede-90aa-4cb5-8c1b-5e8ec281ec7e
[XMP-xmpMM]     Instance ID                     : xmp.iid:51fc4a53-e1c2-4eee-8221-69a723a1919b
[XMP-xmpMM]     Original Document ID            : xmp.did:ef98aede-90aa-4cb5-8c1b-5e8ec281ec7e
[File]          Current IPTC Digest             : c0f85379ac156a3f67afdf7dd114636c
[IPTC]          Envelope Record Version         : 4
[IPTC]          Coded Character Set             : UTF8
[IPTC]          Application Record Version      : 4
[IPTC]          Keywords                        : WER, person, Jan, Simone
[Photoshop]     IPTC Digest                     : c0f85379ac156a3f67afdf7dd114636c
[File]          Image Width                     : 522
[File]          Image Height                    : 704
[File]          Encoding Process                : Baseline DCT, Huffman coding
[File]          Bits Per Sample                 : 8
[File]          Color Components                : 3
[File]          Y Cb Cr Sub Sampling            : YCbCr4:2:0 (2 2)
[Composite]     Image Size                      : 522x704
[Composite]     Megapixels                      : 0.367
[Composite]     Modify Date                     : 2020:04:11 02:53:04+02:00
This is very misleading. Thats why i thought xmp is filled. It would be better to highlight that this "content" is not from XMP.

[desmooloch]

seems like i cannot edit my post anymore
I'm trying to find files with missing CreateDate und DateCreated via "Filer Panel". I use Data Filter with CreateDate, with content and inverted. This does not find my files with missing xmp data. Imatch filters them cause "they have content". But they have not

[Mario]

Filters work on what's in the database. IMatch has no features to directly search the physical data in your files.
Most likely this could be done with ExifTool and some clever command line.

Why don't you just set a date and time in the Metadata Panel for these files and write back? This is how this is supposed to work.

[desmooloch]

Filters work on what's in the database. IMatch has no features to directly search the physical data in your files.
Most likely this could be done with ExifTool and some clever command line.

Why don't you just set a date and time in the Metadata Panel for these files and write back? This is how this is supposed to work.

Yes if i find them i can correct it. But because imatch showing File Date as xmp Date how should i find them via search?
I understand that imatch needs a Fallback for Date and time. But cause i cant See File Date in imatch and cant find files with "unreal" xmp createDate its almost Impossible to find them.
As stated beforce, i cant see File Date in Metadatapanel.

What about give the user a hint (Symbol, Text, etc...) If xmp createDate is suggested from File Date?
Can i search imatch' internal Database for files without xmp createDate?

[Mario]

As documented in the release notes, I have changed the behavor in the next release for this peculiar case. IMatch now marks the tags as pending.
Hopefully this does not falls on my feet and tons of users complain about many of their files are marked as pending after import - even if they did not change anything...

If these are image files, you should be able to find the files without EXIF date and time information by searching for the native EXIF tags, not the XMP tags which may have been filled by IMatch.
Tags to look for are Exif::Main\36867\DateTimeOriginal and Exif::Main\36868\CreateDate. If these are empty and none of the fallback tags analyzed in addition by IMatch contain values, IMatch uses the last modified date.

[thrinn]

Maybe try to use a Value Filter on Exif::Main\CreateDate?

   
• Select "Files with value"
• Check "Invert the result"
• Make sure that the "Invert results of the filter panel" (symbols at the top of the filter panel) is not activated.

If the files you are looking for have no Exif date tags, this should work - at least in theory...

[desmooloch]

As documented in the release notes, I have changed the behavor in the next release for this peculiar case. IMatch now marks the tags as pending.
Hopefully this does not falls on my feet and tons of users complain about many of their files are marked as pending after import - even if they did not change anything...

Thanks you!! I updated to the Version including ich je Change. I have no Files Marked as pending.
If i understand your Changes right every File without xmp Date should have pending Changes (new xmp Date).

[Mario]

Only for files added with the new version. This enhancement does not go back to files already in the database.
You can do a forced Shift+Ctrl+F5 > Reload Metadata for the files where you see this problem if you like.

[desmooloch]

Only for files added with the new version. This enhancement does not go back to files already in the database.
You can do a forced Shift+Ctrl+F5 > Reload Metadata for the files where you see this problem if you like.

perfect thank you