Faulty Timeline View Behavior

[Darius1968]

I'm posting in this department instead of that for bugs in the program because I want to rule out error on my part or a faulty file.  Anyway, I have a said png file, for which all date/time information (created/modified/accessed) are for July 1, 2015.  So, when I go to July 1, 2015 in Timeline View, that file isn't there.  So what is wrong? 

[Mario]

IMatch determines the File 'Date and Time' according to a set of rules, explained in the help in the "How IMatch fills Date and Time". Type date in the help index, then double-click on Timeline in the results to find the info. IMatch uses EXIF, XMP, IPTC, ID3, Office and other date and time information, depending on the file format and which metadata is available.

The selected date and time is displayed in the File Window Info Tip, in the Status Bar for the currently selected file and everywhere the File Date and Time us used (sorting, the time line etc.). If is also the base for the {File.DateTime} variable.

Tip: You can use the "Goto Timeline" (Shift+Ctrl+G) command in the file window to locate the file in the timeline.

[Photon]

The original posting is old, but the issue not.

From time to time I am creating edited photo files with external tools. The original file in Imatch has correct date and time. But from the edited files, Imatch does very often take the date and time of file modification and not the correct EXIF info. With this the timeline in Imatch is wrong, the displayed date/time is wrong and most edited files have a diferent date/time than the original RAW and JPG photo files from camera.

Example of edited file:

• Exif DateTimeOriginal: 2014-04-26 19:13:49
• Exif DateTimeDigitized: 2014-04-26 19:13:49
• Exif DateTime: 2014-04-26 19:13:49
• File date/time: 2015-06-18 / 00:40:45 (this is, when the editing did take place and is replaced with current date/time when you download the attached file)

Please find the attached jpg file for reference.
Remark: I am using Exif-Tool with command line to copy metadata from source files to edited files.
What is please the problem and solution? I am using Imatch v5.5.6

Regards from Martin


[attachment deleted by admin]

[jch2103]

What is please the problem and solution? I am using Imatch v5.5.6

Check to see if a Force Re-scan of the file(s) and then a Categories Refresh helps.

[Photon]

Thank you John for this ultrafast reply, the forced re-scan did it.
With Ctrl-Shift-F5 both options "force update" and "reload metadata" resulted in the correct date/time.

Do I have to make such a scan now for all thousands of files in my database?
Or how can such wrong files be identified?
An why does this error occur? This shouldn't happen, isn't it?
And how could I prevent this in future? Always a forced re-scan?

By-the-way, a metadata write-back Ctrl-Alt-S does also correct the wrong date/time of display and time line. Strange!

Martin

[jch2103]

An why does this error occur? This shouldn't happen, isn't it?

I think (although I'm not sure) it happens when Windows/IMatch doesn't 'notice' that a file has been updated by an 'outside' program and doesn't know a re-scan is required.

I suspect the easiest way to catch all possible out-of-date files would be to force a full update/metadata write-back at a convenient time (e.g., in the evening or other time you don't plan to be working in IMatch) and let her rip. But maybe someone has a better idea.

[Mario]

I don't know what your 'external tools' are doing to the metadata in the file.
But when you modify a file outside of IMatch, the "last modified" date and time of the file changes in the file system (if the other application does not explicitly override this). This means that IMatch rescans the file. Does this happen?

When it rescans the file, the "date and time" as extracted by IMatch is wrong?
IMatch produces this date and time information from the three EXIF tags as explained in the help.
Does the file have perhaps unwritten metadata in the IMatch database while you modify it outside in other applications? In that case, the "protect metadata" option in IMatch may interfere with your external changes, because IMatch tries to protect the metadata in the database from external changes.

Forcing a rescan of the file will disable the protection temporarily.
Writing back the file will also solve this, because IMatch afterwards re-synchs the EXIF, IPTC, GPS and XMP metadata.

When you write metadata in other applications, make sure that

a) No unwritten data exists for these files in IMatch
b) That whatever other application you use properly applies the MWG standards. Just changing the EXIF data without changing the XMP data will cause trouble, and you later will pin this on IMatch, not the other software.
c) Make sure you understand what you are doing and what the other software does to your metadata.

[Photon]

Thank you Mario, your description is good and valuable.
My process is as follows:

• Generate edited, cropped, resized and sharpened JPGs from RAWs and other JPGs with external batch tool Lightroom or ImageMagick.  Imatch is closed.
• Add my metadata with ExifTool. Imatch is closed.
  exiftool.exe artist=... -copyright=... -iptc:by-line=... -iptc:by-linetitle=... -iptc:source=. -iptc:copyrightnotice=... -overwrite_original "filename_fhd.jpg" > NUL
  (No other metadate is modified)
• Start Imatch. Rescan the folders with Imatch.

It might happen, that later I overwrite/modify the files with Lightroom or ImageMagick. No need and intention from my side to modify any metadata outside of Imatch! In this case it can happen, that I forget to rescan or rewrite afterwards in Imatch.  But why the hell is then the file date/time updated wrong in Imatch? Something motivates Imatch to display NOT the unchanged and correct exif and date/time, which is apparently also correct in the Imatch database.

My preferences in Imatch are: "write-back changes to metadata immediately = enabled" and "protect unwritten metadata = yes".
Even when unwritten metadata is existing, the displayed and timeline date/time should not change to something else (= file modification date/time). Don't you agree?

Regards and thanks
Martin

[Mario]

IMatch only falls back to the "last modified" timestamp when it cannot find usable EXIF data in the file.
IMatch determines the file date and time every time the image is rescanned or metadata is changed for the file.

Why this is the case, I don't know. I would need to see an image which produces this effect. And it seems to be only temporary, since when you rescan the file the date and time is corrected?

I'm not aware of a similar case, and I don't know what your other applications do for the file, if and which metadata is migrated, if XMP matches EXIF etc. I suggest you carefully analyze a file when this happens. Check the data in the file with ExifTool Command Processor (especially the EXIF date and time fields) and then use the Browser mode in the metadata panel to see if all the data in the file has been properly imported by ExifTool / IMatch. Again, pay special attention to the EXIF date and time tags.

Maybe this leads to something. I don't think that Imagemagick keeps EXIF and XMP in synch, and LR has also it's own ideas sometimes.
Also keep an eye on the log file, maybe some of the files are still locked when IMatch tries to rescan them and ExifTool is unable to access the files.

[Photon]

My attached JPG file in the posting above seems to have correct meta-data in my humble opinion.

Idea: Might this be an issue just for write-protected files?
Does Imatch take on first rescan the "last modified" timestamp, when it detects read-only files?

When I rewrite files, I see two options:
1. No xmp sidecar file is created (when metadata of database is synced). Ok!
2. A xmp sidecar file is created (when metadata of database is different). Ok!
In both cases the file date/time is correct afterwards. Ok!

When I change a rating, color, flag or something else of such files with wrong date/time,
the date/time is updated correctly and the xmp sidecar is created correctly. Ok!
This is for all files (read-only) and read/write.

There seems to be a certain short moment, when IMatch cannot find usable Exif data in the files.
I will try to identify this moment.

[Mario]

There seems to be a certain short moment, when IMatch cannot find usable Exif data in the files.
I will try to identify this moment.

The read-only status of a file is unimportant.
External XMP files are only created for certain file formats. By default, IMatch embeds XMP in JPEG, TIFF, PNG, PSD and some other formats.
The standard IMatch "File Date & Time" is determined from EXIF data on import.
The only explanation in your environment for this to fail is when the file is still locked when IMatch rescans the folder, and thus cannot load data from the file. If you work with LR or other software, close the other applications before you rescan the folder in IMatch. IMatch does not perform multiple attempts when it encounters locked files. It reports the problem in the log file and then the user must manually rescan the folder later when he has closed the other applications.

[Photon]

..the only explanation in your environment for this to fail is when the file is still locked when IMatch rescans the folder...

Thanks, may be such investigation is the right approach.
The mentioned files are created with Lightroom or ImageMagick, but I never ever perform rescans in Imatch during such external processes.
May be the Imatch application was active in the background and monitoring something? Do I have to close Imatch before any external image creation, update or replacement?
Or is there some Imatch automatism which I have to disable?

Certainly it is my fault, but I think other users will run in the same problems with new files in existing directories.
So it is worth to be analyzed. I just tried it with v5.5.6. May be the following description can help:

• Nearly all of my separately generated files (resized, cropped, sharpened) do not show the exif information (from the image file),
  no metadata "author, date digitized, date created date, date modified and copyright" in the Imatch metadata panel "Default view".
• When I perform a forced rescan, only the displayed date/time in thumbnail view and timeline is updated correctly
  (so no longer the wrong file modified date), but all the metadata panel information is still empty.
• When I edit one empty metadata e.g "description" with e.g. "asdf", all metadata fields are locked for some seconds until IMatch performs a write-back.
  And SUDDENLY after this write-back all missing metadata information is displayed correctly. Whaoo! Why this? Why not on the very first rescan of the directories?
  A write-back alone seems not to help, I have to edit first some metadata in Imatch. Strange!
  

So there are two problems, why is the file date/time not displayed correctly and why does a forced rescan only update the correct file date/time, but not the metadata?

Thanks for any support,
Martin


P.S.: Please please don't blame me for setting the MWG compliance to "No". I am working with read-only media files and with XMP sidecars for all file extensions. There are several good reasons to do so and other tools support the same sidecar approach. I hope this setting is not the reason for trouble. What happens during a test if this setting is changed temporarily from "No" to "Yes".

[Mario]

May be the Imatch application was active in the background and monitoring something?

IMatch monitors all folders indexed in the database for changes. This way it detects new and updated files and automatically rescans them.
This is usually never a problem because IMatch lets some time pass when a file is changed, and starts the rescan only a certain time after the file has changed. This allows other applications to 'settle" and complete their file operations.

This also handles schemes like Adobe LR or PS use, with their three steps when save file operation: Saving the file under a temporary name in the same folder, deleting the original file, renaming the temporary file to the original file name. Windows of course does not know about this and announces "new file in folder", "file deleted in folder", "file renamed in folder". If IMatch would react immediately, it would fail to track changes to the file.