metadata update never stops.

[jcldl]

I installed imatch 15 days ago, coming from "photo supreme". But having done some changes in my keywords, metadata updated. But now "metadata update "does not stop, I have always "importing metadata either from 0 to 6040 or 6180" or something else and it works without stop. If i leave imatch working after many hours the system hangs. I have a huge dump file of 1.3 Go. What can I do?

[Mario]

I have always "importing metadata either from 0 to 6040 or 6180" or something else and it works without stop.

I understand from your post that IMatch is importing metadata from files, but is either very slow, or fails to import the data and tries it again?
The first thing we need to look into this would be the log file. Please lookup logfile in the IMatch help for details.

The most likely cause for such an effect is that ExifTool is stuck and fails to import data from some of your files. Or anything like that. Since I don't even know which file formats you are processing, I cannot say more before you attach (!) a zipped log file. The DUMP file will be of no help with this.

[jcldl]

I agree it's a problem with exift. I process jpg, arw, mrw and some tif files. I have a log file produced with dump file and one called "logging". I join these two files. Is it what you want? Thanks

[attachment deleted by admin]

[jcldl]

I copy the changes made by imatch from one xmp to another one: only 3 lines are changed, i do not know the meaning of this. Thanks for your help.

[attachment deleted by admin]

[jch2103]

I'm not an expert on analyzing IMatch log files, but:

- IMatch starts with a warning that 95% of memory is in use (later 85%). That's a lot. How much memory do you have? What other applications were you running at the same time? (Some like Adobe can be big memory hogs.)

- There are a lot of ExifTool warnings; ExifTool is having problems reading the metadata for some of your files. These include warnings for 'Error finding parent structure for LocationShownCity' (XMP-iptcExt:LocationShownCity) and 'Excessive number of items for dc:subject. Extracted only the first 1000'.

I think Mario will need more information. If you can run the ExifTool Command Processor and do 'List Metadata' on one of the problem files (e.g., E:\Mes_Images_JC\2010\02\[2010-02-12]\JPG\DSC02521.jpg), post the output.

[jcldl]

Since I have deleted the files with warning "more than 1000..."(11 files). My memory : 8 go with win8 64 bits so I think there is no problem. Imatch uses 200 Mo of this memory and most of the time memory use is less than 60%.
The image: E:\Mes_Images_JC\2010\02\[2010-02-12]\JPG\DSC02521.jpg which had a warning for" more than 1000" has been deleted, but nothing changes

[jcldl]

Nobody has ideas? thanks

[Mario]

The log file starts with error messages from ExifTool like:

Error setting file time E:/Mes_Images_JC/2007/08/[2007-08-15]/RAW/PICT2444.xmp_exiftool_tmp

Then ExifTool reports

It took 2 tries to rename E:/Mes_Images_JC/2005/11/[2005-11-19]/RAW/2005-11-20 009595.xmp

so it seems that ExifTool is having trouble setting file information for temporary files it created itself. Very unusual. What kind of drive is E:? Maybe a security-related issue? Your user account not allowed to write?

There are many files which contain obviously broken metadata, or at least more than 1000 (!) keywords. Or the file structure is damaged and ExifTool ends up in an infinite loop while trying to extract the data. ExifTool reports this  Excessive number of items  for many of the processed files (see the log file for details).

These two issues (ExifTool needing to retry to rename files, set the file date, and the  broken metadata) is what's causing the performance to drop so badly. These are very unusual problems!

I suggest the following:

First remove all files reported as having an excessive number of dc:subject (keywords) from the folder. You may send me one of these files for analysis.
Then retry to rescan the folder in IMatch.
Check the log file for remaining warnings (search for W> to find them in the log file). If you again get warnings related to renaming files or setting file date and time, check the security settings of the folder. If E: is an external disk or network storage, check cable connections etc. as well.

[jcldl]

Mario thanks for your help. My disk is an internal disk which I think has no problem. I check the disk for errors and authorisations, nothing.
I have deleted pictures with" errors >1000 keywords". But nothing changes. I check the difference between two pictures and you can see that the only changes in xmp are  metadatadate, modifydate and instanceid (255.jpg). The imatchlog (imatch5_log3) has only one warning with w> for:" Cannot find a dictionary for language 'fr' or failed to load.  'IMatch.cpp(2985)" I have no french dictionnary.
You can see also activity (256.jpg) with as always "entry in queue of over 6000 and reading or writing of many metadatas.

[attachment deleted by admin]

[Mario]

This log file looks OK. IMatch is writing metadata to your files. It takes between 2 and 4 seconds to update 10 files, so for 6000 files it will take about 30 to 60 minutes to complete the write-back.

This write back has been triggered because IMatch imported the EXIF/IPTC/GPS data from your files into the XMP record, and is now synching the data back into the files. This not only updates the XMP, but may also update existing IPTC/EXIF etc. You are only looking only at the XMP file so you don't see this.

BTW: Are you using XMP files with JPEG files? It seems so, from your screen shot. This is non-standard and may cause all kinds of problems. Or do you also process non-JPEG files?

Do you use any unusual metadata settings?
Maybe IMatch 'creates' new metadata when it imports your files (usual) and if you have enabled background write-back (under Edit > Preferences > Background Processing) it will write the changes back to the files immediately. If there is now the odd chance of unusual metadata settings (especially how to flatten keywords), the re-import following the write may again cause a change in the file metadata, which then forces another write back...

Keep an eye on the Info & Activity panel. If the number goes down to zero IMatch should stop processing metadata. If it starts to write-back files again, we need to find out what combination of settings, in-file metadata etc. you have which causes this. This can be tricky.

[jcldl]

Thank you.
I do not use xmp with jpg files, only with raw. Unfortunately count is always around 6700 and never goes down or only to 6500 but immediately go back to over 6800, even if I see writing numerous metadata . And it's like that since over 7 days!

[jcldl]

Checking my pictures, I found many with the yellow pencil, if I click on it it desappears the reappears immediately. Clicking right I have: "xmp::dc\subjetct, writing impossible". When I look this xmp file it is a writable file for windows. Could this explain my problem.

[Mario]

Checking my pictures, I found many with the yellow pencil, if I click on it it desappears the reappears immediately. Clicking right I have: "xmp::dc\subjetct, writing impossible". When I look this xmp file it is a writable file for windows. Could this explain my problem.

You are not giving me much to work with.

Please retry this test, but this time:

1. Set IMatch to debug logging (Help > Support > Toggle Debug Logging)
2. Open the ExifTool output panel (<F9>,<O>)
3. Write-back (click the pencil)
4. Select the output in the output panel, copy, paste into a text file and attach
5. Attach the log file

I have never seen ExifTool reporting "writing impossible" so I cannot say what this means. Seeing the full output may tell me more.

[jcldl]

Thank you, in fact I did a mistake, it is not indicated "write impossible" (it was what I thought but not what was writen). In the window opened when I click on the pencil is writen:" this file has non writen metadatas, click on the pencil to write them. xmp::dc/subject. "
But as I said after clicking on the yellow pencil, the pencil disappears and on "exiftool output" is what is writen on the file "exiftool.txt". And two seconds after the yellow pencil reappears.
I join exiftool.txt and debug file and the pict2010.xmp

[attachment deleted by admin]

[Mario]

IMatch writes keywords to the XMP file without any problem. The ExifTool output is OK.

In your log file, there are tons of write-backs and re-imports. For example:

09.26 16:25:13 IMatch writes data successfully to PICT2010.MRW and 96 other files.
09.26 16:25:34 Data is re-imported from that file.
09.26 16:25:47 IMatch checks the PICT2010.MRW file (file is current) because it receives a message from Windows about changes in that folder
09.26 19:56:18 IMatch writes the file again
09.26 19:56:20 IMatch re-imports the data (OK after write-back)
09.26 19:56:37 IMatch writes the file again (because of a propagation)

IMatch also often writes back JPEG files, but ExifTool reports that it has nothing to do!
This looks like you are using file relations and versioning. Please give details about how you have configured the file relations for your database.

I also see that IMatch is bombarded by "system settings changed" messages by Windows. These messages are sent by Windows when global system settings change and applications may have to adapt their screen layout, font metrics and other important settings. Usually these messages are only sent when the user works in the control panel, changes display resolution etc. Do you have an explanation why these messages are sent so often on your system? Where you working in the control panel? Do you run any unusual applications? Please try to close all other applications and whatever little tools you may run while diagnosing this.

[jcldl]

Finally I have deleted my database and build a new one. I have no problem with the new one. Thanks Mario for your help.