IM5 has started freezing

[George]

IM5 has been reasonably robust for me. However, this week I have started to experience difficulties. Almost every session results in IM5 freezing - leaving for 1 hour makes no difference. All other progs running at same time continue to run smoothly. The db reports no errors. On the last session, the task manager showed 8 exiftool processes running for 1 hour. I have to use task manager to close IM5 down. I am attaching the log file - although I have edited out sensitive info  with XXX - as IM5 was so stable I switched to a 'live' db. 

[attachment deleted by admin]

[Mario]

8 ExifTool processes a normal. When you say "running", do you mean they were consuming CPU time?
What have you changed in the last week? Other images? Updated WIC codecs?

You are not running the log file in debug mode (Help > Support) so the info contained in it is minimal. Nothing unusual, except that IMatch reports that your system is low on memory...

[George]

exiftool - using various amounts of RAM but no cpu time
recent changes include upgrading office 2010 to office 2013 + diary synchronizer (Gsyncit)
This am - 6 images added - IM5 froze when image 3 was passed over to PS . at that time 4 episodes of exiftool were showing - still there after an hour - no cpu activity - however when I started to work on the last few images (outside of IM5) the episodes of exiftool increased to 6 and then stayed static

The WIC codecs have not been deliberately changed. Other progs running in parallel include PS, Outlook, Firefox remain unaffected.

The system is an older custom built rig with 4 GB RAM - so I do sometimes run into system specific problems not experienced by others.

I will switch back to my test db and also switch on the debug log

[George]

I am attaching the new log file following a freeze. I would appreciate your observations on the log content or any pointers to help me tract down this difficulty. 

However - please do not spend a lot of time on this, as I can see no other problems of this nature reported here, and given my older custom 'rig' this will be a difficulty specific to my system. 

I suspect this coincides with the upgrade to ms office 2013 on my system. Last few days have been spent bringing IM3.6 back up to speed, but I have become so use to the sleek operations of IM5 that I would much prefer to crack this difficulty, if I can.

Please be aware the log file is from my 'live' db - so I have edited out some of the text.

Thanks again - George

[attachment deleted by admin]

[Mario]

OK, lets see:

IMatch starts and uses about 134 MB of memory.
Your database has an enterprise level of files: 175,000 files and a database size of 13 GB. That's quite a lot, already.

When IMatch has the database loaded and the memory caches filled it uses only about 600 MB of RAM.

You then view images in the preview panel, collections are calculated, cache images generated,
You changed some global options.
New files are added to the database, cache images generated. Data-driven categories and collections updated. Relation rules checked and applied.
Images loaded into memory, cached, and thrown out of the cache .
File histories updated and written.
IMatch used up to about 1120 MB RAM maximal.

Then you start Photoshop and your memory load immediately shoots up to 87% in use.
Photoshop seems to keep the disks very busy because immediately afterwards IMatch has very slow response times from the database, which means that the disk is too busy to handle requests in time.

This settles after about 30 seconds later (Photoshop should be open now). Memory still low, about 88% in use.

IMatch now scans folders in the background, probably "touched" by Photoshop.

Disk access very slow, ExifTool needs 9s to extract 114 tags from a JPEG file. Something is not only using a lot of memory but also keeps the disk busy. Probably Photoshop causing Windows to swap memory in and out, occupying the disk ("trashing").

The log file ends when IMatch starts loading a new file into the Preview Panel.

Besides the low memory warning no errors or warnings in the log. Did you just copy the log or was IMatch unresponsive at that time?

[George]

Many thanks

IM had become unresponsive at that time - it just sits there as if stuck - all other progs active at the time including PS continue to respond. The mouse pointer moves but is not 'seen' by IM - I cannot shrink the IM window either. There are no messages in the 'top' bar of IM (eg no 'not responding' message) - but if I go to TM IM is reported as unresponsive, and there are several episodes of ExifTool open (with no CPU activity) There is no CPU activity from IM in TM. My only option is to close IM with task manager - and as it closes the ExifTool episodes all close too (as expected). Even leaving IM for an hour (or longer (overnight) - does not cause IM to start responding).

This log file was generated after restarting IM

Your comments seem to suggest insufficient memory for the 'load' - perhaps the recent upgrade of the MS office suite has overstretched the system resources - would this result in the IM freezes?

Again - your time on this is much appreciated - many thanks - George

[Mario]

I cannot see anything in this particular log file which indicates that IMatch is frozen. The last operation is loading an image into the preview panel, and IMatch did that 10 times before in that log file. And unresponsive UI with zero CPU load usually indicates that IMatch sits in a dead-lock (several concurrent parts of IMatch locking themselves while fighting for a shared resource), but the log contains no indication of this either. IMatch has features to monitor and prevent dead-locks and race conditions so these are usually under control.

I see potential problems in the log file right after you started Photoshop.
Memory usage shoots up. IMatch loads a JPEG file into the cache.
In the background, IMatch is updating the data-driven @Keywords category. This takes a very long time (almost 57 seconds), most likely caused by the heavy disk activity caused by launching Photoshop and PS doing it's initial major memory grab. Probably Windows is also busy making room in the RAM by swapping out memory to disk. At the same time, IMatch starts to ingest new files, which have been modified by some external application (T:\Hold1\My Pics 30\2014\Assess\Month 05\Assess 08). This all causes a heavy load on the system for some minutes, afterwards things settle back to normal.

Don't forget: Your database has almost 200,000 files. That's enterprise level - I know several professional image agencies which have maybe 50% of this amount of files and they run IMatch on dedicated computers. Not on a older box which also runs Photoshop and what else.

IMatch can do a lot but you are maxing it out here. Your 200,000 files have so many keywords that IMatch needs almost a minute to refresh the @Keywords category. I don't know if this is the normal time or if your system was just totally slow at this time because of all the other things you were running. And by the looks of it you have also several other data-driven categories, maybe even all categories IMatch creates per default. And keeping these categories up-to-date takes a lot of time for 200,000 files. Consider disabling the background category updates (Edit > Preferences > Background Processing) to speed up the system and to leave IMatch more breathing space. You then need to manually refresh the data-driven categories when needed.

[dpop100]

Mario,
Excuse me if this is hijacking this thread. You caught my attention when you mentioned the size of George's database was "Enterprise". I am approaching 300,000 images in 10 years of using iMatch and, although I am not really experiencing slowness, I was wondering if you could point me to a FAQ or discussion thread that discusses suggested workflow regarding how to manage the size. I would prefer to be able to query and filter on all of my images, but I have though at some point I should create an archive of the last 10 years and start anew. I would be interested in how others have gone about managing their large database.
Thanks, David.

[George]

Mario - Thanks for taking the time out to examine my log, and for your comments on this. 
Here is some feedback from my 'experiments'

Using your comments as indicators of potential problems I initiated a series of steps to try and overcome the freezes.

I examined the TM/services and removed auto loading of anything that was not absolutely essential.
I switched off background processing of the data-driven@Keywords category as you indicated.
I also examined the automatically generated @Keywords these included keywords made up of Mandarin/Chinese script and Arabic/Persian.
There were also some 'symbols' eg currency/financial/mathematical symbols
I deleted the entire list - with a view to eliminating this area as a source of the 'freezes'

IM5 is back to running smoothly 

Some observations:
In retrospect - I regret not implementing these one at a time - tiredness, lack of free time, and a late night glass of wine clouded my clear thinking. 
Given some of the 'interesting' words used by IM5's auto-generation of keywords - IM5 may be including too much from the metadata. Is it worth providing an option for limiting this or even for switching this off prior to ingestion of files into the db? (I think its automatically on after a fresh install of IM5?)
The size of my collection may be a factor - but it is not the whole problem - IM5 had been running perfectly smoothly until a few weeks back - Also I created another 'test' db ingesting a small sample from the folders used in my main db (eg about 70,000 files) - with keyword generation switched on. After working with this for half an hour IM5 froze - (and again - the log file justs 'stops) - this suggests (to me) that the freezes are linked (on my system) to some aspect of the keyword/categories processes during routine running - rather than total size of collection.

I am pleased to have IM5 back up and running - (IM 3.6 is an old friend - but it really showed its age when I had to run it side-by side with IM5.

Given that this is limited to me and my 'custom' rig I wish to say 'thanks' for the time you took to assist me overcome this. 

[Mario]

Given some of the 'interesting' words used by IM5's auto-generation of keywords - IM5 may be including too much from the metadata. Is it worth providing an option for limiting this or even for switching this off prior to ingestion of files into the db? (I think its automatically on after a fresh install of IM5?)

No. The @Keywords category is automatically synchronized with the hierarchical keywords IMatch manages for you. The hierarchical keywords are build on ingest time from existing hierarchical and flat keywords (if any) in the XMP data (if any) of your files.

If you enable mapping of existing "flat" keywords (Edit > Preferences > Metadata) IMatch will also look at keywords contained in the IPTC records in your files and import these keywords as well. I have tested this mechanism with quite a lot of 'rubbish', including files with several thousand (!) keywords, files with damaged/corrupted keyword records, files with keyword in non-Latin character sets etc.

I maintain a backward collection of all the files users have sent to me over the past 10 years, plus a large set of sample files from the Internet with all kinds of weird stuff in the metadata, broken image records and so on. My standard 57,000 files test collection of real life files, 10,000 of my own RAW, PSD and DNG files, agency files, photo library files, really large image files, really big files and all the problem files I have in my archive has to be processed without crash or other 'issues' before I ship a new build. Creating a database from these files is part of my standard test cycle.

Naturally, there can always be other files which cause problems in ExifTool, IMatch, one of the image libraries used by IMatch, a WIC codec, ...

If IMatch stops, you can:

a)  secure and then review the log file. The last files reported in the log file (near the end of the file) are most likely the cause of the problem. IMatch processes files in batches, and several of such batches simultaneously (depending on the number of processors you have) so it's not easy to find the problem file. But usually when you can isolate the last 100 or 200 processed files in a separate folder, and then process this folder into a new database, the crash/hang can be repeated. And then we'll have a chance to debug this.

b)  follow the instructions here to produce a DUMP file and then upload it to my FTP. The DUMP file may show me where IMatch has stopped, and why.

This may help solving this issue.