Should I update my computer?

[sinus]

Hello Mario
I have a question, what for you can be curious (not to say silly  ).

With IM3 I had never problems with crashes or slow responses. Photoshop CS6 works fine, except if I load, say 30 images (4000x3000 pixel), then it becomes also a bit slow, but I can easy work with it.

But with IM5 I have really problems. It crashes seldom, but "hangs" with a completely white monitor often between 1 minute or 10 minutes, and sometimes I end the program.

Also, if I change from the folder-view to the collection - view, for example, it go white for 1-2 minutes, before the collections are there (at the moment I have only a few dots, pins and flags added, most images from 200'000 have no collection) and I can work further.

If I do a diagnosis or compact/optimise, all seems to be ok.

Now I begin to think, that my computer is simply to slow, have not enought RAM or whatever.
Hence my question:

Is there a simple way (I am not a computer crack) to see, if my computer is good enough or not for IMatch?
Because I have a quite big db, this could be the reason (though with IM3 I had never problems, but IM5 is much more complex, I guess).

If so, then I could update my computer and see further. If my computer is good enough, then I could look very carefully on IM5.

IF IMatch 5 does run fine and correct, then it is a pure fun work with it, really. But yesterday, for example, I had to create 50 photos for a client. I tried it in IM5, but had so long to wait (white screen), that is was almost not possible, switched to IM3 and after short time it was done.

I know, BTW, that I did never upload a logfile or so, because I thought always, it is my fault or I want not to use your precious time for look into my log. But I begin to think, it is not me  ... it is my big db, or IM5 itself or my "old" computer.

If I click on "over IMatch" and then on "Systeminformationen", then I get this, but I do not know, is this enough to say ROUGHLY, if this is good enough:

Betriebssystemname   Microsoft Windows 7 Home Premium
Version   6.1.7601 Service Pack 1 Build 7601
Zusätzliche Betriebssystembeschreibung    Nicht verfügbar
Betriebssystemhersteller   Microsoft Corporation
Systemname   LARA-7
Systemhersteller   Acer
Systemmodell   Aspire M5910
Systemtyp   x64-basierter PC
Prozessor   Intel(R) Core(TM) i7 CPU         870  @ 2.93GHz, 2934 MHz, 4 Kern(e), 8 logische(r) Prozessor(en)
BIOS-Version/-Datum   American Megatrends Inc. P01-A3, 17.05.2010
SMBIOS-Version   2.6
Windows-Verzeichnis   C:\Windows
Systemverzeichnis   C:\Windows\system32
Startgerät   \Device\HarddiskVolume2
Gebietsschema   Schweiz
Hardwareabstraktionsebene   Version = "6.1.7601.17514"
Benutzername   lara-7\zuki-7
Zeitzone   Mitteleuropäische Sommerzeit
Installierter physikalischer Speicher (RAM)   8.00 GB
Gesamter realer Speicher    7.99 GB
Verfügbarer realer Speicher    5.49 GB
Gesamter virtueller Speicher   16.0 GB
Verfügbarer virtueller Speicher   13.4 GB
Größe der Auslagerungsdatei   7.99 GB
Auslagerungsdatei   C:\pagefile.sys

[Mario]

When you see the white screen, what is IMatch doing?
Do you have the log file?

There are a dozens reasons IMatch may become to busy to update the UI. Writing back. Reading, Calculation categories. Collections. Without the log file I cannot tell what's slow and how to speed it up.

I develop and test IMatch on a PC which is five years old. I have a SSD and 8 GB RAM. My largest database has 180,000 files.

[sinus]

I develop and test IMatch on a PC which is five years old. I have a SSD and 8 GB RAM. My largest database has 180,000 files.

Thanks. So I think, my computer is still good enough. (SSD I do not know, will read about it, if I can do this also).

The next time, when something will happen, I try to catch the logfile.

[cytochrome]

During the IM5 beta test period I had a lot of crashes, endless white screens, I spent (cumulated) hours watching the task manager and trying to figure if there was still life.. To the point I stopped for some month using the beta and I wondered what I will do with my photos once IM3 was no longer supported because IM5 was so slow and unstable.

Then, for mainly other reasons, I changed my old Dell for a new computer,  i7-4770 CPU @ 3.40GHz, 3401 MHz, 4 cœur(s), 8 processeur(s) logique(s), 16 Go RAM, system (and IM) on SSD plus 2 HD (2 and 3 To), USB3, etc, and most all problems were solved. There are still some white screens but under a few seconds, and I don't remember when the last crash happened.

I should add that my workflow is heavy because I use no sidecars, so Exiftool is running a lot to update raw files, your workflow is different so the computer bottleneck may be less severe. And also since the beta Mario has improved a lot the speed of IM.

Nevertheless I think a faster machine and more RAM would makes life easier for IM5, and for you too, although your present machine seems quite up to date. Maybe also you have too many other programs or services running in the background?

Francis

[sinus]

Thanks, Francis

much appreciated, your answer.
So my machine seems to be not that bad

I will try to look, what other prog are running in the background.
And also I will look, if I can add a ssd or whatever to my computer.

What is for sure, that IM5 has much more power and mighty than IM3, but that has also its prize for the computer.
Because IM3 is running super stable and quick on this computer, and IM5 not yet.

But I will try to optimize everything.
At least I am excited, what IM5 can do! Really!

[herman]

It has been said that performance gets a real boost when you have your database on a fast drive.
An SSD or a fast USB3 stick are way faster than a fast "spinning" hard disk.
Personally I use a spinner (actually a set of 7200 RPM disks in RAID 1 configuration) for storage of my images and documents.
One 128 GB SSD for Windows and applications.
A second 128 GB SSD just for databases and caches (ASP, DXO, IMatch).
This makes most applications fly 
When your PC has a USB3 port it may be worthwhile to buy a fast USB3 stick and use that for your database to see what that does for your performance.
Mind you that there are differences in USB3 memory sticks, not all sticks are fast performers.

[Richard]

My laptop's specifications are close to yours and I have never seen a white screen but my database but is a tiny fraction of yours. And I do use the fastest USB 3 flash drive I could find. Since a SSD is many times faster, that is what I would recommend that you add for your database.

[cytochrome]

I think Exiftool is a major power user. So it really depends how you run IM. If with sidecars and no keywords it should be fast and no white screen. If you write metadata and keywords to the image files it comes at a cost.

I didn't try a fast USB stick (had no USB3 before switching computer) but will do, my DB is only 3.8 GB. But in my case I think it is not the DB access that may slow things but that a fast HD that is really crucial since IM has to update the images very often. At present the image files are on a SEAGATE internal 3.5" BARRACUDA 3TB 7200 rpm. It doesn't fly but it is OK (and silent...)

Francis

[Ferdinand]

The two things that I did that helped were to put the DB on an SSD, and to turn off the automatic update of data-driven categories (preferences | background | other). Actually a third - don't enable auto-write-back, so that I control when it happens.  Using sidecars for RAW is a good idea, although it you have a lot of large TIFF then write-back to those files ain't going to be fast.

[Mario]

There are still some white screens but under a few seconds, and I don't remember when the last crash happened.

If you experience a white screen, please use the Help > Support > Copy Log File command right afterwards and ZIP/send me the log.

I'm tuning things in IMatch all the time, based on the performance stats I collect in the log files. Whenever I get a log file, I check how IMatch performs and try to figure out how tune things. IMatch has to perform many, many things to work all the magic, especially collections and data-driven categories can slow down the system.

One of the biggest problems recently was caused by the Pending Write backs collection! On some computers, IMatch took 60 seconds (!) to calculate the number of files in that collection. Because it requires IMatch to scan the entire table holding the metadata to find out files which have at least one updated metadata tag. The effect is especially bad for the users who have 200,000 files in the database, or about that number. I have users where it takes 10-20 seconds to calculate the collection, and for others it takes 60 seconds. Same size of database, just other hardware.

The first measure to counter that was to implement a cache for that collection. IMatch now needs to calculate it only once. This change was shipped in build 5.1.10. For the upcoming 5.1.12 release I played with the database, trying to find a way to calculate the information faster. I was successful and most users should see a 50% improved performance (for large databases > 100,000 files). For small databases it's fast (< 1 second) anyway.

Usually execution times like this are not a problem because IMatch performs them in the background. But when IMatch needs to results of a collection in order to update the screen and it cannot get the results fast enough (e.g. because the database is currently busy because the user is indexing files or writing back data), the UI may become unresponsive for some time. This is the white screen or ghosting effect.

Again, this depends on what the user is doing, what IMatch is doing etc. The log files give me useful information about these situations. It' not always easy to find out what exactly happened, or why IMatch suddenly needs to wait 5 seconds to calculate a data-driven category instead of 0.25 seconds, but I'm also adding better performance logging code in each release.

IMatch 5 has a lot to offer. But if a user utilizes all that's there (collections, data-driven categories, many panels open, filters maybe) and combines that with a 200,000 files database, the hardware may be pushed to and beyond the limits.

[cytochrome]

OK, I will do  next time I see one. As said they are short now.

And thank you for the detailed explanation.

Francis

[Erik]

The two things that sped things up a lot for me was moving the DB to the SSD and changing codecs (of all things).  It seems that often when IM would rescan an image due to updates outside of IMatch it would also update the cache image (which makes sense), which slowed down the whole process.

I'm still in a phase where I'm cleaning up my database and workflow for the future.  Starting over with a few ideas, etc.  I suspect we're all doing that to some extent.  I'm finding that as I make progress, IM5 is running much better.  Working with hundreds to a couple of thousand files at a time does take time if EXIFTool is used. 

The biggest challenge now is dealing with data driven categories and how many I need (versus how many I think I want).

[jch2103]

Markus -

Your system information list doesn't specify what kind of storage devices you have, but I gather it or they are rotating hard disks. As several posters have noted, putting your database file on an SSD (or USB 3 memory stick) can have a dramatic impact on performance (also helps OS performance). The IMatch DB requires lots of random reads and writes, which are much faster on solid state memory than on rotating disks; image files don't benefit to the same extent because reads and writes of them are sequential. Other tips mentioned above sound good also. And see https://www.photools.com/community/index.php?topic=2102.0

[ubacher]

Yes there are still a lot of performance /bug issues in IM. But it is so fantastically powerful and versatile that I find it
worth struggling with it and helping to improve it.
In order to pin down problems I have resorted to use a minimalist configuration:
No data driven categories,
No stacks
Few file relations/versioning
No automatic file indexing.
Almost completely default settings.
I do have about 200 000 files though.

This way I know when there is a delay/hangup that it is not caused by any of the above.
And I can confirm this periodic delay (white screen) which at times does end up so long (many minutes) that I have to kill it.
(I find running diagnostics/optimize helps in those cases to get going again).
I am glad when I find other users reporting problems I am encountering too - this way I don't feel I am bugging Mario with
problems which  (seem to) only occur on my machine.

[sinus]

Hey folks,
THANKS a lot for your answers. I will write moore  in the next time, because at the moment I am only at my Handy. Sorry!!!

[cytochrome]

Hello,

After reading this discussion I bought a fast USB3 memory stick (sandisk extreme) and copied my DB (3.8 GB) to it. It is clear that IM runs substantially faster but since Mario continuously improves the speed of the program it is difficult to ascertain what is really due to the USB3 memory.

I will test whether transferring also the cache to the stick can further augment the performance.

Francis

[Mario]

And I can confirm this periodic delay (white screen) which at times does end up so long (many minutes) that I have to kill it.

Always keep the log file because it will tell us which operation blocked IMatch. If the Compress improves things, the problem is disk I/O.
You should have noticed a considerable increase in speed with version 5.1.12 because the slow case you reported (calculating write back files for a 200,000 files database) is no more. This is now cached and 10,000 times faster.

[voronwe]

I had also the problem in the beginning of running IMatch5, that it often stays in the [Not Responding] state - not for long, but you know, waiting is allways irky.
So that gave me the final kick to do what I allways wanted to do: Tune my PC. So I bought additional 5GB of RAM (it is 8 now) and a 128GB SSD, where I did a full new installation of Windows 7.

The SSD is a good thing for all program - damn, Firefox is starting like hell now, but I also had the feeling that working wit IMatch is also more smoother now (I have also the DB on the internal SSD.

So I'm not sure if the better performance of IMatch is now because of the SSD or of Marios fixes (Maybe both of them - sometimes I have the feeling that every time I start IMatch I have to install a new version with new fixes  )

[Mario]

IMatch does not directly benefit from more RAM - but Windows will be able to cache more in memory and if you run memory hogs like any Adobe product, you will have more memory for other applications. Unless Adobe ships a new update which needs even more memory.

The SSD is what speeds up everything.

sometimes I have the feeling that every time I start IMatch I have to install a new version with new fixes  )

IMatch 5 uses an agile development model. My aim is an update every two to four weeks. More often if I fix a potentially harmful bug.

[voronwe]

IMatch does not directly benefit from more RAM - but Windows will be able to cache more in memory and if you run memory hogs like any Adobe product, you will have more memory for other applications. Unless Adobe ships a new update which needs even more memory.

But it was anyway good to give it more memory, because in a workcase I run IMatch, Gimp and Scribus at the same time (not to mention Firefox which is a monster memorywise. It makes work generally more comfortable (and also my Ubuntu VM gets benefit from it)

sometimes I have the feeling that every time I start IMatch I have to install a new version with new fixes  )

IMatch 5 uses an agile development model. My aim is an update every two to four weeks. More often if I fix a potentially harmful bug.

Maybe my text could be misunderstood: That was positive feedback: Thanks for replying on bugs that fast, keep on (But I think nobody would be angry if you stay away a longer time for vacation). Unfortunatly there is no "Like"-Smiley, maybe that could be included in the forum

[jch2103]

IMatch does not directly benefit from more RAM - but Windows will be able to cache more in memory and if you run memory hogs like any Adobe product, you will have more memory for other applications. Unless Adobe ships a new update which needs even more memory.

The SSD is what speeds up everything.

I recently increased the memory on my 'old' Q6600 processor desktop (already had a 256GB SSD). The extra memory does make a difference in responsiveness when I'm running IMatch plus other programs (Chrome, PS, etc.); before, there were times when there was very little free memory left and things did seem to slow down. I suspect ExifTool instances take advantage of some of the extra memory even if IMatch doesn't.

[Mario]

ExifTool usually takes between 5 and maybe 20 MB RAM per instance. This is not Adobe software 
When processing huge files, ExifTool may temporary allocate 100 MB or so, but only the instance processing the given file.
But, in general, more RAM is like more cylinder capacity...the more the better...Windows can use RAM to cache many things, and it is much more likely that IMatch can allocate larger consecutive memory blocks without running into Windows heap fragmentation issues etc.

[loweskid]

I also get the 'white screen' sometimes but my computer is getting past its sell-by date and I intend to upgrade later this year.

Forgive me for going slightly off topic but following on from the discussions here about SSD drives I was wondering if the recommendation about having the Photoshop scratch disc on a dedicated drive still applies to an SSD drive?  Would it be a good idea to have a SSD drive for the OS and/or database, plus a separate one for the Photoshop scratch disk?  Or is that over the top?

[jch2103]

Forgive me for going slightly off topic but following on from the discussions here about SSD drives I was wondering if the recommendation about having the Photoshop scratch disc on a dedicated drive still applies to an SSD drive?  Would it be a good idea to have a SSD drive for the OS and/or database, plus a separate one for the Photoshop scratch disk?  Or is that over the top?

I suspect it's over the top/out of date, although I haven't benchmarked it. I assume the reason for the original recommendation was to maximize random access speed/avoid mechanical access latencies in random access. An SSD pretty much gets away from those issues. I suspect it would take an enterprise situation to saturate the SSD drive controller sufficiently to justify another SSD drive.

[loweskid]

I suspect it's over the top/out of date, although I haven't benchmarked it. I assume the reason for the original recommendation was to maximize random access speed/avoid mechanical access latencies in random access. An SSD pretty much gets away from those issues. I suspect it would take an enterprise situation to saturate the SSD drive controller sufficiently to justify another SSD drive.

Pretty much my thoughts as well.  Thanks.