trial version

[mvkuilen]

I have recently been looking for a DAM in order to bring some order to my many photos. I can usually find a memorable picture but it's getting harder and harder so I figured, let the computer do the hard work of remembering.

I downloaded the trial version and now I'm waiting for the database to be ready. I dove right in to the deep end and pointed it to all my photos to evaluate the software. Oops, that may have been biting off too much. Almost 357,000 files and the after a long Importing of Metadata, I'm now watch adding and Updating crawl along, it bounces around somewhere between 56 and 122 hours remaining. I could dismiss it but the notice says it will take longer and response will be slower.

Can't wait for this to finish so I can start evaluating and see how I will be able to integrate this into my work flow.

I use DownLoader Pro to take my Canon raw files from the SD card and place them in a new folder. At the same time each file gets named with a date-time-sequence_number. I'm now using DxO to process these Canon raw files. On an irregular basis, I copy the folders to a larger network drive for safe keeping and to make room on the local drive. Older files are moved to a second network drive as space is needed.

[Mario]

370,000 files is an enterprise-size image collection. There are image agencies out there which have less images.

This amount of images would exceed the capabilities of many DAMs out there, requiring you to split the database.
Some of the high-end DAM vendors would recommend a dedicated server machine or even server farm to manage your amount of files.

IMatch can handle this, but of course there is no free lunch and the initial indexing can take a while.
You did not write which kind of computer you run, if the database is on an SSD (good!) or a normal disk. Your images seem to be on a network disk, which usually means some sort of NAS box. Processing images on a NAS can be 5 to 50 times slower than on a local disk, this has to be taken into account. If your NAS is not connected via a cable-based GBit network but via a WiFi, this will further slow things down.

You are processing enterprise-grade data volumes on your (most likely) 'regular' hardware. So IMatch will take a while to process all this data. But it will come to an end.
If IMatch or a component used by IMatch crashes during the process (which can happen if badly corrupted files are encountered), IMatch will continue where it stopped the next time you start it again. No real problem. You may need to remove / fix the broken file in your image editor before you restart IMatch, if the error happens all the time.

Besides that, let IMatch run. Run a compact & optimize from time to time (IMatch will prompt you), this can improve the overall performance significantly.

[mvkuilen]

"... enterprise-size ...", hmmm, prolific but not that much. They include scanned photos from the family photo albums going back to the late 1800's. Also a lot of "junky stuff" kept for those just in case moments. Many of those are from places that don't exist any more of can't be visited again.

This is being run on an i7 920@2.67 Ghz with 12 GB of ram, using Win7 64 bit.
Most of the images are on NAS hard drives. Task manager reports the LAN utilization between 94 and 99%
The 8 CPU cores are only pushing 5% usage and 3.6 GB of memory is being used.

The time remaining is jumping between 771 and 1066 hours which puts it past the 30 day trail. I better stop and create a new database of a much smaller subset of the images.

Will adding extra folders a few at a time be faster than trying to do them in one fell swoop?

[Mario]

If the network is utilized at nearly 100% but your CPU's are idle (despite the fact that IMatch utilizes them all when ingesting files) the network is the culprit. It delivers the data as fast as it can, but not nearly fast enough to make IMatch busy. This will not change much when you add folders individually.

Note: You can close IMatch at any time and start it later. It will continue to work where it left of.

Also: Maybe you can make the system run smoother and faster by changing how many parallel "threads" IMatch runs when ingesting files. This may lower the utilization of your network and may speed up things.

See the article Configuring Process Control for Slow Media (CD-ROM, DVD, NAS, ...) in the DAM Knowledge Base for some background info.

Besides: Initially creating a DAM that spans 200 years of files has the right to take a while.

I think you mentioned that IMatch has already read the metadata, which is usually the slowest part. If IMatch is now ingesting all images to produce the thumbnails, visual query data etc, it should be a lot busier. All CPU cores should be utilized a lot more, because after reading the file into memory, most that follows is processing data and then writing out lots of stuff into the database. I think this is also slowed down by the remote NAS and how fast it can deliver the data. If the utilization is near 100%, that's as fast as it gets, I'm afraid.

Once IMatch has indexed all files, it should run very fast on your i7 system.

[mvkuilen]

Perhaps every image was a bit ambitious so I only pointed at the local photo folder. That part went well.

Some of the more recent Canon raw CR2 files are not being displayed in the main window. The QuickView does show them but that might be from the embedded JPG. I get a popup about a new codec being required but when I follow the link to Canon's website, it says nothing is available.

I think this is for the new CR2 files made by a Canon 80D. Does anyone else have issue displaying them?

Another issue I have is displaying JPG images. For some folders they show in both the main window and the Quickview window at the bottom right. For other folders, the main window just shows a red rectangle with various symbols around the edges and a small file icon in the middle. For the JPG images, the icon contains the IrfanView logo. Again, QuickView shows the JPG image correctly.

As for older CR2 files made by a T4i (650D), some are shown in the main window while others are not. They all show in the QuickView window. Any idea what is happening?
PS:  I removed one of the offending folders from the database and using drag and drop, added it back in. Now this folder is showing in both the main window and the QuickView window. For a small folder this is easy to do but for more substantial folders, this could be a drag.

[Mario]

Some of the more recent Canon raw CR2 files are not being displayed in the main window. The QuickView does show them but that might be from the embedded JPG. I get a popup about a new codec being required but when I follow the link to Canon's website, it says nothing is available.

IMatch processes RAW files only via WIC codecs provided by the Microsoft, the camera vendor or 3rd party vendors.
You can test which WIC codecs are installed on your system and if they are able to handle the CR2 variant produced by your camera by:

1. Select one of the files in the File Window
2. Go to Help menu > Support > WIC Diagnosis.

If no codec on your system is able to handle the file, you need to get an updated codec. Windows 10 includes a number of codecs which usually do a good job. Since you run Windows 7, you need to get an updated codec from Canon (note: The links I provide in the help are as current as they can be, but Canon moves around stuff on their web site all the time). Canon now has about 50 different formats in use, all with the extension .CRW or CR2. That's the risk of using proprietary file formats, you depend on the vendor to deliver a WIC codec.

You may also want consider spending a few dollars on the very good Fast Picture Viewer Codec Pack. Many IMatch users use this with great success.

After installing a WIC codec that understands your CR2 variant, select the 'failed' files in a file window in IMatch and then press <Shift>+<Ctrl>+<F5> (Or hold down <Ctrl> and choose the Rescan command from the context menu in the file window). In the dialog choose "Force Update". IMatch then reloads the file again with the current WIC codec.

[ckwork]

I've just completed indexing my 320K ish images into Imatch 5. Yes it took a while. Doing the whole thing in one swoop seemed to take ages (even with the database on an SSD and files stored locally) and eventually resulted in a windows exception error.

Importing by folder went smoothly, though it did mean I had to be in attendance more of the time.

I too had WIC problems - hunting down the codecs for a number of manufacturers proved frustrating, so I spent the money on the Fast Picture Viewer Codec Pack which is well worth it - painless, Imatch is happy, and a few other benefits beside (can see RAW thumbs under Windows).

Now the fun begins - you really need to think hard about how you want to organise your images (categories/keywords). Imatch offers many possibilities so it can be difficult to figure out the best options. I'm struggling and have been using Imatch 3 for years!

[jch2103]

Now the fun begins - you really need to think hard about how you want to organise your images (categories/keywords). Imatch offers many possibilities so it can be difficult to figure out the best options. I'm struggling and have been using Imatch 3 for years!

Indeed! This is something we all have to work through (and in my case, it's not finished even after several years with IMatch 5), given that there's really no one best way to organize. Best advice for now: spend time reviewing the organization you already have to see how it matches your current (and future) needs. The good news is that IM5 facilitates changes if you believe they're necessary.