Move IMatch5 folders

HansEverts · December 12, 2013, 06:57:05 AM

I just noticed that of the 110 Go used on my C drive, IMatch5 takes 35 and I only have 15 left unused, which is not an awful lot. So I want to move the IMatch5 folders to a larger and safer drive. What is the best way to do that? Under Preferences I see there are many links (cache, settings, etc.). What is the best way to reestablish the correct links?
Thanks in advance for the advice .

Mario · December 12, 2013, 07:52:55 AM

IMatch stores resources and other vital data in the official Windows AppData folder. You cannot move these files and folders to somewhere else. But the amount of data IMatch 5 stores in the AppData is minimal, usually less than 50 MB (about the size of two or three RAW files).

You can move the Cache Folder (Edit > Preferences > Cache) to another disk. This folder can become quite large. Close IMatch, move the cache folder, then restart IMatch and change the folder under Edit Preferences > Cache to match the new folder.

Tip: If you have created multiple databases and deleted them yourself, not by using the Delete Database Command from the Database menu in IMatch, the cache folder may hold many files from these old database. You can free a lot of disk space by deleting this old data.

Note: The cache folder used by the currently open database is displayed in the Info & Activity Panel. If you only have one database, make a note of this cache folder and delete all other folders under the common cache root folder.

HansEverts · December 12, 2013, 08:14:24 PM

Wow, that makes a difference!! I've got plenty of space on the C drive again and the start up process is at least twice as fast..

Thanks

Mario · December 12, 2013, 10:17:33 PM

So you had many cache folders from old and deleted databases?

With 'startup' you mean IMatch now starts faster?

HansEverts · December 13, 2013, 05:48:39 AM

Yes, I had about 25 Go worth of cache of old databases.

After having taken care of those, starting IMatch was much faster.

However, the first start this morning took 3 minutes. IM gets stuck in loading specific folders, even though I did not shut it down last night with those folders open. The second start took 15 seconds and the third 20.

Mario · December 13, 2013, 08:37:38 AM

Can I have the log files from these three tests, please. Without them I cannot do anything about it or even tell you what the problem is.

See How to report bugs

HansEverts · December 13, 2013, 07:07:40 PM

Yes, sorry. Here are the log files from 2 starts, the first one took 2'15" and the second 15". IMatch gets stuck on "Loading 2013 RAW", one of my folders.

[attachment deleted by admin]

Mario · December 14, 2013, 11:51:06 AM

The slow load requires about 30s to load the database. The fast load about 18s. This can be explained by the Windows file system cache kicking in. For the first load, IMatch has to load all data from the disk, while for the second load much of the data is already in the file system cache.

In the slow load IMatch then loads files in the file window in 0.1 seconds.

This starts at 18:58:39 and is basically completed one second later.
IMatch then loads metadata from the database in 0.2 seconds (for the file window or the metadata panel).
Then IMatch calculates collections in the background (several, in about 0.2 seconds).
Then the log file shows a period of inactivity for about 90 (!) seconds. Judging from the log file, IMatch does nothing. Nothing is written to the log for 90 seconds.

Did you notice anything special during the slow start? My guess is that IMatch was blocked in a "system call" (e.g. waiting for the file system or something). Maybe a virus checker kicked in and blocked IMatch while scanning the database file?...

HansEverts · December 19, 2013, 06:55:50 AM

No, nothing special during a slow start. The message in the left-lower corner says that IMatch is loading a folder. That folder is not always the same as the one I had open when I closed the database and it is not always the same as the one that will appear on screen, but I don't know if that should be the case.

I use Windows Defender as anti virus.

I attached this morning's log file.

[attachment deleted by admin]

Mario · December 19, 2013, 11:16:01 AM

This log file contains an interesting detail.
IMatch requires about 21 seconds (!) to calculate the contents of one of your collections. Unfortunately the log file does not include the name of the collection (I've added that info for the next build).

Your database has about 60,000 files so the initial calculation of collections after the open is expected to take a bit. But 21 seconds is quite a lot.

It seems to be one specific collection, because all the other collections don't even bother to log execution time (which means they take less than 0.1 seconds).

After I ship the new build, please produce a new log file. This time we'll see the name of the collection and this may help me to analyze this further.

HansEverts · December 19, 2013, 08:52:25 PM

The collections I have modified are:
1) Label: I have renamed the yellow label in Printed
2) Relations: this collection was updated automatically, but what is strange is that it puts the NEF files under Master in the JPEG folder and the JPEGs under version also in the JPEG folder under Version.
Is that correct?

[attachment deleted by admin]

Mario · December 19, 2013, 10:51:15 PM

This depends on your version setup, and which files you make masters and versions for which of your relation rules.

IMatch maintains many collections, and most of them are automatically maintained by the system. It can be any collection but my guess would be it's one of the metadata collections (rating, label, etc.) that is taking so long. IMatch has to load a lot of data to fill some of the collections, and if the database is not in the file system cache (first load) or the disk does not perform well, loading all that data can take quite a while...

For example, to fill the collections for rating and label, IMatch must load the rating and label data for your 60,000 files. It then must analyze these records to find out which ratings and labels were used, and how many files are in each of the resulting collections. This requires quite a bit of CPU time (which current PC's usually have in abundance) and a lot of disk I/O, which causes the delay.

HansEverts · December 20, 2013, 05:23:11 AM

I probably use the simplest master-version setup possible (see attachment). The Master folder is '2013 RAW' with NEF files only. The version folder is '2013', containing only JPEG. But this differs from what the Relations collection indicates. For the time being, I limit versioning to one year of photos

[attachment deleted by admin]

BenAW · December 20, 2013, 08:43:36 AM

Since your masters are in the 2013 RAW directory, it shouldn't be in the list of dirs where IM looks for versions.
It makes no sense to force IM to look 5 dirs UP for versions if they are IN or BELOW the listed dirs.
Depending on the structure of the dirs. below the listed version dirs perhaps 1 or 2 dirs DOWN should be enough.

cytochrome · December 20, 2013, 06:26:46 PM

If this is the rule for JPG, what do you use for NEF versioning?

Francis

HansEverts · December 21, 2013, 10:44:00 AM

As you suggested, I took the 2013 RAW folder out of the List of folders and it works fine, but that option is not obvious in the help file.

I had the master 2013 RAW included, because I only want the NEF files of the current year versioning. I figured including the master file will tell IMatch to look only in that folder and not waste time on the other 25 000 NEF files.
The 5 levels up and down come from the fact that the RAW files are on 1 drive and the JPEGs on another and each at 4 levels from the root (plus 1 level just to be sure

. I switched now to Specified folders only and it works, but I had tried that before ad for some reason it did not work.

Both changes do not seem to speed up IMatch during the first start of the day.