Getting started and constructing the required database.

[poppyrob]

How to Begin? Several months ago, I downloaded the trial iMatch software and spent several hours exploring it's functions and capabilities. I found it extremely beneficial and purchased the software. However since purchasing, I have not yet found the time to begin creating my database. Until now.

To create my database correctly, I would appreciate advice so I get it right the first time without encountering annoying errors that may prove difficult to correct. So here is my photographic library details:

210,484 files
2,170 folders
2.83 TB of data

My file structure is along the lines of:

PHOTOGRAPHS  ...  Parent folder holding ALL folders and files
   2003 (parent folder for the year 2003 holding many individual folders and files)
   2004 (as above...)
   2005 (as above...)
.... etc
   2019 (as above...)
   SCANNED SLIDES (parent folder)
   SCANNED NEGATIVES (parent folder)
.... etc ...
.... etc, etc .....

I would like to maintain this exact structure for my iMatch database, but to drag and drop the PHOTOGRAPHS folder directly to the database would probably take an unimaginable time to create. So what is my best option? Manually create a PHOTOGRAPHS parent folder, along with individual child folders before dragging and dropping my folders into these directories? Or what is my best option?

Once I get started, I feel very comfortable with the software and although as I have a long task in front of me, I would like to get it correct from the beginning, without duplicating folders and/or files.

I apologize if this topic has been covered in tutorials or other forum discussion, but I don't seem to be able to find an answer to this question. Perhaps I'm looking in the wrong areas.

Thank you
Rob

[PaulS]

Hi Rob,

IMatch is very flexible regarding the adding of folders.  If you are happy with the structure you already have, I would not change it to start with.

Multiple folders can be added at one time by selecting the Media & Folders tab and clicking on the left most icon, Add or Update Folders..., or choosing it from the Commands menu.

Simply start by adding one folder, noting the approximate number and size of files and how long it takes.  You can then use this for guidance to determine the optimum number of folders/ sub-folders to add at a time.  (For reference, on my 5 year old Intel i7 notebook with 8GB RAM and 1TB SSD, it took ~15 minutes for 2,074 JPGs using 12.4 GB disk space).

See also "Using the Add or Update Folders Dialog" in the online help here: https://www.photools.com/help/imatch/#rmh_dlg_scanfolder.htm.

Good luck!

Paul

[Mario]

Adding 210,000 files is not really a problem. I do that often when creating test databases.
A 200,000 files DAM system would be rated "corporate-grade" by the big DAM vendors.
Which usually means dedicated workstation/server hardware or full cloud.
Just saying. IMatch handles 200,000 files easily. Or a million.

Having a stable system (I use workstation-grade computers) is important. Reliable power supply. Storing the database on your fastest disk (SSD much preferred).
Notebooks with all kinds of power-saving, going-to-sleep in the middle of work, reducing CPU speed when getting too hot due to extended work etc. are more prone to unexpected side-effects.
See my tips below for that.

Experience tells us that there will be some problem files in such a massive amount of files.
Damaged files. Files with corrupted metadata. Files which break a WIC codec, a PDF handler or one of the 3rd party image libraries used by IMatch.
That's normal. IMatch may even crash. But your database will not be harmed and IMatch will continue when you start it again.

Tip: If IMatch crashes, secure (make a copy of) the log file (see log file) before you restart IMatch.
At the end of the log file you will find the error messages and the names of the files which have caused the problem.

1. You can close IMatch at any time while it is ingesting your files. It will continue the next time you start it.

2. You may speed up processing by running a database optimize every 20,000 to 50,000 files. IMatch will prompt you.

3. It may be helpful to make backup copies of the database.
If the worst thing happens and your PC shuts-down dirty because of a power failure or the hard disk dies, you can restore your last database backup and IMatch will continue from there.

When building new DAMs with IMatch from existing large file collections I have seen many things. Sometimes we build a database from 200,000 files in a day without any problem. Sometimes we find thousands of corrupted files which were never noticed before nobody because used them for years. Often the metadata is a mess and IMatch is the first software that shows the problem to the user...

If you process RAW files, the installed WIC codec on your system is very important. The Windows 10 standard codecs are quite robust. But don't handle all RAW files. Some 3rd party WIC codecs are known to have issues after a few thousand of files have been processed. Rebooting then helps.

If you process video or PDF files, this will create additional stress on your system. And reduce the throughput of IMatch.

1. I would first create a small test database. And then add some of your folders (or copies of your folders so you are totally free to experiment).

2. Find the settings for metadata, buddy files and versioning you need. Maybe you don't need anything special. Maybe you need to setup buddy files. And versioning. I cannot tell. Best to decide that up-front, before you add all your files.

3. If you have find your settings, create a new database and apply your settings.

4. Find a time where you can keep your PC running for a day or two.
Create the database and make the folder an exception in your virus checker. 90% of all "IMatch is performing slow" messages is due to virus checkers interfering.

Then just drag and drop PHOTOGRAPHS to the Media & Folders view and let fly.

All this is also explained in the help:

First Steps
- Creating the Database
- Indexing Folders and Files

[poppyrob]

Thank you for the replies Paul and Mario. Both extremely helpful.

I've started my database and have imported the "first" few years of images. Rather than add all the images to the database, I have begun to rate, class, keyword and generally organize my collection.

Everything is going well at this point, and although iMatch is an expansive creature containing many more functions that I will probably ever require, I am able to use the software to what and where I require.

I do however have one small question that I have yet been able to answer. I would like to be able to view the pixel dimensions of any image without double clicking in the default panel making the image full screen and thereby displaying the pixel size (eg 5202 x 3465) underneath. The pixel size is not displayed in the Metadata panel nor in any of the File Window Layouts in the default panel.

This is probably something I am totally overlooking but a point in the right direction would be greatly appreciated.

Thank you
Rob 

[JohnZeman]

I do however have one small question that I have yet been able to answer. I would like to be able to view the pixel dimensions of any image without double clicking in the default panel making the image full screen and thereby displaying the pixel size (eg 5202 x 3465) underneath. The pixel size is not displayed in the Metadata panel nor in any of the File Window Layouts in the default panel.

This is probably something I am totally overlooking but a point in the right direction would be greatly appreciated.

Thank you
Rob

Welcome to the wonderful world of IMatch Rob.

At the top of the file window you can click the icon for file window layouts and choose the option to edit.  Once you do that you can customize your display to show almost any information you want for your thumbnail images.

Attached is a screenshot of my most commonly used file window layout which include image pixel dimensions.

[poppyrob]

Woah! Thank you for the EXTREMELY rapid reply John.

I'll give it a try immediately.

Thank you
Rob

[JohnZeman]

You're welcome Rob.
You can do a lot in the file layout editor.
For example if you look at my above screenshot you can see I told the layout editor that in order to conserve space on the thumbnail to replace specific long words with abbreviations.  National Park shows as NP in the thumbnail, State Park shows as SP, and all state names are replaced with the 2 letter abbreviations for the name of the state (California shows as CA).

Help explains how to do all that and so much more.

[JohnZeman]

Ha!  I'm glad you posted this question Rob, as I looked at my screenshot I realized I had a typo in my configuration to replace state names with abbreviations (Alberta should have displayed as AB. 

Now it does.

[poppyrob]

Thanks again John.

This software is amazing and so very, very detailed. The more I look around, the more I find. I don't know if I will ever fully utilize a lot of it, let alone remember where to find it again! I think I've got a headache.....

My Layouts are now looking as I had hoped and after the editor tip, I have changed them either further!

OK, I better get back to it. There are still many, MANY images to process.

Thank you
Rob

[Mario]

This software is amazing and so very, very detailed. The more I look around, the more I find. I don't know if I will ever fully utilize a lot of it (...)

IMatch is developed in close cooperation with its users - no marketing / sales departments which puts in demands 
This means that all the options and features exist because at least a larger share of the IMatch user base needs them.
Most IMatch users use about 50% of the software, but each user uses different 50%.

Customizing file window layouts is already a more advanced thing to do (Tip: Left-click in the header of a file window panel to quickly select the information you want to see in the panel).
For more advanced things, use the full layout editor. @John: Abbreviating long terms is a good idea. I assume you are doing it with variables? My, IMatch is flexible, isn't it?

Once you've got your files indexed and organized, you can start to explore all the cool features in IMatch. The Visual Index is a good place to learn about all the features in IMatch. Each main feature is explained with a few words and there are links to the corresponding sections in the help.

The IMatch Learning Center offers free video tutorials for IMatch.

let alone remember where to find it again!

IMatch 2020 (currently in the works) will include a neat new app which makes this easier.

Until then I recommend creating a simple text file in a folder and write down your findings there.
- Why you changed a certain option.
- How to fill in metadata
- Your personal workflow
- ...

If you click this text file, IMatch shows its contents in the Quick View panel.

[jch2103]

I do however have one small question that I have yet been able to answer. I would like to be able to view the pixel dimensions of any image without double clicking in the default panel making the image full screen and thereby displaying the pixel size (eg 5202 x 3465) underneath. The pixel size is not displayed in the Metadata panel nor in any of the File Window Layouts in the default panel.

One more thing in addition to the excellent advice above: There's a file window tip in the thumbnails (hover your mouse over the 'i' on the right side). This can be configured in Edit/Preferences/File Window Tip. One option there (it may be a default, but I don't remember) is to show image dimensions.

[Jingo]

So many wonderful ways to customize the thumbnail display.... if you go back and forth between the editor and the main view - don't forget about the Ctrl-Click shortcut to go directly the "pro" editor bypassing the "simple editor".  Here is my thumbnail display to show another option... you can display a TON of info right on the thumbnail... even icons to highlight different info!

[Mario]

if you go back and forth between the editor and the main view

Tip: while editing a layout you can see your changes immediately by pressing <Apply> in the dialog.

[sinus]

Here is my thumbnail display to show another option... you can display a TON of info right on the thumbnail... even icons to highlight different 

Very nice, Andy!
Yes, indeed, IMatch gives us a ton of info to display.
If someone has a problem to solve, often users (and of course Mario) can give one or even several solutions.
Here is my a bit complicated layout, but it works fine and quickly.

Have fun with IMatch! 

[ColinIM]

Your thumbnails themselves make beautiful pictures Marcus 

[sinus]

Your thumbnails themselves make beautiful pictures Marcus 

Thanks, Colin!
I am sure, other users (including you  ) have very interesting layouts.
Hm, we should maybe once open a thread for different layouts. 

[poppyrob]

Thank you all for such comprehensive replies.

Not only have you answered my question, but you have all provided further information to expand my iMatch knowledge and experience.

I have however encountered another question, not really a problem, that you may be able to assist but for that I will start another thread as it is a completely different subject.

Again, thank you
Rob

[Jingo]

if you go back and forth between the editor and the main view

Tip: while editing a layout you can see your changes immediately by pressing <Apply> in the dialog.

Thx Mario - this works perfectly... but sometimes I need to exit because I need to grab tag values from the metadata panel or find a particular image that has certain data to see how the change looks as I'm hitting apply... in these cases, being able to jump back into the editor with the ctrl short cut saves some time (and I thank you for that)!