Out of memory and ActiveX problems

[ccs]

I have recently converted a 500K image database to Imatch 5 (ver 5.3.12) and have been experiencing performance problems that are hard to characterize.  The database that I converted passed diagnostics/optimization before and after conversion.  imd5 files have been excluded from virus scanning.  However, I routinely encounter "out of memory" problems and/or the program hanging up and terminating.  I typically do not have other programs running at these times.

I don't know if it is related, but I need to import my properties into attributes via CSV, but I have huge csv files to deal with and have yet to have a file >10 lines succeed.  One error that I receive is an ActiveX automation error "unspecified error".  I have also had the import dialog tell me that the attributes were successfully imported, but when I look no data have been entered. 

I apologize for providing such vague details, but any help as to as solution would be greatly appreciated.  I can try to attach a log file, but have also had trouble opening those on occasion.

Thanks in advance for your help

[Mario]

Please see also How to report bugs for more info.
The first thing we need would be a log file in debug mode. See log file in the IMatch help for more information.

You did not specify during which operations IMatch reports out of memory. The most memory-intensive operations in IMatch are the Viewer and the Slide Show.

As a general note: A database with 500,000 files is a enterprise-level image collection. Competing vendors would manage such a database on dedicated multi-server systems, with consulting contracts and annual five-digit license payments. A 100 US$ software may not be the best solution to handle 500,000 files.

Please understand that I design and construct IMatch for the 'typical' user, which manages between 20,000 and 200,000 files. Maybe even 300,000 files - tops. But there are limits of what can fit in the ~ 3.5 GB of RAM IMatch can use. And all the cool and useful features added in IMatch 5 require memory.
Exceeding the upper limit by 2 and a half may just break IMatch. I don't test such file volumes because, most likely, you are the only user in the world who tries to stuff half a million images in IMatch. No sense in trying to optimize the system for that.

The CSV import sample script was not written to handle CSV files with hundreds of megabytes or anything. It's a demo of how to implement a user interface in HTML and how to import text data from files into IMatch Attributes. It loads the entire file into memory, in a collection object. This works well for CSV files with 20, 30 MB. But if you attempt to load a CSV file which cannot fit in memory, you will need your own purpose-built script which processes the file directly, without loading it into memory. Processing files of such size is enterprise-level data processing.

I suggest you look at the offerings of companies like Canto, Oracle, FotoWare or Extensis. They offer dedicated server platforms where separate hardware is used to store and maintain the image catalog. They are also more expensive that IMatch, though. They will be happy about a new customer. 500,000 files is much more than a typical stock photo agency manages...

[ccs]

Thanks Mario.  That's disappointing, as I have been using Imatch since around 2001.  The same database worked fine in the prior version of Imatch, and those other services are well out of my reach.

[Mario]

Without even as much as a log file or some info about when you get out of memory messages, I cannot say anything more.

[ccs]

I am just having some trouble generating a log file from within Imatch 5 (without it hanging), but will attach one as soon as I am able.  I was also considering creating a smaller database first to see if the size of the database is my problem.  I was just very much hoping that database size was not the issue, in light of my many years of happy use of this program thus far.

[Mario]

You don't need to 'generate' the log file. IMatch automatically maintains the log file. It's always generated and you can always access it in the TEMP folder. See log file in the IMatch help for all details.

[Mario]

If you can provide a log file, we may be able to help you solving the problems you are facing.