Database repair v slow

[chips]

Initial impression of iM5 was how fast it generated a database.  This week the database is corrupted after a series of software/hardware issues and I elected to scan and repair.  At first the prediction was 171 hours to completion!   Now it's down to 3+ hours but every time I check progress it is 3+ hours.   It's been running approx 17 hours now.  Is there a way of seeing what is actually going on.  With 1M3 the progress window woul show which files were being processed.  Not keen to switch off repair in case I have to start again from scaratch.

[Mario]

What do you mean by scan and repair?
IMatch has no such feature.

Please attach the IMatch log file.

[chips]

Apologies for imprecision.   Database diagnosis reported corruption and/or damage.  I have tried to "update folders" and "add new folders" and it is taking an age.   LOg seems very large - six attempts at reply have stalled,  Cheers DSA

[Mario]

What kind of damage? Physical database damage or just a problem the diagnosis can fix?

IMatch writes a log file (see the IMatch help for details) and the diagnosis also writes a log file. These files can be found in the TEMP folder on your system and are named IMATCH5_LOG.TXT and IMATCH5DIAG.TXT. You can copy both files in Windows Explorer and attach them here (zipped). This will give us some info to work with.

[chips]

Log proved too big even when zipped.  Machine froze again in the early hours and I've just found the problem is with the hard drive on which the imatch database is stored - and of course the newest and most expensive of all my disks.  This would certainly explain the original damage report and the slow update and the huge log file.  I am just about to inspect and try to complete the database so if you hear screaming...

[chips]

Onward and upwards but still excruciatingly slow (unless my memory is playing tricks0.  New hard disk installed and building database again from scratch.  10 years worth of press photography. The annual folders are created in reasonable time  but my attempts to capture metadata are s-l-o-w.   I need caption , keywords, headline, city and capture date.  Bearing in mind I have done this before it seems a hellish complicated endeavour.  All the instruction I can find is for migrating a database from imatch3 (long gone) .  I do not want to re-order or re-categorize pictures - they are all in dated folders so I want to preserve that structure and search the categories mentioned.  First time I did this (on a much less able computer) the speed in comparison with im3 was astonishing - not so now - it's been two days and has not yet applied metadata to year 2.  perhaps tomorrow if there is time and energy to spare the log will be forwarded - as for now - sleep is imperative.

[Richard]

I don't understand why you are "building database again from scratch". I would think it would be faster to revert to a backup from before the disk problem started and update that.

[Mario]

I'm not exactly sure what you are doing.
You had hard disk problems and you now need to create a new database, right? Because you had no backup?

IMatch has to ingest all your files, extract metadata, create thumbnails, cache files etc.
The performance here depends of course if you ingest 50,000 files or 200,000 files.
Do you create cache files on-demand or immediately? This can make a 300% speed difference.
What is slow? Reading the metadata? Ingesting the files? Where are the files stored?
Without a log file I cannot help you because I don't know a thing.

but my attempts to capture metadata are s-l-o-w.   I need caption , keywords, headline, city and capture date.

What do you mean by "Capture"? Entering metadata in the Metadata and Keyword Panels?
This is a sub-second procedure. Unless you have configured IMatch to write back metadata immediately, which can slow things down if you work with large files or slow storage.

Bearing in mind I have done this before it seems a hellish complicated endeavour.

What is complicated? Creating the database or entering (capturing?) metadata?

IMatch 5 is slower when reading metadata that IMatch 3 because metadata is much more complicated these days, ExifTool is more throughout, much saver and handles a lot more metadata than IMatch 3 ever could. Since you have not provided a log file (neither attached nor emailed nor uploaded to my FTP server?) I cannot even say if you have configured IMatch in a way that it automatically writes back files during import, which can mean a 2 to 5 times slower processing...

Please send me the log file (less than 10 MB) to my support email (include a link to this thread) or contact me for details about where you can upload the file.

[chips]

I have my reasons for starting all over again - thanks for your concern folks.  New OS and hoping that with knowledge gained I'd make a better job of it this time being two.

I also appreciate Mario that you need a log file and if I were you I'd be as exasperated by me as I am by iM5 at this moment.

This user has only just discovered %temp% - this user is not a computer geek. 

I have known a few press colleagues who gave up entirely on iM3 having found it impenetrable:  I am persevering now because it's a wonderful tool when working.  Utterly selfishly I'd prefer if it did less and was more user friendly.  In all the start guides and how-tos there is not a step-by-step on creating a database of existing images already captioned etc etc (in photo mechanic) and making selected fields searchable finis.  Now fifty readers have just bridled in defence and shouted "of course there is you moron...".  OK tell me where - I am prepared to spend a little time following instructions.  Someday, who knows,  I may want for unforeseen reasons to create categories for rated images but for now I just want to type in " Joe Bloggs" and see all my pics of Joe Bloggs  or a date and see that day's pics.  That's the wood but acres of trees get in the way.  I shall attempt to attach the latest log and hope it makes sense.  I do appreciate Mario's fast and helpful service and advice from the well meaning (if not that from the self righteous - if all you're going to contribute is "I don't understand why you are doing this" ...



[attachment deleted by admin]

[Mario]

This user has only just discovered %temp% - this user is not a computer geek. 

There is a help topic about log file in the IMatch help, which explains log files and how to use them in case of problems.
The log file can also be accessed directly from the Help > Support menu in IMatch. If IMatch is working, you don't need to locate the TEMP folder yourself.

Utterly selfishly I'd prefer if it did less and was more user friendly.

User friendliness depends on the user and what he/she expects. As for simpler software, there is plenty to chose from, starting from Google's Picasa to the built-in dumbed down features in Windows itself.

In all the start guides and how-tos there is not a step-by-step on creating a database of existing images already captioned etc etc (in photo mechanic) and making selected fields searchable finis.

When you add your files to IMatch, IMatch imports all existing metadata automatically. And IMatch can search all metadata contained in the database. No need for you to do anything, except adding the files.

but for now I just want to type in " Joe Bloggs" and see all my pics of Joe Bloggs  or a date and see that day's pics.

The easiest way to do this is to type "Joe Bloggs" into the search bar in the File Window. Just make sure that the files you want to search are visible/loaded in the file window, because the search bar searches whatever is in the file window (the scope). This way you can search a folder, a category, all files on a disk or even your entire database in the same way - while keeping control over where you search.

You did not mention where "Joe Bloggs" can be found. If you added this name as a keyword in your other software, IMatch will automatically create a category named "Joe Bloggs" under the @Keywords parent category. This category automatically contains all images with the keyword "Joe Bloggs". Just click the category instead of searching.

Often PJ's use a standardized format for captions/headlines. Creating a data-driven category on the headline metadata tag in this case makes it very easy to find images by a specific person, of a specific event or whatever information you record in the headline. If you can show us an example of the metadata you have in your files, we can give you more tips.

Searching for images taken at a specific day can be done via a filter in the Filter Panel. Much easier is the Time Line View, where you can just pick the data and see all photos taken that day. Or that week/month/year...



It's is impossible to come up with "step-by-step" guides for each user type. Every user has different demands, expectations, understanding of metadata, workflows etc. I did my best trying to give as much guidance during the initial steps and to explain each IMatch feature in detail and associating the most likely keywords with each help topic. Nothing is ever perfect and if you have suggestions about a help topic, use the link available at the bottom of each help topic to send feedback. I update the help all the time based on user feedback.

We have a FAQ, Tips & Tricks and Workflow where users describe their workflow and how they achieve different things. Maybe you can start a "For Photo Mechanic users who recently migrated to IMatch" topic, and other users can join in.

It always helps if a user tells "I want to do this, how?" because this allows us to show him which features to use or check out.

[chips]

Mario thank you for the response.  I'll take those points on board.   Now I am puzzled because when I added the first year folder (2005) the pictures were there and the captions appeared but searching did not return results no matter that I could see the data displayed.  We have been here before and you sorted the problem out but so long ago it doesn't appear on a forum thread search and emails will be on a previous computer.  So what I intend to do is wipe the slate clean yet again and set that year up anew. Can't do it straight away as the computer is rescuing lost data from the dodgy hard drive and that will take until tomorrow.  A reinstall is probably in order to undo some desperate changes to defaults.

An apposite analogy occurred since earlier - I have a washing machine for laundry with probably 45 programme combinations of temperature, time, rinse, spin options.  Perhaps once a year I use an option other than "30degree wash".  Can I buy a machine with that one option ?  I cannot.  A friend has just purchased one that plays music when the wash has finished.  Can you imagine the design meeting where that was discussed ?  and the sub-committee formed to choose the music ? 

Deep breath.  Tomorrow is a New Day. Start again.

[Mario]

Remember: The search bar and the filters work with the so-called active scope. And this is what you see in the file window.

If you display the folder "A" in a file window, you can only filter / search the files in that folder.
Make sure you bring the files you want to search in or filter into the file window. This can be a single folder, a folder with all child folders (enable the hierarchy mode in the file window for that), a category etc. If you want to search all files in the database, click on the database node.

No need to start fresh or wipe anything clean. But you are not giving me sufficient info to work with.

1. Where do you search?
2. What do you see in the file window? (Attach screen shot)
3. What do you search for?

I cannot figure this out without you giving me some more details to work with.

[chips]

Cheers Mario . I think my instinct to start afresh is paying off.  Whatever was amiss with the computer and whatever subsequent panicky ticking of boxes I tried had left confusions galore.  Delete and start again is working and more like the speed I remembered from initial installation so three years worth of files now complete and I'm optimistic that if I don't mess the other years will be processed tomorrow.  Looking forward to a complete computer by the weekend.  Yes I touched wood just then.  Night all and thanks again

[sinus]

 

Great, good luck!