scanning removeable media failures in thumbs and names

[Steve01]

Hi,
testing latest iMatch dl'd yesterday 28apr2018
I have many usb_HDD drives from Western Digital with images,  videos and non raster files on them.
Inserted first into PC, it appears and is readable in win explorer.
Go Commands>Add or Update folders and select the drive letter F.
the scan starts , impressive speed  , and after a few minutes with all the files whizzing through it ceases, but nothing appears in the folder view at left, no F drive, nothing.
try a second one, ditto, try a third ..ditto.

What is happening ?  I need to catalog removeable media such as usb_HDD, CD, DVD etc. and see thumbs of what is on them if I click their name in folder panel, whilst they sit on the shelf.

it did a CD, then another, though I couldnt give the result the CD label it is stored and shelved by  CD001 CD002 etc, so unable to locate it afterwards when it asked for AAD687B  !!!

I would need to give the result scan of the failed usb_HDD, if it had succeeded !.. the user name usb_HDD_001  and usb_HDD_002 etc as per their sticky labels.

Steve01

[Mario]

it did a CD, then another, though I couldnt give the result the CD label it is stored and shelved by  CD001 CD002 etc, so unable to locate it afterwards when it asked for AAD687B  !!!

IMatch groups removable media under the drive from which they were added by their unique media serial number.
You can use the friendly name feature (See: https://www.photools.com/help/imatch/#mf_basics.htm) to specify a user-defined name for your media.
Frankly, removable media like CD or DVD are nearly dead these days. Most users have already moved their assets back to regular hard disks or USB sticks. Much faster to work with, easier to backup, no bitrot problems etc. A 128 GB USB stick costs 30US$ and holds the data of 28DVDs or 256 CDs.

Scanning images from external USB disks is usually not a problem. Many users (including me) work with assets on external disks.

Please attach the IMatch log file from that session. See log file for more info.
If you did not keep the log file, repeat your test and attach the log file from that session.

Note: IMatch will not add folders marked as hidden or system by default. Check your Indexing options under Edit > Preferences > Indexing  an the Attributes of the folders in Windows Explorer.

[Steve01]

Hi Mario,
You can use the friendly name feature, sounds promising, as its vital to be able to use the labels we give removable media and store them by,.I can see but one mention of this in the help pages, and that was by going ctrl F and typing Friendly in there as nothing was found using the search window top right, but it doesnt explain how one can get the user created labels to show in the folder tree at left.
I am used to having
CD001
CD002
CD003
DVD01
DVD002
DVD003
usb_HDD_001
usb_HDD_003

etc there and clicking on one reveals the thumbs in central window.

I have boxes of archive CD and DVDs built up over 30 years, and need to locate all my files. I use Archival gold DVD tested to 100yrs, having lost contents of a usb stick, through windows corruption, also a western Digital usb HDD, also an internal HDD, it happens ! Sticks are the least reliable I was told.  The bit that is responsible for us seeing all our files and folder names can get corrupted, then we see just hyroglyphics. I will when I get time rewrite the CDs to DVDs but there is never time for such in life. CD and DVD dont suffer that fate. I have however most of my files on usb HDD and 4Tb of internal drives, almost now full, as its impractical to burn that many DVDs and anyway I am forever working on them, accessing them all in one go, but I do need to find those on optical drives. If I were to offload them all onto usb HDDs I would have to find yet more money of which I dont have much.

I have also had iMatch crash on my internal G drive, as well as two out of two usbHDDs, so out of 3 drives scanned so far all 3 have crashed.

I rebooted iMatch, set it to the debug logging, then clicked the database node and right clicked on that and chose 'add or update folders' browsed to F drive and scanned usb_HDD_004 circa 21:46 and after a short while scan ended and nothing appeared.

closed iMatch, relaunched it, it said about hires debugging enabled, I this time set it to scan G drive, again same thing happened. 21.56

I can see G in the folder pane but clicking on it shows no thumbs of its contents. This was circa 21:55 so again copied that log  using the help support option.  G is with 11Gb used of 39Gb all raster and video and a Steam sim backup folder. usb_HDD_004 is 451Gb used of 465Gb of mostly DVD_RTAV videos and photos.

I attach both, none feature any folders or files I notice. called the one zip suffix 21-46 crash.

Steve01

[Mario]

When IMatch crashes, it produces a detailed DUMP file and a log file. You see a corresponding message with instructions in a dialog.
Did you see this dialog? See https://www.photools.com/help/imatch/#im_dumpfile.htm for details.
If IMatch crashed without showing this dialog, the problem was caused by some external component, driver, graphic card, file system driver, video codec, WIC codec or something. Almost impossible to track down.

Using "friendly names" is an old but still working feature. As I said, most users have abandoned CD/DVD/BR-D storage long ago. Modern hard disks, USB etc. are so much more convenient, fast and secure. To work with friendly names in the Media & Folders View enable the "Show friendly names" in the options (Gear icon in the toolbar). To specify a friendly name for folders, enter something in the Friendly Name property of that folder in the properly panel below the tree.

See https://www.photools.com/help/imatch/#mf_basics.htm and scroll down to Configuring the Media & Folders View

To make it easier to understand your explanations, please consider including screen shots of what you are seeing. You can attach up to five images with every post.

The two log files you haver attached show only a normal IMatch startup. Not much more. The only warning is about a missing spell dictionary. You have not installed on yet for your language.
The last processed file was G:\test file cataloging software video files\VR_MOVIE_FantasticGoal F.mpg. But the log looks normal, IMatch completed the operation and the folder scan.

If there was a crash, something outside IMatch crashed and Windows shut-down IMatch in response to that.
Since IMatch was unable to produce a DUMP file, its almost impossible to analyze this further. May a video codec problem, file system problem, ...

[Steve01]

Hi,
I called it a crash as it was scanning merrily away on F drive, the usb_HDD_004 then that scan progress windows vanished, and there was nothing in the folder tree at left, the usb_HDD was not shown there as F drive, there was no F drive, and when I scanned G drive, which has a lot of images at root level, again it set off scanning merrily, then that scan vanished as if it had finished and clicking on G drive doesnt show anything other than one mpg video.
as such I presumed it had crashed but there was no warning panel that appeared, it seemed to have finished but displayed NOTHING for its efforts.

see attached.

If I ever get a removable media usb HDD to scan then entering a friendly name,  will I see that name appear for the drive itself instead of 8HGFT56F  in the folder tree, is that how it would be ? I am after needing naming the drive, not the folders within it.
I guess I could try it on the two CDs as they worked. ok,...Tried that, but unable to name the CD as CD001, see attached of what I got.

Regards

Steve01

[Mario]

You have two filters active. Why? This may explain why you don't see files in the File Window.

You cannot change the media label or media serial number IMatch displays. The friendly folder name feature can only be used to specify a friendly name for folders. Not for disks or medias.
Maybe give the root folder of your DVD/CD a friendly name.

[Steve01]

Hi Mario,
I am unable as yet to get iMatch to scan any internal drives or usb HDDs.

I set ACDSee onto the usb_HDD_004 last night and it did the lot.. trouble is that prog doesnt allow removal and a second one inserting, so all that is lost.

You say the log file is of no use unless a crash panel appears ?

I am at a loss to understand and solve this.

What can I try to see if it is the cause ? Not sure though why other progs are not affected.

Secondly you say users have to place all their files and folders into a  new folder on their usbStick or usbHDD and friendly name that folder then that will appear in the folder tree at left. Obviously CDs and DVDs that are non rewriteable or finalised cant have this done, but I have wallets of gold archival DVDs.

WhereIsIt prompted for a user name for the disc or drive about to be scanned and displayed that afterwards, is that feasible in the future as the folder naming can't happen and may also trip up on the max character address string length for windows of 255 ?

but unless I can get iMatch to even scan and complete anything this is academic anyway.

G drive is internal and as you can see in my images there is nothing showing, even after I untick the two filters top right, I still only see one mpg file.

Steve01

[Mario]

Friendly names are stored in the database. They don't require changing anything on the media.
As I said, you can give each folder a friendly name. But you cannot change the drive label or serial number in IMatch. IMatch always organizes media per drive/media label. You cannot change that or replace the information with your own text if this is what you want.

Here I have switched the Media & Folders to show friendly names, and then entered a friendly name for the "images" folder on an external disk:



The real name of the folder is "images" (that's what you see in Windows Explorer) but in IMatch now it also tells me the color of the DVD.

I have no idea why IMatch may refuse to index files on external hard disks or DVDs only on your system. Without logging anything in the log file and even crashing. Most IMatch users work with files on external disks, NAS storage or files on servers. No problem reports.

What do you do to index a folder from your external USB disk? Do you drag the folder from Window Explorer into the Media & Folders View? Or do  you click the "Add Folder" button in the Media & Folders View?

[Steve01]

Hi Mario,
some progress, slight,...firstly though..
I do need to get my names allocated to the usb HDDs, CDs and DVDs in the folder tree as I store them by those names, not aware of anyone storing them by GFR574DR5F for example. I was always able to do that in WhereIsIt,. see attached..

I have a folder empty that I use for telling me what usb is being viewed 0_this is usb_HDD_004 so I friendlyNamed it in properties usb_HDD_004 but it doesnt occur at the top of the folders relating to that drive when drive removed but appears lower down, I then scanned usb_HDD_008 and friendly named that identifier folder, as per attached, but see they dont relate to their contents.

having user names for the drive itself as per WhereIsIt is what is needed, obviously its do-able as he managed it.

drive scanning failure...
G drive is an internal 7200rpm western Digital HDD
usb_HDD_004  is a western Digital 'elements' and usb_HDD_008 WD MyPassport, both are external usb hard drives plugged into a powered usb3 hub.

I go Commands>add or update folders and browse to WD Elements(F:) drive, expand it to view the contents, to double check, select F drive and click ok then start.

see attached.

it is usb_HD_004  and I wish for this name in the folder tree afterwards.

465Gb 451G  used
it has 75 DVD_RTAV folders
39702 jpg  11.4Gb
370 .psd  18.1Gb
986 .tif 7.28Gb
358  .png   196Mb
666   .3dm   6.77Gb
85   .dxf  367Mb
18   .dwg   1.67Mb
167   .pdf  313Mb
21   .fh9 2.66Mb
1827   .bmp   2.29Gb
amongst other files

it scanned for 2 mins or so then vanished.

I have now tried just selecting one folder at root level on it and it scanned it and I see F in folder tree. 
set it upon a folder with three sound files in it and it scanned that and added it to F   
set it to scan another folder with .fh9 and other files in, after 1 minute or so I have grey panel appear. 
Auto paused, the auto paused threshold was reached.  I see what it refers to is set at 1000 files, not sure why it paused then !
as only 230 files 1.62Gb was in that folder  .tif .bmp .psd .fh9  .txt  .pdf    .dxf     .dat (Breezebrowser for any images viewed in that prog)  .igs  .3dm  .eps  .zip 
I alter that to never pause and click the pause bars., and it completes.

iMatch just cant handle a full blown set of files it seems.

G drive was 11Gb, thats not a lot. 26.8Gb   (was 18Gb until I added three video files) and its 14953 files

is such too much for iMatch ?

should we do drives in small portions at a time ?

I try for usb_HDD_008 using the same method, with that pause turned off.
It ran the longest its ever gone.
here is a log start  21:40:36
21:45:36  121,200 files 4900 folders
21:50:36  241,900 files  9000 folders
21:56:36  308,000 files  12,700 folders
22:01:36   366,200 files  16,500 folders
22:02:10 scan vanished, nothing to see in folder tree 

attached the logs at the time.

I might manage scans by selecting one folder at a time but it will take me forever, far rather select a drive, there is no way of selecting multiple folders it seems.

frustrated 

Steve01

[Mario]

not aware of anyone storing them by GFR574DR5F for example.

That's why you get the chance to give your DVD a proper label when you burn it. This label cannot be changed later and IMatch has no feature to allow you to display a "fake label" for your media. If you need this, you need to find some other solution or software.

IMatch sorts folders alphabetically. Either by their physical name or friendly name. If you want your friendly name to be shown on top, use a _ as the first letter or an A. Press <F5> after changing the friendly name to reload the folder tree to see your changes.

I go Commands>add or update folders and browse to WD Elements(F:) drive, expand it to view the contents, to double check, select F drive and click ok then start.
see attached.

This looks as if IMatch has not yet linked in the drive node. Click into the tree and press <F5> to force a refresh. Then you will see the drive nodes as in my screen shot.

iMatch just cant handle a full blown set of files it seems.

A typical user manages between 50,000 and 300,000 files in IMatch. I'd call this a full-blown file set. In the old days when DVD storage was still common I managed over 100 DVDs in my personal database. Of course i have long since given up on using DVDs. it's just not worth all the trouble.

Please note that scanning DVD drives may be extra slow, specially when IMatch hits PDF files which need to be processed by Windows / Acrobat or whatever PDF reader you have installed. IMatch first scans the files and brings them into the database. Then it starts processing them, extracting metadata etc. This can be a lengthy process.

it scanned for 2 mins or so then vanished.

What vanished? IMatch? Or the progress dialog you see while IMatch is scanning the file system? This is the dialog in the upper screen shot in your3rd attachment.

Your log file seems to be from another session. None of the folders in your screen shots is processed in that log file. Only one folder is added to the database, but is found current. No new or updated files are found.

Note that IMatch only indexes known files (files supported by IMatch), not all files that may be in your file system. IMatch supports over 120 file formats. If you want to add other files, you can configure them as user-defined formats under Edit > Preferences > File Formats.

Your screen shot shows IMatch scanning a folder with the dictionaries used by Adobe Photoshop CS 6. These files are nothing you typically manage in a DAM.

[sinus]

If I had such a lot of CDs, DVDs and so on, I would be buy a proper, large Harddisk and copy all CDs and so on on this Harddisk.

Maybe for each CD one folder (with a proper name).

After this I would put all CDs aside and work only with the new Harddisk, means scan then the Harddisk into IMatch.

[Jingo]

Agree 100%... used to use CD's/DVD's back "in the day" for backup.. they were cheap and quick... have a stack of 200 of them that I plan on dumping when I get around to it now that these same media is backed up on a single USB drive - multiple times.  Technology comes... technology improves.... important to stay up-to-date with it for optimal performance and management!

Mario says:

A 2.5" USB 3.0 hard disk holds 450 DVDs and fits into my shirt pocket. It's super fast, too.
Buy 2 to have a 100% backup on two physically separate devices.

The DVD media may be labelled with "100 years lifetime" but I don't think we'll be able to buy a DVD drive 10 years from now...

SORRY. It seems I have accidentally modified Jingo's message (see above in the quote) instead of replying. -- Mario

[Steve01]

Hi,
some success with naming.
I firstly added the suggested _ as prefix to my naming folder and it placed it right next to the meaningles drive name so now I have an idea where the folders come from.
see attached phase 1.

then a call to WD they tell me to r/c on the drive in win explorer and alter name, so alter it to usb_HDD_004 Elements and ceating a new catalog I scan the drives again and now I have my user sticky label storage name as the parent and can clearly see the children relating to it, in alphabetical order as well

Hope that is of use to anyone wishing for a sense of order and storage.

see phase2 pic attached.

one hurdle sort of over, can't do this to optical discs though. Mine are all nicely boxed in numerical written on the spines, order so:-
7GD58GD
86GDHJU54
975FHUYR5
KH79KJ8

instead of

DVD001
DVD002
DVD003
DVD004

is still a big problem, unless I copy the entire lot onto a HDD.

I have already lost an entire internal and an external HDD and a stick to corruption, so am reluctant to do so when at least years of CD and DVD is safe. There again one could argue I have these archived as optical so no great loss !

That's why you get the chance to give your DVD a proper label when you burn it.

If I receive CD and DVD the sender hasn't considered my filing system though. They never bother to use a name just place it in trough and burn to it. They dont know my naming convention for other users opticals anyway.

Again copy paste the contents onto a 2Tb HDD I hear you say.

The big problem though is iMatch simply wont scan a drive, gets so far and the scan dialog vanishes, hit F5 but nothing in the folder tree.

I just tried for my internal G drive (test drive) again, thats just 28Gb internal. again it scanned for 10 secs then dialog box vanished.
attached the log for this.

if I select a folder it scans, the only way I could manage that usb HDD 004 and 008 naming test, but there is no option to select multiple folders. one at a time will take forever.

I am win7 64bit pro.

What I meant by " iMatch just cant handle a full blown set of files it seems. "
Was per scan session, as I have managed to scan by selecting a folder,. but feed it 1.8Tb of files and it ceases, though 2nd attempt it ran for 20 mins and had got as far as 166,000 files then scan progress dialog box vanished., F5 but nothing in folder tree for F. I get the feeling that selecting a few folders at a time it would manage the entire drive, unlike feed it the entire drive., thats what I meant. I have always been able to elect a drive letter and scan on progs used in the past. I wish multiple folder selection was possible.

I also wish excluding folders was possible as there are some I dont want scanned so I could select drive then exclude the unwanted ones.
e.g  Acronis backup 58Gb file how does iMatch tackle that one ?

So desperately want to overcome this as this is the prog for me, it oozes class and support is great, thank you by the way  ,  if only it would scan.

amended wrong attached log file for todays vanishing scan box 10 mins after posting !

later on plugged in usb Buffalo mains powered drive, went commands >add or update folders chose M drive scan vanished immediately, log also attached now. 
Steve01

[Jingo]

Agree 100%... used to use CD's/DVD's back "in the day" for backup.. they were cheap and quick... have a stack of 200 of them that I plan on dumping when I get around to it now that these same media is backed up on a single USB drive - multiple times.  Technology comes... technology improves.... important to stay up-to-date with it for optimal performance and management!

Mario says:

A 2.5" USB 3.0 hard disk holds 450 DVDs and fits into my shirt pocket. It's super fast, too.
Buy 2 to have a 100% backup on two physically separate devices.

The DVD media may be labelled with "100 years lifetime" but I don't think we'll be able to buy a DVD drive 10 years from now...

SORRY. It seems I have accidentally modified Jingo's message (see above in the quote) instead of replying. -- Mario

[Carlo Didier]

I also wish excluding folders was possible as there are some I dont want scanned so I could select drive then exclude the unwanted ones.
e.g  Acronis backup 58Gb file how does iMatch tackle that one ?

If you have Acronis backup files mixed with image files, you might reconsider the folder structure of your drive.
ALL my images that I want to manage are under a single main folder (with subfolders of course), so I only have to tell iMatch to index that folder and all subfolders.

As to the Acronis file, iMatch probably ignores it as I don't think it's on its list of file types to index.

[Steve01]

I havent got the acronis backup file in a folder for images, its in a folder called acronis backup. I have 100's of folders, from CAD work to internet acquired images.  A drive solely for programmes acquired. One for video editing and one for capture. When the one drive was full thanks to raw files, I have overflowed onto a second drive.  ext usb also feature overflow and video storage. I have the largest Tb drives there are, but to put all the existing files onto those is all eggs in one basket and would fill them up again so leaving me PC less.

Trouble is with iMatch one cannot select a drive then say exclude folder acronis backups.

its either 1 folder or 1 drive. those are the selection options.

I dont have one folder called all my images. I save camera photos to individual folders on D drive, each folder is named with date and subject matter. I have a need to catalog all the files and folders on my user drives, I dont save anything to C drive. I have video editing drive full of folders, video capture drives etc etc. thousands and thousands .  folders for subject matter. I am an historian with tons of subject folders, its the only way of looking at all photos acquired of a subject. Its why I need a decent DAM software, rather than put a photo which has 5 subject matters in it into 5 separate folders, named for each of those subjects. I need to be able to just say scan that drive, as picking my way through it a folder at a time, not enough time in life for that. If I am going to recommend a prog to fellow historians etc, they wont want to do that either.   Had always aimed a prog at a drive and it scanned it.

THE BIG PROBLEM IS THIS PROG REFUSES TO SCAN, IT VANISHES AFTER A FEW SECS.

last night I ran another DAM prog on these drives and a load more, it did every one. but it has no means of flagging up folders with changes, which is no good.

WHY DOES THIS PROG REFUSE TO SCAN ANYTHING IF FED A DRIVE ?

Desperate to fix this now.

Steve01

[Mario]

Trouble is with iMatch one cannot select a drive then say exclude folder acronis backups.

1. Usually you don't add entire hard disks to your image DAM.
Most users have one root folder (e.g., "images") per drive and then manage only files in and below that folder.
Bringing an entire hard disk into your DAM can cause all kinds of unwanted effects. Windows may be constantly updating files in arbitrary folders and this then triggers rescans and background activity in your DAM.

2. You can exclude folders under Edit > Preferences > Indexing: https://www.photools.com/help/imatch/#rmh_config_indexing.htm  (Skip these folders) option

THE BIG PROBLEM IS THIS PROG REFUSES TO SCAN, IT VANISHES AFTER A FEW SECS.

None of your log files indicates any of this. Your screen shots shows that IMatch infact has added multiple drives and media to your database.
The first folder indexing dialog will be quick. Then the folder and files are indexed. And the dialog will go away. IMatch then starts processing the files in the background.

Please double check that you have not accidentally disabled the loader progress overlay dialog. Go to Edit > Preferences > Application and search for load prog This option must be enabled. IMatch then displays a progress dialog when processing more than 100 files. Else IMatch processes all files in the background and you can monitor the progress in the IMatch status bar and the Info  & Activity Panel.

[Steve01]

Hi,

1. Usually you don't add entire hard disks to your image DAM.
Most users have one root folder (e.g., "images") per drive and then manage only files in and below that folder.
Bringing an entire hard disk into your DAM can cause all kinds of unwanted effects.

I have need of locating not just photographs taken on a camera, but all sorts of images, artwork, scans, plans, manuals, CAD work, and I am trying to find a prog that will enable me to locate them all, as well as keyword and thumbnail the raster images that I need to locate easily and not keep storing in several folders as I do dependent on the subject matter in the photo. It should be one photo and several keywords not one photo into several folders ! My fellow compatriots do the latter as they havent used cataloging progs. I did, then extensis portfolio ceased and here I am as well as all other ex portfolio users starting over. I used WhereIsIt to catalog al my images and other user created files, then that prog folded. iMatch is able to latch onto all file types so I can search on file name and find non raster, or raster not worthy of keywords.

If I was to have dumped all these various images into one folder I would never have managed to do a days work as rummaging about in thousands of images of a thousand different subject types and requirements in a folder is for the PC dimbo.   If I need to locate a photo to place into my CAD prog of subject G I can go into my hierarchy of folders to where that is. Photos taken with camera are a different area totally to scans of aircraft plans or archive images. Some of them may end up on Alamy, some get used in photogrammetry, and so on. I am into a workflow that most 'photographers' are not even aware of. We both have photos we have taken, pleasure, Alamy, agency, etc and then I open a door beyond that field of interest on a vast amount more of all sorts of purposes.

Windows may be constantly updating files in arbitrary folders and this then triggers rescans and background activity in your DAM.

I am only cataloging user file storage drives, nothing is ever stored on C drive. Windows is always altering files on C drive, I should be ok surely ?

Windows shouldnt be altering my user files or is it that windows is writing files to D E F G H etc drives outside of the user folders there ?

I cant select more than one folder in the scan option so to scan a drive selecting one folder at a time will take forever, my only way of having all my folders scanned is to select the drive. cataloging progs WhereIsIt and ADC had no problems being fed a drive. Extensis Portfolio didnt support non raster so it was only ever fed raster folders and I used it for just camera folders, but I have images far more than from my camera I need to catalog, and they live in subject folders in a vast hierarchy.

iMatch refuses to scan......None of your log files indicates any of this.

I wish the log would answer why a drive scan fails. The dialog vanishes in a second or so, longest it ran was 20 mins then it was gone. No other DAM prog tested does this.

Your screen shots shows that IMatch infact has added multiple drives and media to your database.

No, they are only there because I aimed iMatch at a folder instead of the drive, and it of course has to show the drive letter. There would be hundreds of folders otherwise on the expanded view.

Loader progress overlay dialog is set to yes.

I am going to try for  reinstall, something is fundamentally not right unless iMatch is by default unable to be fed a drive letter.


Steve01

[jch2103]

This has become a very long thread, so excuse me if I missed something...

For clarity, I think this is a factual recap:
- You apparently have a reasonably modern computer running win7 64bit pro.
- You use the C: drive only for the OS and programs, but not data.
- You have an internal G drive (7200rpm western Digital HDD) and two external USB hard disks (usb_HDD_004 Western Digital 'elements' and usb_HDD_008 WD MyPassport, both plugged into a powered usb3 hub).

Questions:
- How big is the G: drive? How big is it compared to the total volume of files on your external drives (and other media?)? As mentioned by others, it may simplify things to consolidate all files onto a single drive for use in IMatch.
- I assume the USB ports on your computer are USB 3?
- Are you able to successfully copy files using Windows Explorer from a USB drive to your G: drive? (I.e., no issues with file transfers?)
- I assume you've made sure antivirus programs are excluded from your database as Mario has mentioned?

In response to your questions about IMatch capabilities, IMatch is indeed designed to handle large volumes of images and many other kinds of files. My collection of images is around 640 GB, but other users have much larger collections of images and other kinds of files, so IMatch software shouldn't be a limitation unless you have a truly enormous collection.

By the way, I don't think Mario was recommending that all your files be dumped into one folder. Rather, I think he was suggesting something like the following:

G:
\Photos
   \date 1
   \date 2
   etc.
\CAD files
    \Folder 1
    \folder 2
    etc.
Etc.

[Steve01]

Hi John,
I have had a eureka moment  , no one has suggested I drag drop many folders to iMatch, using the command add or update has 1 folder limitation, but I read in manual or maybe the intro video, having played the entire lot today, I can select more than one folder.

AS SUCH I CAN NOW DO AN ENTIRE DRIVE BUT NOT CLICK THE DRIVE, INSTEAD DO ALL THE FOLDERS ON THE DRIVE, AS SO FAR FOLDER SELECTION SEES IMatch fail within seconds, not sure why as the other progs tested dont, but help manual says dont select a drive as well !

For clarity, I think this is a factual recap:
- You apparently have a reasonably modern computer running win7 64bit pro.
- You use the C: drive only for the OS and programs, but not data.

yes

- You have an internal G drive (7200rpm western Digital HDD) and two external USB hard disks (usb_HDD_004 Western Digital 'elements' and usb_HDD_008 WD MyPassport, both plugged into a powered usb3 hub).

yes, for testing, as I was told to try for a test area first, so used empty G drive to add carefully chosen examples of my file types I need to be seen by catalog prog.
In fact I have:-
Internals
D data 1.63Tb 1.63Tb used (full)
E 1.09Tb  1.07Tb used
G scratch (temp test catalog) 39Gb 33.5Gb used
H av_capture  585Gb 582Gb used
I av_edit  585Gb 583Gb used
J video 651Gb 649Gb used (pure pics and vids)
T avedit_backup 585Gb  578Gb used (pure pic and vids)
U videobackup 651Gb  647Gb used (pure pics and vids)

externals:-
Buffalo 500Gb HD01 full
Buffalo 500Gb HD02 full
usb_HDD_001  500Gb full
usb_HDD_002 466Gb  462Gb used
usb_HDD_003  466Gb 463Gb used
usb_HDD_004  466Gb 450Gb used
usb_HDD_005  931Gb 930Gb used
usb_HDD_006  465Gb 463Gb used
usb_HDD_008  1.82Tb 1.76Tb used
usb_HDD_009  1.82Tb 1.82Tb used

Ex Internal HDDs in Caddy powered usb also with video.

Questions:
- How big is the G: drive? How big is it compared to the total volume of files on your external drives (and other media?)? As mentioned by others, it may simplify things to consolidate all files onto a single drive for use in IMatch.

see above !

- I assume the USB ports on your computer are USB 3?

4 usb2 and 2x usb rear all in use with scanners and other items, usb3 feeds to 8 port usb3 powered hub.
front 2 x usb2

- Are you able to successfully copy files using Windows Explorer from a USB drive to your G: drive? (I.e., no issues with file transfers?)

yes.

- I assume you've made sure antivirus programs are excluded from your database as Mario has mentioned?

I now have, excluded the catalog folder from Kaspersky IS
excluded imatch exe from Kaspersky IS.
have uninstalled iMatch, brought windows up to date, reinstalled iMatch, saw the prompt to exclude catalog folder so then did that.
I see imatch latched onto existing settings as it declared debug report activated, so I guess it pulled such from the catalog folder ?
It also displayed my scans so far.

In response to your questions about IMatch capabilities, IMatch is indeed designed to handle large volumes of images and many other kinds of files. My collection of images is around 640 GB, but other users have much larger collections of images and other kinds of files, so IMatch software shouldn't be a limitation unless you have a truly enormous collection.

what do you reckon of the above, safe to say most space mopped up is pics and videos., raw files , I never shoot jpg, though edit into tif or jpg afterwards, have only edited a fraction of the raw files I have. Photogrammetry requires loads of shots and they dont need keywording but still need to know what they are, unless a title image thumb could be used for just one in the folder of 300 ! I have though folder by date so there are often other pics there that do need thumbs. I could subdivide the folder keeping those not needing KW separate and the suffix text avoiding a windows clash of same folder names.

By the way, I don't think Mario was recommending that all your files be dumped into one folder. Rather, I think he was suggesting something like the following:

G:
\Photos
   \date 1
   \date 2
   etc.
\CAD files
    \Folder 1
    \folder 2
    etc.
Etc.

for RAW camera I have D drive and overflow to a couple of others now :-
within folder 20D raw I have:-
20D_16_05_23 Holiday Seychelles (just joking...I wish !)
20D_16_06_22 Steam Engine day

and so on,

but then the other images , scans, CAD, internet, etc, not always from camera are up to 5 or so layers deep due to hierarchy needed., and fill the entire alphabet A to Z at first level !

ditto CAD, 3 or 4 layers deep.
there are more than just camera and CAD, all sorts of subjects,

DVD_RTAV video folders though are all at root level, as are .mpg derived from them.

many drives are pure pic and vid, some just vid. some though are stuff from previous PCs where one gets a new PC and dumps D onto it theh addss to it, it gets full then files get written , to overflow.

As images are linked to CAD and vector moving such cause issues. Most though are not, but one needs spare space to move things around to sort things out. I never seem to have time and space. copying Tbs of data takes forever, always a chance of corruption as well.

I hope to trim some stuff down by seeing thumbs, one reason for iMatch etc. and using it to be the prime mover.
Steve01

[Mario]

no one has suggested I drag drop many folders to iMatch, using the command add or update has 1 folder limitation

That's the first thing that is explained, with pictures, on the "How do I add files and folders to IMatch" help topic:

https://www.photools.com/help/imatch/#rmh_dlg_scanfolder.htm?dl=h-1

("Adding Folders Using Drag and Drop")

I can only recommend that you revisit the IMatch help. The Add/Update folder command also can handle any number of folders of course.

(Using the Add or Update Folders Dialog). The explanation starts with "This dialog allows you to add multiple folders at once, "...

Please don't try to somehow "figure out" how IMatch works. 5 minutes reading will bring you much faster forward.

[jch2103]

- You have an internal G drive (7200rpm western Digital HDD) and two external USB hard disks (usb_HDD_004 Western Digital 'elements' and usb_HDD_008 WD MyPassport, both plugged into a powered usb3 hub).

yes, for testing, as I was told to try for a test area first, so used empty G drive to add carefully chosen examples of my file types I need to be seen by catalog prog.
In fact I have:-
Internals
D data 1.63Tb 1.63Tb used (full)
E 1.09Tb  1.07Tb used
G scratch (temp test catalog) 39Gb 33.5Gb used
H av_capture  585Gb 582Gb used
I av_edit  585Gb 583Gb used
J video 651Gb 649Gb used (pure pics and vids)
T avedit_backup 585Gb  578Gb used (pure pic and vids)
U videobackup 651Gb  647Gb used (pure pics and vids)

externals:-
Buffalo 500Gb HD01 full
Buffalo 500Gb HD02 full
usb_HDD_001  500Gb full
usb_HDD_002 466Gb  462Gb used
usb_HDD_003  466Gb 463Gb used
usb_HDD_004  466Gb 450Gb used
usb_HDD_005  931Gb 930Gb used
usb_HDD_006  465Gb 463Gb used
usb_HDD_008  1.82Tb 1.76Tb used
usb_HDD_009  1.82Tb 1.82Tb used

Ex Internal HDDs in Caddy powered usb also with video.

You certainly have a lot of full/almost full internal/external hard disks...
(Just curious - How full is your C: drive? Also, I assume these internal/external drive don't contain duplicate data?)
I wouldn't be surprised if this is causing your system some performance issues, even if some of the full drives aren't actively being accessed. I've also heard that USB 3 hubs can have performance issues, although I don't have any personal experience with this.

You might want to consider picking up one or more large capacity internal hard disks (e.g., 4-8 TB) to consolidate your files; prices have come down considerably. In addition, many folks are using Network Attached Storage (NAS units). Of course, having a separate backup is critical (multiple copies of all data).

[jch2103]

...and so on,

but then the other images , scans, CAD, internet, etc, not always from camera are up to 5 or so layers deep due to hierarchy needed., and fill the entire alphabet A to Z at first level !

...

Another thought: How many folders/files are there in the root folders of your drives? Too many folders/files at the root level can cause performance issues with Windows and other operating systems. On the other hand, having lots of subfolder levels isn't as critical.

[Steve01]

Hi,

(Using the Add or Update Folders Dialog). The explanation starts with "This dialog allows you to add multiple folders at once, "...

Please don't try to somehow "figure out" how IMatch works. 5 minutes reading will bring you much faster forward.

I sat and watched all the training videos that are on the link on the website today, and full marks Mario, well explained , well spoken, well filmed, perfect.

I cannot get Command>add or update folders dialog to allow me to select more than one folder at root level.

I can drag drop from windows many folders in one go, as windows allows multiple selection, but I select first folder, then hold shift for second and first loses selection, also tried ctrl, see attached swf video which will play in a file browser.


as for C drive, 232Gb and 206 used SSD drive.

internals are C drive SSD
and two WD 7200 rpm each 4Tb and another pair as backup for these.

many folders at root level, simply because there are so many different subjects .

I cant go putting access databases into the same folder as aircraft manuals for example. Music into the same folder as photogrammetry project files, no parent folder can cope with such diversity, I have to be able to go to D drive and go to P for photogrammery or A for access, i can navigate up and down and get to whatever subject is thrown at me damn quick. 32Gb ram PC custom built, mouse set fast,  1inch does entire screen width.

if I alter addresses to images then linked images in CAD fail. also images in access fail.

If it works then I leave alone.

Steve01

[jch2103]

as for C drive, 232Gb and 206 used SSD drive.

internals are C drive SSD
and two WD 7200 rpm each 4Tb and another pair as backup for these.

many folders at root level, simply because there are so many different subjects .

Based on your list of logical internal drives, it sounds like your two WD drives are split into multiple logical drives? And the other two WD drives are backups of the other two (duplicates)? Is this partly because of the old Windows 2 TB partition limitation (which can be dealt with these days)? Having lots of logical drives can considerably complicate disk and file management...

Regarding the SSD, if 206 GB of 232 is used, that means it's ~86% full. I don't know what model it is, but unless the drive has significant hardware reserved space, that could cause performance issues; see  https://www.seagate.com/tech-insights/ssd-over-provisioning-benefits-master-ti/.

These kinds of things can affect any software that requires significant data access, including IMatch.

[Carlo Didier]

Also, filling up disks over ~80% also degrades performance (except for SSDs).
How about Windows file indexing? Do you need it on those drives? If not, switch it off.