Can I filter out just the files without an offline cache?

[andrewlep]

Firstly I would like to congratulate Mario on a superb update to IMatch5. I have been a long term user of V3.6 and that program (I thought) offered everything I needed but V5 takes things to a new level

I have successfully updated my large database of over 300,000 files in several formats and the program is still very responsive to most display layouts, searches, slideshow etc. Very impressive.

The many extra features make this upgrade something I am already recommending to friends.

But enough of the praise!!

I do have one problem that I am guessing can be solved... but my research and experiments have failed so far.

I am making a full res offline cached file for every pic. This is to allow me to let a client see a full res preview when I take the database onto my laptop (which doesn't have the original files).

The process of caching has been mostly OK but I am left with the info window reporting that the cache holds 304,037 files but my complete database consists of 306,613 files. Somehow I must have missed scanning some files

I found that the option in Var Toy to add a confirmation that shows the cached image details when hovering over the i button in file view works fine. If a name is displayed I know a cached file is available.

My problem is that at the moment the only way I could find the files without a cache would be to check every pic individually! Is there a way to construct a filter that, when the whole database is selected, the file window will filter out just the files without a cache.

I could then just select those filtered files and choose the option to create a cache for them.

If this needs some sort of formula could you make the explanation as simple as possible as the techy stuff is not my forte!

BTW that is another comliment to IMatch5 as even I can make it work well

Thanks
Andrew

[Ferdinand]

My initial reaction was to wonder if you had enable cache files for JPG, but the percentage that you're missing is so low I assume that you must have. 

I am often guilty of thinking in IMatch 3.6 terms and resorting to scripting when there are simpler ways in V5, but I can't see a simpler way.  I'd write a script that bookmarked all the files with no cache file, by testing whether or not this variable had a value.  You could then refresh the cache for these files, or you may discover a reason why they hadn't been cached.

[Mario]

Whether a file has a cache image or not is not stored in the metadata, and hence you cannot access this information via a Filter or a data-driven category or a category folder. It can be done via a script. I have a script which does this.

1. Create a new script in the Script Manager Panel.
2. Copy/Paste from below into the new script. Run.
3. The result is a new category which contains all files for which the {File.CacheFileName} variable returns an empty result.

Code Select Expand


'#Language "WWB-COM"
Option Explicit

Sub Main
Debug.Clear

Dim skipJPEG As Boolean
Dim msgres As VbMsgBoxResult
msgres = MsgBox("This script scans all files in your database in order to find files without associated cache files. " + vbNewLine + vbNewLine + "Do you want to ignore files in JPEG format?",vbYesNoCancel)
If msgres = vbCancel Then
Exit Sub
Else
skipJPEG = msgres = vbYes
End If

Dim ctx As New VarContext
Dim fset As Files
Set fset = Database.Files
Dim result As New Files

Dim ph As New IMProgressDialog
ph.SetRange 0,fset.Count

Dim cancel As Boolean
cancel = False
Dim f As File
For Each f In fset
If Not skipJPEG Or f.FileFormat <> "JPEG" Then
ctx.File = f
If ctx.Parse("{File.CacheFileName}") = "" Then
result.AddFile f
End If
End If

ph.Step
If ph.Canceled Then
cancel = True
Exit For
End If
Next

ph.Close

Set ctx = Nothing

If Not cancel Then
Dim c As Category
Set c = Database.Categories.Item("FILES_WITHOUT_CACHE_FILES")
If c Is Nothing Then
Set c = Database.CreateCategories(Nothing,"FILES_WITHOUT_CACHE_FILES")(1)
Else
c.UnassignAll
End If

c.AssignFiles result

MsgBox(result.Count & " files withiout cache image found.",vbOkOnly)

Application.GotoCategory c

End If

Set result = Nothing

End Sub

[andrewlep]

Thank you Ferdinand for the quick reply.

Thank you Mario for your reply. I will try the script when I get back to the office this evening.

I'll reply to this post with my results.

Andrew

[andrewlep]

Mario I have run your script and it worked perfectly - first time!

It only took a minute to check over 300,000 files to see if a cached version had been made. The results that were returned in the newly made category exactly matched the number that were reported as missing in the info window.

I was then able to select all the files in the new "FILES_WITHOUT_CACHE_FILES" category and force a cache update for just these. As the 'missing' files were in several different folders this has saved a massive amount of time.

Once completed I removed the category and re-ran the script. This showed that I now had just twelve files that refused to produce a cached version. This was an excellent result. It only took a couple of clicks and a few minutes. Once again IMatch has displayed its versatilty and power. Thank you.

I now need to examine why the twelve files failed to cache. There was a mixture of JPG and PSD files. The one similarity seems to be that they were all big resolutions (6000+ pixel wide). Could this be the problem?

Maybe it would be worth including this script within the standard program as it will be very useful for general maintenance. I know I will make regular use of it.

Thank you again for your exceptional tech support.

[Mario]

I will include this script as a standard "Utility" script from now on.
If you enjoy IMatch, spread the word! Tell others about how awesome it is.

The log file (see the IMatch help) will provide additional details about why IMatch cannot render a cache image for these files. They may be corrupted, the WIC codec may fails or many other reasons.

Switch to Debug Logging under Help Support > Toggle Debug Mode
Restart IMatch to get a small log file.
Try to re-create the cache images.
Attach the zipped log file to your reply.

[andrewlep]

I have had success with eight of the troublesome files by re-loading them into Photoshop and re-saving them. This allowed them to create a cached version. I guess that they were 'corrupt' in some way.

This left four PSD layered files which were large - 2 sized at 18,000 pixels longest edge, 2 sized 21,600 pixels longest edge.

I tried to re-load/re-save these however they still didn't cache.

I experimented by flattening one file and saving as a JPG. This has allowed it to create a cache.

This has led to an odd behaviour in the viewer panel. This large JPG version loads correctly into the viewer panel when selected and pressing enter.

It displays 'sized/fitted' into the panel. I can click it to zoom to 100% and scroll around but if I release the mouse and try to zoom to a different spot the panel just shows the 'cube' logo. If I then select another normal sized pic for the viewer that one displays OK but then if I return to the original large file it still only displays the cube.

The same large file displays perfectly and correctly and scaled in the slideshow panel.

With the 'fixes' that you have already made I consider this remaining odd behaviour to be of very low importance and priority. I only comment for your information.

I have added a debug log for you as requested. Thanks again.

Andrew

[attachment deleted by admin]

[Mario]

Perhaps the file size is too large for IMatch to handle. IMatch may be unable to allocate sufficient consecutive memory within the 3 GB address space is has.
If you have a problem in the Viewer, please add a bug report. I think there is already a similar bug report open, so please do a search before posting a duplicate.

[andrewlep]

Update regarding the creation of offline cache files for very large, multi layered PSD files.

Image dimensions were as large as 21,000 pixels on the longest edge. File sizes around 665mb.

I have now successfully created a cache file for the few remaining stubborn ones.

This was achieved by restarting IMatch5 after every new cached file was made.

By doing this each time I succeeded with another one update if I selected the pic in the file window immediately after the re-start. I have set-up automatic cache updates in the preferences.

Maybe Imatch5 was unable to allocate sufficient consecutive memory within the 3 GB address space as suggested by Mario?

Whatever the reason the re-starts have allowed me to completed the offline caching. Try this if you have similar problems.