- Welcome to photools.com Community.
Recent posts
#61
General Discussion and Questions / Re: Folders requiring rescan
Last post by rolandgifford - September 02, 2024, 10:44:37 PMAll 13575 images Updated Today are raw files of one type or another which are versions of master JPG files (under my relationship rules)
#62
General Discussion and Questions / Re: Folders requiring rescan
Last post by rolandgifford - September 02, 2024, 09:14:35 PMI moved my working/editing folder out of sight of IMatch, added two new categories for images added automatically and modified automatically (both chosen on Edit/Preferences/Indexing) then turned on background indexing. It took IMatch a little under 2 hours to catch up. I have done a recent full manual rescan. All of the flagged folders haven't been changed since that rescan.
6 images were set as having been updated automatically. The Keywords for all look as I would expect. None of these images were recent
No images in the added category
Separate to these new categories 13575 images are in the Updated Today category, it was zero before turning on background processing. The images I have random checked have the Keywords I would expect
I have uploaded the non-debug log file which I suspect is no help
I'm confused/concerned
6 images were set as having been updated automatically. The Keywords for all look as I would expect. None of these images were recent
No images in the added category
Separate to these new categories 13575 images are in the Updated Today category, it was zero before turning on background processing. The images I have random checked have the Keywords I would expect
I have uploaded the non-debug log file which I suspect is no help
I'm confused/concerned
#63
General Discussion and Questions / Re: IMatch failed to import GP...
Last post by Mario - September 02, 2024, 06:43:30 PMUnder which condition has IMatch removed the GPS data?
I have no idea how this can happen. Any hint appreciated. I cannot diagnose things I cannot reproduce.
What's the workflow? Versioning? Propagation? MD Templates? AutoFill? MapPanel? Locations?...
IMatch modifies the timestamp only when setting GPS coordinates for a file, which is a decisive operation caused by e.g. the Map Panel and Locations.
I have no idea how this can happen. Any hint appreciated. I cannot diagnose things I cannot reproduce.
What's the workflow? Versioning? Propagation? MD Templates? AutoFill? MapPanel? Locations?...
IMatch modifies the timestamp only when setting GPS coordinates for a file, which is a decisive operation caused by e.g. the Map Panel and Locations.
#64
General Discussion and Questions / Re: IMatch failed to import GP...
Last post by Tveloso - September 02, 2024, 06:29:28 PMMario, this may be related to Bug Fix #02557, which you have made for Release .15.2 (although this is really the "opposite problem" from that - where the file has the GPS Flag, but no lat/lon), so perhaps that fix will address this "opposite issue" as well?
But I just wanted to mention a variation on this issue that I noticed today. I actually haven't had the:
...issue in a while. But today, I found a file that had neither GPS coords, nor a GPS Flag Icon, but did have a GPS TimeStamp showing in the MD Panel:
The Pen Icon's ToolTip indicates that GPS Data is among what's pending to write back:
I'm quite sure that I didn't change those values though. All I've done with this file so far is Face Recognition.
As with the other posts here, the ECP shows that the file does have a complete set of GPS data, so it does seem that IMatch has somehow removed the coordinates (but left the TimeStamp). Unlike in those other instances though, the Diagnostics didn't complain about this file (since no lat/lon, and an absent GPS flag, represent the in-synch condition).
As before, a rescan (with the reload metadata option) fixed the issue.
But I just wanted to mention a variation on this issue that I noticed today. I actually haven't had the:
Code Select
Warning: File [<OIDvalue>] with GPS flag but no GPS lat/lon. Fixed....issue in a while. But today, I found a file that had neither GPS coords, nor a GPS Flag Icon, but did have a GPS TimeStamp showing in the MD Panel:
The Pen Icon's ToolTip indicates that GPS Data is among what's pending to write back:
I'm quite sure that I didn't change those values though. All I've done with this file so far is Face Recognition.
As with the other posts here, the ECP shows that the file does have a complete set of GPS data, so it does seem that IMatch has somehow removed the coordinates (but left the TimeStamp). Unlike in those other instances though, the Diagnostics didn't complain about this file (since no lat/lon, and an absent GPS flag, represent the in-synch condition).
As before, a rescan (with the reload metadata option) fixed the issue.
#65
General Discussion and Questions / Re: Folders requiring rescan
Last post by rolandgifford - September 02, 2024, 05:42:48 PMQuote from: Mario on September 02, 2024, 05:33:01 PMQuoteThat one is turned off as I don't want to import new files automatically. If there were the option to update automatically but not add new ones, I'd turn that on.
Well that's unusual and explains why all this even becomes an issue for you.
It is usually not wise to deliberately hide new files from IMatch that way. Unless you are very disciplined and know that there might be files IMatch does not know about and is also forbidden to scan the pending folders automatically. Your choice of course.
I am very disciplined about files I create. That is why IMatch saying "something has changed in this folder" is a concern.
Having a file added (or changed) automatically isn't good from my point of view. I can perhaps have files added/changed in the background assigned to specific categories so that I can review them and move my Affinity/editing folder outside the scope of IMatch rather than a sub-folder. I'll think about that.
I have checked my mirroring software and that isn't what is changing folder modified dates.
#66
General Discussion and Questions / Re: Folders requiring rescan
Last post by Mario - September 02, 2024, 05:33:01 PMQuoteThat one is turned off as I don't want to import new files automatically. If there were the option to update automatically but not add new ones, I'd turn that on.
Well that's unusual and explains why all this even becomes an issue for you.
It is usually not wise to deliberately hide new files from IMatch that way. Unless you are very disciplined and know that there might be files IMatch does not know about and is also forbidden to scan the pending folders automatically. Your choice of course.
#67
General Discussion and Questions / Re: Folders requiring rescan
Last post by rolandgifford - September 02, 2024, 05:26:53 PMQuote from: Mario on September 02, 2024, 05:05:29 PMSorry. I meant Edit > Preferences >Background Processing (should have "import new and updated files automatically" enabled).
That one is turned off as I don't want to import new files automatically. If there were the option to update automatically but not add new ones, I'd turn that on.
The reason it is off is because of my standard workflow:
I start with a folder containing all images from a birding trip.
I import these to IMatch and cull, label, GPS tag, and so on.
When I have a set I want to keep I copy all of them to an "Affinity" sub-folder for editing outside IMatch. I don't import this folder to IMatch at this stage.
I then work through this sub-folder and edit images. I move the ones I have done to a sub-folder off Affinity.
When I have finished editing I move all of them back to the Affinity folder and at that stage manually import to IMatch, propagate Keywords etc based on relations and job done.
QuoteThere seem to be some expensive (slow) filters enabled. The Filter Panel reports delays of over 20 seconds when accessing the database, while collections are updated (> 20s to recalc for Reject because the database was super busy doing lots of stuff). The File Properties filter took 20 seconds to apply. Either large scope or system not coping too well.
...
Maybe disable or close filters you don't need so IMatch does not have to keep them up-to-date.
Hide Rejected takes a long time on my system, you have investigated previously and not found a reason. I'm happy to accept the "cost" of that delay as it helps my workflow a lot.
#68
General Discussion and Questions / Re: Folders requiring rescan
Last post by Mario - September 02, 2024, 05:05:29 PMSorry. I meant Edit > Preferences >Background Processing (should have "import new and updated files automatically" enabled).
The AddOrUpdateFolder operation in the log file clocked at 2844ms.
Of course the UI is not immediate, there is always dome delay before the status bar updates, due to the parallel nature of the processing. Also, folder rescans invalidate categories and collections, which need to recalc.
The database has ~230,000 files and ~12,000 categories.
Several missing cache images were updated or created.
There seem to be some expensive (slow) filters enabled. The Filter Panel reports delays of over 20 seconds when accessing the database, while collections are updated (> 20s to recalc for Reject because the database was super busy doing lots of stuff). The File Properties filter took 20 seconds to apply. Either large scope or system not coping too well.
230K files is substantial, and keeping categories, collections and filters updated will take at least twice as long as with a 100K database. IMatch copes, but it basically depends on how fast the database can read and write data. And some operations need to be performed in a "transaction", temporarily preventing other background threads from accessing the database.
Maybe disable or close filters you don't need so IMatch does not have to keep them up-to-date.
The AddOrUpdateFolder operation in the log file clocked at 2844ms.
Of course the UI is not immediate, there is always dome delay before the status bar updates, due to the parallel nature of the processing. Also, folder rescans invalidate categories and collections, which need to recalc.
The database has ~230,000 files and ~12,000 categories.
Several missing cache images were updated or created.
There seem to be some expensive (slow) filters enabled. The Filter Panel reports delays of over 20 seconds when accessing the database, while collections are updated (> 20s to recalc for Reject because the database was super busy doing lots of stuff). The File Properties filter took 20 seconds to apply. Either large scope or system not coping too well.
230K files is substantial, and keeping categories, collections and filters updated will take at least twice as long as with a 100K database. IMatch copes, but it basically depends on how fast the database can read and write data. And some operations need to be performed in a "transaction", temporarily preventing other background threads from accessing the database.
Maybe disable or close filters you don't need so IMatch does not have to keep them up-to-date.
#69
General Discussion and Questions / Re: Folders requiring rescan
Last post by rolandgifford - September 02, 2024, 04:40:43 PMI haven't deliberately turned off background scanning and can't see an option to do that on the Indexing tab in Preferences.
I'm not aware of anything running on my IMatch PC which would change folder timestamps. I do run mirroring software on a different PC which synchronises the indexed images to another PC. Perhaps that updates the source timestamp when it finds an image to copy, I'll check on that. It is a potential culprit given the large number of GPS changes I made recently.
The scan process (with the updating message bottom right) took a lot longer than 3 seconds.
If this flag is supposed to clear in the background, it could be that the number needing rescanning has built up as I haven't run IMatch for about a week. I'll leave it running to clear the backlog.
Thanks for looking
I'm not aware of anything running on my IMatch PC which would change folder timestamps. I do run mirroring software on a different PC which synchronises the indexed images to another PC. Perhaps that updates the source timestamp when it finds an image to copy, I'll check on that. It is a potential culprit given the large number of GPS changes I made recently.
The scan process (with the updating message bottom right) took a lot longer than 3 seconds.
If this flag is supposed to clear in the background, it could be that the number needing rescanning has built up as I haven't run IMatch for about a week. I'll leave it running to clear the backlog.
Thanks for looking
#70
General Discussion and Questions / Re: Folders requiring rescan
Last post by Mario - September 02, 2024, 04:18:20 PMUsually you won't even notice this.
IMatch detects that the folder timestamp has changed or responds to a Windows message and then rescans the folder in the background. When no changes are found, this is a very quick process.
The "refresh" icon is only visible next to the folder name as long as the folder has not been processed. When multiple folders are updated, IMatch processes them in order.
A quick scan of the 7,000 lines in the log file you've provided does not show that a file has actually been added or updated. The scan took 3 seconds.
Do you have background scanning disabled (Edit > Preferences > Indexing)? This would mean that you have to manually rescan folders when IMatch detects a change in the "last modified" timestamp.
Maybe you run any tools or software which causes the folder timestamps to change?
I see a number of "Cache image outdated or missing" for RW2 logs. Do you use cache purging or do you have deleted / moved the cache folder?
IMatch detects that the folder timestamp has changed or responds to a Windows message and then rescans the folder in the background. When no changes are found, this is a very quick process.
The "refresh" icon is only visible next to the folder name as long as the folder has not been processed. When multiple folders are updated, IMatch processes them in order.
A quick scan of the 7,000 lines in the log file you've provided does not show that a file has actually been added or updated. The scan took 3 seconds.
Do you have background scanning disabled (Edit > Preferences > Indexing)? This would mean that you have to manually rescan folders when IMatch detects a change in the "last modified" timestamp.
Maybe you run any tools or software which causes the folder timestamps to change?
I see a number of "Cache image outdated or missing" for RW2 logs. Do you use cache purging or do you have deleted / moved the cache folder?