What's your typical Database Diagnosis runtime?

Started by Mario, November 26, 2024, 06:53:03 PM

Previous topic - Next topic

Mario

I've made some changes to the database diagnosis in IMatch 2025 to improve performance.

For me, the diagnosis for me 850,000 files database runs in about 4 minutes now. Notebook with fast .m2 SSD.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

rolandgifford

IMatch database diagnosis logfile created: 26/11/2024 18:03:11.
Last diagnosis run: 12/11/2024 14:51:06 (14 days ago).

TIP: Search for the words 'warning' and 'error' (without the quotes) to find warnings and errors in this file.

Application Info:
    Version:                      23.14.0.2
    Filename:                     C:\Program Files\photools.com\imatch6\IMatch2023x64.exe

General Database Info:
    Database file name:           C:\IMatch Database\IMatch Database.imd5
    Database file size on disk:   14.03 GB
    Number of folders:            393
    Number of files:              232,855
    Number of categories:         12,666
    Clearing oid cache.

Checking Visual Query Data:
      Data items:  931,408
Completed.

Checking Favorites:
Completed.

Checking photools.com metadata:
    Result: 0 files with missing data updated.

Checking photools.com metadata tags for consistency:
Completed.
    Clearing oid cache.

Analyzing database:

Optimizing Database, rebuilding optimial index structures and query plans:
Completed in 509s with 66240 counts.

WriteTest:
Completed.

Results:
    Errors:    0
    Warnings:  0

IMatch database diagnosis logfile closed: 26/11/2024 18:12:58 (00:09.47)

JohnZeman

I just ran diagnostics on my 110,000 photo database using my 1TB NVMe .m2 SSD and it finished in 3.5 minutes.

That's significantly faster than diagnostics ran on my old computer.

axel.hennig

Approx. 160'000 files and Database Diagnosis runtime was 11min 8sec on a Laptop also with .m2 SSD.

mastodon

Quote from: Mario on November 26, 2024, 06:53:03 PMFor me, the diagnosis for me 850,000 files database runs in about 4 minutes now. Notebook with fast .m2 SSD.
What was the result with IMatch 2023?
For me:
General Database Info:
    Database file name:          C:\Downloads\All_pics.imd5
    Database file size on disk:  12,98 GB
    Number of folders:            535
    Number of files:              49 780
    Number of categories:        4 520
Completed in 63s with 8930 counts.
Samsung SSD 990 PRO 2 TB, Intel Core i7-9700, 48 GB RAM

axel.hennig

Quote from: mastodon on November 26, 2024, 11:37:50 PMWhat was the result with IMatch 2023?
I would be also interested in before/with IMatch 2025.

jch2103

My computer: Dell Desktop XPS 8960, i7-13700, 32 GB, RTX 4070; NVMe WD_BLACK SN850X 4000GB, NVMe Timetec MS12 1907 GB)
IMatch database diagnostics time: 1:29

Application Info:
    Version:                      23.14.0.2
    Filename:                    C:\Program Files\photools.com\imatch6\IMatch2023x64.exe

General Database Info:
    Database file name:          C:\Users\jchar\Documents\IMatch Database\IMatch Photos Database.imd5
    Database file size on disk:  5.72 GB
    Number of folders:            1,585
    Number of files:              116,432
    Number of categories:        13,239
John

Tveloso

My execution time was 00:01:14 for this database:

    Application Info:
        Version:                      23.14.0.2
        Filename:                     C:\Program Files\photools.com\imatch6\IMatch2023x64.exe

    General Database Info:
        Database file name:           C:\PhotoDb\PhotoDb.imd5
        Database file size on disk:   6.22 GB
        Number of folders:            1,000
        Number of files:              95,801
        Number of categories:         16,964

...on a laptop with:

    Processor = 12th Gen Intel(R) Core(TM) i9-12900HK, 2500 Mhz, 14 Core(s), 20 Logical Processor(s)
    C-Drive = NVMe PC801 NVMe SK hynix 1TB
--Tony

Jim K.

My execution time was 00:00:36 for this database:

IMatch database diagnosis logfile created: 27.11.2024 05:52:31.
Last diagnosis run: 18.11.2024 06:05:45 (8 days ago).
Application Info:
    Version:                      23.14.0.2
    Filename:                     C:\Program Files\photools.com\imatch6\IMatch2023x64.exe

General Database Info:
    Database file name:           C:\Users\JK\IMATCH_DB\IMatch Database ALL_PHOTOS.imd5
    Database file size on disk:   1,33 GB
    Number of folders:            2.333
    Number of files:              34.309
    Number of categories:         2.364
    Clearing oid cache.
...
Checking Visual Query Data:
      Data items:  136.280
Completed.

Checking Favorites:
Completed.

Checking photools.com metadata:
    Result: 0 files with missing data updated.

Checking photools.com metadata tags for consistency:
Completed.
    Clearing oid cache.

Analyzing database:

Optimizing Database, rebuilding optimial index structures and query plans:
Completed in 17s with 21227 counts.

WriteTest:
Completed.

Results:
    Errors:    0
    Warnings:  0

IMatch database diagnosis logfile closed: 27.11.2024 05:53:07 (00:00.36)


Run on:
HP Z240 Workstation
XE3-1245V5 3.70 GHz
32 GByte RAM
256 GB SSD-Platte   -> IMatch Database
2 x 3 TB SATA-Platte (WD Red)   -> Folders (Photos)
Windows 10 Prof. - 64-Bit


Mario

Quote from: axel.hennig on November 27, 2024, 12:14:43 AM
Quote from: mastodon on November 26, 2024, 11:37:50 PMWhat was the result with IMatch 2023?
I would be also interested in before/with IMatch 2025.
On a cold database (not in the file system cache) the diagnosis took 6 minutes before. Now between 3:30 and 4:00 minutes.
The improvement is in the "Files" step, so the more files a database has, the bigger the improvement will be.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

RainerG

Application Info:
    Version:                      23.14.0.2
    Filename:                    C:\Program Files\photools.com\imatch6\IMatch2023x64.exe

General Database Info:
    Database file name:          F:\Daten\_Programme\IMatch5\IMatch Datenbank.imd5
    Database file size on disk:  2,10 GB
    Number of folders:            2.667
    Number of files:              64.800
    Number of categories:        1.122

54 seconds

Win 10
Intel(R) Core(TM) i7-6700 CPU @ 3.40GHz  3.41 GHz
16,0 GB (15,5 GB verwendbar)
Seagate ST4000DM004 - 4TB
RainerGausH
www.BunteReisebilder.de

Mario

#11
Thanks for all who posted. All users will experience more or less faster diagnosis times.
Professional users or users with a lot of images will see the most benefit (including myself, with my personal database now reaching soon 1,000,000 files).

Since I maintain about a dozen test databases for various purposes, and I like to run diagnosis on them every week, this will save me some time ;)
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

Darius1968


My database has almost 366,000 files and sometimes my diagnosis (optimization included) takes 4 min., but sometimes over 8 min.  Why is this? 

Darius1968


I'm attaching a log file here.  It is a recording of my opening the database and running the diagnosis, of which this 1st event took just over 4 min.  However, with the database still open, I performed a 2nd diagnosis, which took almost 8 min.!  I'm perplexed as to why this is happening, and so I've attached a log file for analysis. 

Darius1968

Specs to go along with my posts above: 

Application Info:
    Version:                      23.14.0.2
    Filename:                    C:\Program Files\photools.com\imatch6\IMatch2023x64.exe

General Database Info:
    Database file name:          C:\__OLD-PC\D\_00\IMatch Databases\2021\Personal Assets.imd5
    Database file size on disk:  11 GB
    Number of folders:              4,796
    Number of files:              365,332
    Number of categories:      13,387

Win 11
Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz 
16 GB (6 GB Available)
Micron_1100_MTFDDAK2T0TBN : 2048.4 GB

Mario

Could be anything. Virus checker kicking in, File system cache full, other applications working with the disk, ...

I noticed that IMatch logs warnings about the system memory being used up almost.
When IMatch starts 10 GB of the 16GM in the system are available.

IMatch reports it's memory peak usage as 2GB, but still the system memory utilization is spiking at 94%.
Could be Windows allocating more memory for the cache, could by another application using a lot of memory, reducing the file system cache which then results in slower diagnosis performance.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

Darius1968


IMatch was the only application running at the time. 

Mario

Quote from: Darius1968 on December 02, 2024, 08:39:58 AMIMatch was the only application running at the time. 
IMatch peaked at 2 GB, so with 10 GB available when IMatch started + 2 GB => 12GB, which is not exactly 94% of 16GB.
A service, virus checker, Windows could have allocated the rest. Just that one program is running does not mean that 80 processes and services run in the background. Check Task Manager to see how memory is used.

For a database with almost 400,000 files and 15,000 categories, a diagnosis between 4 and 8 minutes is OK.
95% of the diagnosis runtime is database time, and this time depends solely on how fast the SSD is.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

sinus

Quote from: Mario on December 02, 2024, 09:07:04 AM95% of the diagnosis runtime is database time, and this time depends solely on how fast the SSD is.

I have also about 370'000 images, my diagnosis time is mostly about 30 minutes. Hence 8 minutes looks very good for me.  
Best wishes from Switzerland! :-)
Markus

Mario

30 minutes? This is really slow. Is the database on a SSD or spinning disk?
Virus checker configured with an exclusion for the folder (!) containing the database?
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

sinus

Quote from: Mario on December 02, 2024, 10:25:39 AM30 minutes? This is really slow. Is the database on a SSD or spinning disk?
Virus checker configured with an exclusion for the folder (!) containing the database?

I will check this, thanks. The images and DB are on two separte SSD and I believe, I did the exclude the DB-folder from the virus-checker, but I will check this and report here.

But now first, uhhh, I have to go out in the cold, and must take pictures from 100 persons outside.  :o
Best wishes from Switzerland! :-)
Markus

sinus

My last diagnosis was about 41 Minutes. But this is for me normal, I have such times since a long time, about half an hour.
(same for compacting about 25 Minutes).

I do not know, why it takes so long, but generally I think, it could have two reasons:

1) maybe a too complex category-system
2) maybe my ssd-drives do not respond quick enough

Also other things, what is quites slow, like manual propagation, I think, it could have to do with my SSDs, but I could not find out, how to check them.
I have excluded the whole IMatch-folder and also my database from defender, but not the photos itself.

Best wishes from Switzerland! :-)
Markus

Mario

The diagnosis log file contains no detailed timing data (the regular IMatch log file does), so there is not much to tell.
400K files is a larger database, but not super-large.

Did you keep the IMatch log file of that session? If not, maybe ZIP the log file when you run the next diagnosis and attach it. Then I can see what takes how long, which might be helpful.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

Jingo

well.. my diagnosis ran in 3 minutes which is about the normal amount of time for me... 

Application Info:
    Version:                      23.12.0.2
    Filename:                    C:\Program Files\photools.com\imatch6\IMatch2023x64.exe

General Database Info:
    Database file name:          C:\Users\fuego6\Documents\ImatchDB\PhotoDatabase.imd5
    Database file size on disk:  4.20 GB
    Number of folders:            256
    Number of files:              106,275
    Number of categories:        2,144

Mario

That's about normal.

I've noticed that sinus' log shows 10 million history records and 150,000 files in the cache.
But I cannot tell without the IMatch log file if this caused the diagnosis to be slow.

@sinus:

150,000 cache files is substantial. Maybe consider purging old cache files you no longer need. This will free a lot of disk space.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

sinus

Thanks, Mario, I will check this and the next time I will add the IMatch log file.
Best wishes from Switzerland! :-)
Markus

StanRohrer

Windows 11 booting/running on (C:) m.2 PCIE SSD, home built desktop computer. iMatch database and 14,232 working/current files on same drive.
Photo archive of 201,651 files are on a (P:) SATA rotating hard drive.
3 Minutes, 7 seconds, diagnostics run time.
-----
IMatch database diagnosis logfile created: 12/6/2024 8:50:02 AM.
Last diagnosis run: 12/5/2024 9:21:45 PM (0 days ago).
Application Info:
    Version:                      23.14.0.2
    Filename:                    C:\Program Files\photools.com\imatch6\IMatch2023x64.exe

General Database Info:
    Database file name:          C:\Users\Stan\My Databases\Imatch2017 Database\StanRPhotos01.imd5
    Database file size on disk:  10.57 GB
    Number of folders:            2,053
    Number of files:              215,883
    Number of categories:        15,443

....
Checking file history:
      Entries:  2,932,654
Completed.

Checking time line:
      Entries:  3,560
Completed.

Checking Metabase:
Completed.

Checking Cache:
    29,555 files in cache folder.
Completed.

Checking Annotation Objects:
Completed.

Checking Attribute Database:
      Attribute Sets: 1
        'Image' [1] class 10 with 4 attributes.
Completed.

Checking Entity Graph:
Checking Event Graph: Events:
    0 events loaded.
Checking People Graph: Events:
PersonManager Stats:
    Files with faces: 3310
    Persons: 206
    Faces: 6977
    Confirmed: 6842
    Unconfirmed: 0
    Unassigned: 135
    Manual: 0
    Files: 3310
    Faces: 6977
    Entities: 6842. Confirmed: 6842
    Entity keywords: 18629.
    Vector data size: 4889KB
    Thumbnail data size: 89905KB
    Annotations without face data: 494KB
Completed.

Checking Visual Query Data:
      Data items:  863,172
Completed.

Checking Favorites:
Completed.

Checking photools.com metadata:
    Result: 0 files with missing data updated.

Checking photools.com metadata tags for consistency:
Completed.
    Clearing oid cache.

Analyzing database:

Optimizing Database, rebuilding optimial index structures and query plans:
Completed in 105s with 58802 counts.

WriteTest:
Completed.

Results:
    Errors:    0
    Warnings:  0

IMatch database diagnosis logfile closed: 12/6/2024 8:53:09 AM (00:03.07)

-----------
Device name      Stan2021
Processor          11th Gen Intel(R) Core(TM) i7-11700K @ 3.60GHz  3.60 GHz
Installed RAM    32.0 GB (31.8 GB usable)
System type      64-bit operating system, x64-based processor
Edition  Windows 11 Pro
Version 23H2
Installed on        ‎6/‎15/‎2024
OS build            22631.4460

Mario

Quote from: sinus on December 06, 2024, 02:57:14 PMThanks, Mario, I will check this and the next time I will add the IMatch log file.
The slowest part is the

Optimizing Database, rebuilding optimial index structures and query plans

which takes 33 minutes (!) on your system.
Try this: When starting the diagnosis hold down the CTRL key before clicking Run.
This should make the diagnosis 33 minutes faster.

The Optimizing database is a database-internal routine which rebuilds optimal indices and query helpers based on the current contents of the database. This is required to keep the database in optimal shape, but also a part which can take quite long. Your database 33 GB in size, and has lots of metadata and stuff. This will take the database system a while to sieve through to figure out how to optimally structure things. IMatch has no influence on the performance of this step.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

thrinn

My diagnosis runs in less than 2 minutes, but I only have about 32.000 files. Therefore, not much room for performance improvements.
Thorsten
Win 10 / 64, IMatch 2018, IMA

Mario

Quote from: thrinn on December 06, 2024, 04:37:39 PMMy diagnosis runs in less than 2 minutes, but I only have about 32.000 files. Therefore, not much room for performance improvements.
The average size in telemetry is about 100,000 files per database, with many between 200,000 and 300,000 files.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

graham1

44 minutes 51 seconds, running a second time after a Metabase warning first time.  Around 800,000 mainly RAW files.  Database and cache folder both on SSD, excluded from antivirus access.

Mario

I guess 30 minutes or more is the database optimization step.
How long does the diagnosis run when you hold down CTRL while starting it (while clicking on Run)?

For 800,000 files, IMatch 2025 will maybe squeeze out an extra minute or two.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

sinus

Yes, the optimization takes a long time. Without it my diagnosis runs for the 360'000 files in 5.15 min.
And the optimization takes around 35 minutes.

Mario, this holding the Ctrl while starting the diagnosis works fine, is it an idea to add there a button to click, that is goes not forgotten? Only an idea, really.
Best wishes from Switzerland! :-)
Markus

graham1

With ctrl held down at start, nothing done since last diagnosis, 9 min 41 seconds - a bit of a difference!  My computer is a desktop i9-7900X 3.3GHz with 48GB RAM and an NVidia GTX 1060 6GB running Windows 10 Home.

Mario

The database optimization takes some time, but is very helpful with keeping things smooth.
I've figured that doing that every ~ two weeks (standard diagnosis interval) is worth it.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

Mees Dekker

IMatch database diagnosis logfile created: 08-12-2024 14:41:39.
Last diagnosis run: 06-12-2024 19:48:15 (1 days ago).

TIP: Search for the words 'warning' and 'error' (without the quotes) to find warnings and errors in this file.

Application Info:
    Version:                      23.14.0.2
    Filename:                    C:\Program Files\photools.com\imatch6\IMatch2023x64.exe

General Database Info:
    Database file name:          C:\Users\Public\Pictures\Mees.imd5
    Database file size on disk:  7,89 GB
    Number of folders:            1.721
    Number of files:              160.238
    Number of categories:        6.097
    Clearing oid cache.

Checking files:
    Performing sort array maintenance:
Completed.


Checking file history:
      Entries:  2.741.012
Completed.

Checking time line:
      Entries:  6.709
Completed.

Checking Metabase:
Completed.

Checking Cache:
    10.183 files in cache folder.
Completed.

Checking Annotation Objects:
Completed.



IMatch database diagnosis logfile closed: 08-12-2024 14:47:37 (00:05.58)

Windows 11 Pro for Workstation
Apparaatnaam DELL7820
Processor Intel(R) Xeon(R) Silver 4214 CPU @ 2.20GHz  2.19 GHz
Geïnstalleerd RAM-geheugen 32,0 GB (31,6 GB beschikbaar)

Type systeem 64-bits besturingssysteem, x64-processor
Pen en aanraken Ondersteuning voor peninvoer

Mario

-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

meyersoft

Application Info:
    Version:                      23.14.0.2

General Database Info:
    Database file size on disk:  3,51 GB
    Number of folders:            3.058
    Number of files:              109.271
    Number of categories:        13.884

Checking file history:
      Entries:  277.510
Completed.

Checking time line:
      Entries:  6.980
Completed.

Checking Cache:
    6.511 files in cache folder.
Completed.

Checking Visual Query Data:
      Data items:  437.024
Completed.

Analyzing database:

Optimizing Database, rebuilding optimial index structures and query plans:
Completed in 53s with 55672 counts.

IMatch database diagnosis logfile closed: 08.12.2024 17:40:48 (00:03.06)

3 minutes for 109.000 files seems ok to me.

Mario

Yes. One minute for the optimize part, 2 minutes for all the rest. Fast enough.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

Mario

Thanks for all users who reported their results. Very helpful.

Looking over your data, I've had an idea for an improvement. Not for the diagnosis, but for IMatch.
IMatch 2025 now has a feature to automatically purge the file history (Edit menu > Preferences > Application: Diagnosis).

IMatch maintains a File History for each file, logging events like when the file was added, modified, metadata or categories edited, viewed etc. Can be very helpful, and is also the source for the "Recent" collections and the corresponding Dashboard panel.

I've noticed that several million history entries is normal and OK.
But if you don't use the History Panel or you are not interested in file histories from two years ago, you can enable the automatic purge and e.g. let IMatch purge history entries older than 90 days, six months or 1 or two years.
This reduces disk space usage of the database (not that much), but a smaller history table in the database makes adding new entries a bit faster.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

Pawel

5y old Dell i7 16GB RAM laptop, database on m2 SSD disk C:, files on two USB 3.0 connected external drives: SSD 4TB and HDD 5TB:

IMatch database diagnosis logfile created: 11.12.2024 13:55:12.
Last diagnosis run: 23.11.2024 23:36:44 (17 days ago).

TIP: Search for the words 'warning' and 'error' (without the quotes) to find warnings and errors in this file.

Application Info:
    Version:                      23.14.0.2
    Filename:                    C:\Program Files\photools.com\imatch6\IMatch2023x64.exe

General Database Info:
    Database file name:          C:\Users\pawel\Documents\IMatch\database\IMatch Database.imd5
    Database file size on disk:  13,19 GB
    Number of folders:            7 352
    Number of files:              257 614
    Number of categories:        20 612
    Clearing oid cache.

[...]

Checking file history:
      Entries:  4 896 026
Completed.

Checking time line:
      Entries:  8 085
Completed.

Checking Metabase:
      Warning: 29 superfluous entries in rel_stack for non-masters. Removed.
Completed.

Checking Cache:
    9 618 files in cache folder.
Completed.

Checking Annotation Objects:
Completed.

Checking Attribute Database:
      Attribute Sets: 6
        'Artwork or Object' [7] class 10 with 2 attributes.
        'Artwork or Object catalog' [8] class 0 with 17 attributes.
        'Metadata' [5] class 10 with 1 attributes.
        'Metadata Backup' [1] class 10 with 7 attributes.
        'Person' [4] class 0 with 4 attributes.
        'Source catalog' [9] class 0 with 1 attributes.
Completed.

Checking Entity Graph:
Checking Event Graph: Events:
    0 events loaded.
Checking People Graph: Events:
PersonManager Stats:
    Files with faces: 86050
    Persons: 1461
    Faces: 203956
    Confirmed: 170538
    Unconfirmed: 1978
    Unassigned: 31440
    Manual: 0
    Warning: 4 face records without thumbnail found. The containing files were added to the '__FACES_WITHOUT_THUMBNAIL' category.
    Files: 86050
    Faces: 203956
    Entities: 172516. Confirmed: 170538
    Entity keywords: 0.
    Vector data size: 52582KB
    Thumbnail data size: 1585183KB
    Annotations without face data: 134232KB
Completed.

Checking Visual Query Data:
      Data items:  1 030 232
Completed.

Checking Favorites:
Completed.

Checking photools.com metadata:
    Result: 0 files with missing data updated.

Checking photools.com metadata tags for consistency:
Completed.
    Clearing oid cache.

Analyzing database:

Optimizing Database, rebuilding optimial index structures and query plans:
Completed in 1368s with 48728 counts.

WriteTest:
Completed.

Results:
    Errors:    0
    Warnings:  2

IMatch database diagnosis logfile closed: 11.12.2024 14:25:04 (00:29.52)

Mario

22 minutes for the optimize step, about 7 minutes for all the rest.

I see these warnings:

Warning: 29 superfluous entries in rel_stack for non-masters. Removed. 

This warning is logged when IMatch detects "version records" for which there is no master.
Can you recall what you did with versions / masters recently?
This is a rare (and harmless) issue, but should not happen.

Warning: 4 face records without thumbnail found. The containing files were added to the '__FACES_WITHOUT_THUMBNAIL' category. 

When IMatch creates a face annotation, it produces a thumbnail from the image region covered by the annotation.
This warning tells me that there are 4 face annotations without a thumbnail.
When you look at the __FACES_WITHOUT_THUMBNAIL  category, is there anything special about the images or the face annotations?
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

Pawel

Quote from: Mario on December 12, 2024, 10:09:09 AM22 minutes for the optimize step, about 7 minutes for all the rest.

I see these warnings:

Warning: 29 superfluous entries in rel_stack for non-masters. Removed.

This warning is logged when IMatch detects "version records" for which there is no master.
Can you recall what you did with versions / masters recently?
This is a rare (and harmless) issue, but should not happen.
To replace JPG files in the database with a new versions I created a "replace" relationship, where new files become versions of the old JPGs. After the relationship is created I propagate all metadata from the old files to the new ones and then replace old JPGs with the new versions using IMatch "move" option.  

Quote from: Mario on December 12, 2024, 10:09:09 AMWhen IMatch creates a face annotation, it produces a thumbnail from the image region covered by the annotation.
This warning tells me that there are 4 face annotations without a thumbnail.
When you look at the __FACES_WITHOUT_THUMBNAIL  category, is there anything special about the images or the face annotations?
Nothing special - there are five images in the category, all the images have face annotations that look normal to me. I moved the annotations slightly and removed images from the category, I will see if they return there. 

Mario

#43
NEW is a version of OLD.
Propagate from OLD to NEW.
Move OLD to NEW, replacing the existing NEW with OLD.
The version is thus replaced with the master.
Unusual workflow. Need to check if IMatch handles this.

Update

I could reproduce this in exact this situation (master replaces last version).
I have fixed this for IMatch 2025.
Thanks for helping with figuring this out.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

Pawel

I am glad I could help although I think I use a different workflow to the one you described and fixed. :) Actually I use a method you've described in this post
  • make the new version of the image a version of the old image (old image is the master)
  • let IMatch propagate the metadata from the old version (the master) to the new version
  • delete the old version (the master) so that the new version stops being a version because there is no master anymore

The only modification/supplement to this workflow is that I don't delete the old version (the master) in a separate step but I move the new versions to the folder of the old versions (the masters) overwriting masters with the new versions. 


Mario


QuoteI don't delete the old version (the master) in a separate step but I move the new versions to the folder of the old versions (the masters) overwriting masters with the new versions. 
This generates the same problem I have fixed. The last version of a master is removed (by replacing the master with the version, the version with the master or deleting the master etc.) and this could leave a dangling entry in the version stack. Not harmful, detected, reported as a warning and fixed by the diagnosis.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook