sinnus: IMatch write-back performance

[sinus]

Since your database takes 70 seconds to load (normal would be 10 to 20 seconds from a SSD) I recommend you delete all data-driven categories you don't use or set them to manual. On slower disks data-driven categories can take a long time to update for a 200,000 files database, causing sluggish performance.

Very true.
After some tries with my 270'000 - files - DB, I set all my Data-driven-categories to manual (except one last I set to automatically, I want see, if this affects the system also).
The same for write-back metadata automatically - with such large DBs I think, it is not wise to set it to automatically, but set it to manually.

The problem is, that I cannot forseen, at what time IMatch does update DDs or Auto-write-back.
And often it happend to me, that IM was working on this, when I wrote Metadata or did something else "important for me".

For me a solution would be, if IMatch would update DDs and Write-back, say, only every hour.
Then I would know, if it does update, I have to wait, until it is finnish, then I could work again for one hour.

One hour is just an example, also every 30 minutes would be ok. Or every 2 hours.
Of course the best would be, if I could choose this in the properties myself. I would then choose "every 2 hours" - in my case.

If IMatch had such a possibility, it would be for me the best compromise between manual (what I often forget to let run) and automatically.

[Mario]

This would not help at all in your situation. If IMatch would write-back automatically every 30 minutes it could still interfere with your work.
Check the log file and search for #sl to find what is making your database so slow. I've worked this morning with a 420,000 files database, adding and editing metadata, keywords, tagging etc. When I was done I wrote back 5500 files (mostly JPEG) in about 10 minutes...

[sinus]

This would not help at all in your situation. If IMatch would write-back automatically every 30 minutes it could still interfere with your work.
Check the log file and search for #sl to find what is making your database so slow. I've worked this morning with a 420,000 files database, adding and editing metadata, keywords, tagging etc. When I was done I wrote back 5500 files (mostly JPEG) in about 10 minutes...

Thanks, Mario, I will check the log.

But only to say: it would help a lot, because then I would know, it will take, say 1 or two minute, but then I would know, I have again a pause for 1 hour (as an example), until the next "auto-run" comes.

I have to take pictures these weeks from church bells in the tower itself.
You can imagine, if the bells (Kirchturmglocken) does strike (schlagen), the sound is quite loud and it can be dangerous for the ears.
Hence I must give attention, that in these cases I am not near the bells.

The bells strikes every hour specialy loud and long, so I know, every hour I cannot take pictures. Hence I simply go away every hour (some minutes before), come then back and then I know, I have one hour time to take pictures.

If I would not know, when the bells would ring, it would be very dangerous.

If I compare this with IMatch, it is a bit the same. If I would know, well, every hour IM does some automatically work (DD and autowrite-back), this would be very fine for me.
But now I do not know, when this happens.

But of course, Mario, this was just an idea, when I read this thread.

And if I am thinking more, I quite sure, specially for users with large DBs this could be a fine thing. (man hätte praktisch ein zeitkontrolliertes Autosystem (DD / autowriteback).

But, as you proposed, I will check my log.

[Mario]

That's why IMatch allows you to switch between immediate write-back and manually triggered write-back.
The background write-back is not the problem because it runs in the background and does not block the user interface. But, depending on your configuration, the following reload of the files and metadata updates to the database will trigger lots of things. Category and collection updates, file window reloads, updates to all panels.

This may, or may not, cause any issues. It depends on how fast your computer is. The log file will show which operations are slow and then we can think about improving things. If you have set all categories to manual already we should not see any #slowcat entries, for example.

Unfortunately, another thread was now hijacked with an unrelated topic. I will split your content into a separate thread. The original thread was about an database not loading. Not about slow write-back performance.

[sinus]

Sorry for the hijacking and thanks for split this.

I will report the next time, when I have a lot of images from an event.

[Mario]

Great. When I see the log file I should learn about what's making the system slow. Maybe I can offer advice or even improve things for IMatch 2018.

[ThomasAdams]

Perhaps an alternative solution would be to allow users to set specific times when these types of things happen - much like scheduled tasks that could be performed during hours you would routinely be away.

[sinus]

Perhaps an alternative solution would be to allow users to set specific times when these types of things happen - much like scheduled tasks that could be performed during hours you would routinely be away.

This is also a possibility, from my point of view, thanks for your input, Thomas.
Remembers me at the bells in the church tower.

Btw: it is really quite impressive in such towers, very small (I mean really small), dirty, flies, somtimes bats, dark and mostly each quater of an hour a sound and each full hour a real "chaos-sound" up there.
And it is really important there to have a precise wrist-clock at your arms, so that you know exactly the time. Usuallly 4 minutes before an hour I go downstairs to the "clock-motor" (Uhrwerk) to escape the sound and it is also nice to see these (mostly) old motor to see in motion, the weels, bar guides and so on.
And all this stuff is often hundrets of years old and works still. 

Mario, of course I will upload the log, as soon as I have a good one (mostly after an event)

[Mario]

I have never considered writing back a few hundred files such a nuisance. I do this all the time time, almost daily. It never interfered with my work. I use the "write all pending..." command and fetch me a coffee. IMatch is done when I'm back after a couple of minutes. Setting up schedules for this would actually make this more complicated...

Anyway, feel free to add a feature request in the FR board.

[ubacher]

Writing back a lot of files can take longer than one wants at the moment.
To know how many will be written back it would be nice to have the number shown in the menu.
Saves having to go to the collections view.

[Mario]

If you choose this command, IMatch displays a dialog, telling you how many files need to be written back. You can then proceed or cancel.