How to stop automatic reading of Metadata

Started by voronwe, May 19, 2021, 09:16:23 AM

Previous topic - Next topic

voronwe

Hi Mario

I have the following problem: Using IMatch on two different Computers, the following happened:
I changed the Metadata of a lot of files on PC1 and wrote this Metadata to the files.

Then I copied the DB-File to PC2.

On PC2, the same Image-structure is available, but of course with an older version of the files, wich does not include the latest DB.

Reason why I'm doing it this way is that I'm currently updating a lot of older files and having it on PC2 I can do this also when e.g. travelling or whatever.

When opening the copied DB an PC2, IMatch starts immidiatly to read Meta from the files, which will overwrite the Metadata in the DB (especially the keywords which took a long time to make them).

In "Hintergrundverarbeitung", everything was switched off (as seen in the attached image). However the automatical reading starts and there is no way to stop it.

When changing back to an older version of the DB, the "Häckchen" in "Dateibeziehungen" was set again.

So my questions are the following:

1) Is there a way to completely switch off the automatically reading of Metadata from files? And only do this when I as a uses explicity allow it.
And this setting not based on a DB, but on a IMatch installation

2) Why is it not possible to stop the reading manually?

3) I have the feeling that  some of the setting are for one installation of IMatch and some are DB-related. Is there a way to see in the settings which one is for which?

Greetings

EDIT: I tested it on PC1: There also "Automatic Reading" was switched off. However, when I started the DB, it starts reading.
I did yesterday a lot of writing into Metadata of files, and, as far as I understood, IMatch then reads back the MetaData from File. As this took quite long, it could be that I closed IMatch before this was finished.
However, having "Automatic Reading" switched off, I do not expect the software to read something from the Metadata until I explicitly allow it manually

After finishing this reading on PC1, I closed IMatch and copied the DB to PC2. In this case, it does not start the reading. So it seems to me that IMatch continues the reading, even when being on a different PC.


Mario

I'm not quite sure that I understand what you are trying to do. Sounds like you are trying to break IMatch?

When IMatch detects that a file has changed on disk (newer "last modified" timestamp or a corresponding message received from Windows), IMatch will re-import the file, reload the metadata, thumbnail, cache info, visual query data etc. This is an integral part of the system. You cannot turn this off. And why would you do such a thing?

When I understand you correctly:

- You have two sets of files on two computers (with same structure but maybe different contents!)
- You can two separate databases (or more or less out-of-sync copies of the same database) on both PCs, each indexing a copy of the files
- From time to time you change the contents of files on PC 1 and then copy these files to PC 2
- But you may have changed metadata for the file you just copied also on the other PC => DANGER
- At this point, the database on PC 2 no longer matches what actually is in these files
- But you somehow want to trick the system to not re-import the changed metadata in PC2

Having two databases on two PCs indexing the same files (or, rather, copies of these files) and changing the metadata for the same files (or their copies) on both computers at the same time without properly keeping the databases in sync is a recipe for disaster. This will end in tears. Just don't.

You can force IMatch not to re-index by opening the database in read-only mode (Database menu).
But then you always need to remember which files you have changed on the other PC and copied over, and that what you see in IMatch and what IMatch assumes about the file no longer matches the real contents of the file. A VERY DANGEROUS workflow.

Also, disabling all background and file-system options just like that is very problematic.
I don't recommend this because it can cause a lot of harm by IMatch not seeing file system updates, not updating relations anymore. I don't even test this.

Typically you just would NOT change the metadata of the same file on two computers and WHEN you change the data of files, copy them to the other computer INCLUDING the IMatch database.
This way you have never any sync issues. This is what you usually do when traveling. On return, you copy the new files and the modified database back to your main computer.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

voronwe

Hi Mario

thanks for the fast reply. I made some edit on my first post, maybe you did not see it.

I try again to tell what I do:
PC1 (Master):
- Has the files
- Has the DB
- Will write metadata to files

PC2 (Slave):
- Has a copy of the files (not latest version, because we are talking here about ca. 60.000 files)

Behaviour on PC2:
- Copy DB from PC1 to PC2 -> overwrite existing DB on PC 2
- write Metadata into DB -> Do not write this Metadata to the files on PC2
- When finished: Copy DB to PC 1

Behaviour on PC1:
- Take the lates DB from PC2, overwrite old version, open it in IMatch
- Write changed Metadata to the files.
- Maybe also do some changes in the DB
- Close IMatch, copy DB to PC2

Why I'm doing it this way: PC2 is on my work, and I can change some files maybe during waiting time for my work application to finish it's job

So I do not sit there there the whole day to update the DB, only time-by-time, and I want to keep the copy time as short as possible, that is why I only copy the DB

Important: There is only on DB, this one is copied between the two PCs

Quote from: Mario on May 19, 2021, 09:37:48 AM
I'm not quite sure that I understand what you are trying to do. Sounds like you are trying to break IMatch?

When IMatch detects that a file has changed on disk (newer "last modified" timestamp or a corresponding message received from Windows), IMatch will re-import the file, reload the metadata, thumbnail, cache info, visual query data etc. This is an integral part of the system. You cannot turn this off. And why would you do such a thing?
As explained above, because the files are different on two different systems.

Quote from: Mario
Having two databases on two PCs indexing the same files (or, rather, copies of these files) and changing the metadata for the same files (or their copies) on both computers at the same time without properly keeping the databases in sync is a recipe for disaster. This will end in tears. Just don't.
Again: I'm updating the Metadate on PC2 only in the DB, not writing to the files on PC2

Quote from: Mario
You can force IMatch not to re-index by opening the database in read-only mode (Database menu).
But read-only would mean that I can not update the DB

I the way I work when I change something on PC2 in the DB it is maked as "Metadata changed" and I can use this marker for writing the Metadata back to the file.

I think there is a misunderstanding between us. It seems that my problem came from the point that IMatch was not finished with rereading when closing it.

However my point is the following: A file is a file, and a DB is a DB. So I do not care about whatever is written in the file until I explicitly say to the DB "reread it".
It could be that I have Information in the DB which should not be given to the file. Example: Name of Person in file, because there are cases where I will publish a file and there Person in the picture explicitly forbit to have her/his name in the file (DataCrawler!), but internally I want to have this name in the DB.

And in my case, it tries to update the DB from an older version of the file, which obviously is wrong. I'm not a fan of doing stuff like rereading automatically, because this will also end up in tears.

So from a setting like "Hintergrundaktualisierung aus", I expect that it should not read from files for whatever reason until I allow it

Mario

Try to keep both the database and the files in sync. Don't mix.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

voronwe

#4
Quote from: Mario on May 19, 2021, 10:58:18 AM
Try to keep both the database and the files in sync. Don't mix.

Ok.

But please consider that there are use-cases where this is not possible (I pointed one out)

I the end, trying to keep all in sync so that it will work sounds to me like "redundant data"

loweskid

Quote from: voronwe on May 19, 2021, 10:35:58 AM
Example: Name of Person in file, because there are cases where I will publish a file and there Person in the picture explicitly forbit to have her/his name in the file (DataCrawler!), but internally I want to have this name in the DB.

Could you not use Categories for this?  For this situation I have a main 'People' category and then individual child categories for each person.

voronwe

Quote from: loweskid on May 19, 2021, 03:58:39 PM
Quote from: voronwe on May 19, 2021, 10:35:58 AM
Example: Name of Person in file, because there are cases where I will publish a file and there Person in the picture explicitly forbit to have her/his name in the file (DataCrawler!), but internally I want to have this name in the DB.

Could you not use Categories for this?  For this situation I have a main 'People' category and then individual child categories for each person.

Well, the Face recognition writes into the MetaData, and without the Face recognition, the Person-View does not work.

BTW: That was only an example, maybe I also do not want to store the GPS in the File, or some comments, or whatever.

thrinn

My 2 cents (well, 3 cents):

For information that should be contained in the DB only, but not in the images itself, metadata is maybe not the right place to put the information in. Categories (with exception of @Keywords) and Attributes should be used to store "in-DB only" information. But you are right, some features are connected to or integrated with metadata, so you will loose some functionality.

If you are concerned about putting specific metadata into pictures, this should only be of concern when you share the pictures in question, right? Could you not use the Batch processor to strip the metadata before sharing the files?

If you really want to work on PC2 only "in" the DB, not with the pictures itself: Did you try just to leave the pictures on PC1 only? In this case, on PC2, IMatch would recognize the files as offline. No risk of accidentally rescanning or writing back. If I remember correctly, you should be able to work with offline files at least partially, e.g. change some metadata. I only made a quick test because I do not work this way but maybe you might give this option a try?
Thorsten
Win 10 / 64, IMatch 2018, IMA

voronwe

Quote from: thrinn on May 19, 2021, 05:47:22 PM
My 2 cents (well, 3 cents):

For information that should be contained in the DB only, but not in the images itself, metadata is maybe not the right place to put the information in. Categories (with exception of @Keywords) and Attributes should be used to store "in-DB only" information. But you are right, some features are connected to or integrated with metadata, so you will loose some functionality.

Well, it is diffcult: Of course I want to have the possiblity to have everything in the picture, so all the information will stay independent from IMatch (the basic rule: Never rely on a propritary format -> Worst reason can be found here.
So I'm reworking my DB now completely to keywords, and In the end all categories should be build from this. That is also why I share this DB: Because I was too lazy in the past, it is a bunch work.

Quote from: thrinn
If you are concerned about putting specific metadata into pictures, this should only be of concern when you share the pictures in question, right? Could you not use the Batch processor to strip the metadata before sharing the files?
As far as I understand Mario, the full stripping (without rerendering the image) will be included in Version 2021. But for just stripping specific Metadate, I think I have to write a Python-script for my own.

Quote from: thrinn
If you really want to work on PC2 only "in" the DB, not with the pictures itself: Did you try just to leave the pictures on PC1 only? In this case, on PC2, IMatch would recognize the files as offline. No risk of accidentally rescanning or writing back. If I remember correctly, you should be able to work with offline files at least partially, e.g. change some metadata. I only made a quick test because I do not work this way but maybe you might give this option a try?

I'm not sure how good things like face recognition will work when you have only the preview. And also in some cases I need the full picture to see what's on it.

Well, at least I found out now to wait until the background stuff is finished before closing IMatch, and - of course - take care of the Backup.

I hope at least that there is no hidden feature in Imatch which writes automatically into files, even if I switched off "Write back Metadata"  ;D

Mario

QuoteBut for just stripping specific Metadate, I think I have to write a Python-script for my own.

This can be easily done using the ECP. There are templates to remove XMP, legacy IPTC, EXIF and even all metadata.
You can use these templates as examples to remove even individual tags, when needed.
No Python script required.

-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

RobiWan

@voronwe

maybe I didn't understand it correctly. But if I do, the easiest way to do what you want is to make IMatch see the images as offline on PC2.
So you can change as much as you want, but IMatch can't read and write "old" metadata.

Robert