How to ignore changes on XMP files?

Started by RobiWan, December 07, 2017, 05:54:18 PM

Previous topic - Next topic

RobiWan

Hi,

how do I get IMatch to ignore changes to XMP files that have actually made other software and only read in according to my instructions?


Mario

You cannot. This would make no sense. What would you want to do that?

If you change XMP data, IMatch must reload the data to show you accurate information. Changes done to XMP also affect other things like collections, categories, the time line etc.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

RobiWan

no sense? That may be your view.
A small example of what happened to me. I've created XMP files using C1 through complicated ways. There's a lot of trash here. I read all this in IMatch ~ 45000 pictures. For 2 or 3 days, I sorted everything nicely, corrected bad keywords, brought color marks to a uniform state, and so on.
I wanted to be 100% sure that C1 did not change my RAW files and synced the disk with the XMP files with my "master" Disk. Of course, the eh mostly unmanageable XMP files have been deleted.
Then I reconnected the disk, started IMatch and wanted to write the work of the last days in the XMP files. Well but Imatch has only the broken or no longer existing XMP files read in and thus simply destroyed my work.
From a software that holds something of itself, I expect that such changes will happen only with the consent of the user and not just like that. DAM that is certainly not.

I am very disappointed in any case . What the software has a backup function I do not understand. This has no meaning anyway because data on the disk have higher priority.



Mario

I'm not sure that I fully understand what you did. You updated XMP data in C1? You deleted XMP files? You moved files around on disks?

The typical workflow is this:

A) You update metadata in your RAW processor, image editor or whatnot. It updates the XMP file on disk. IMatch reloads the XMP file and has current and accurate metadata.

B) You modify metadata in IMatch. IMatch updates the XMP sidecar file or the embedded XMP record in your image file. You your other applications see the updated XMP, import it and have accurate and current info.

This is the established XMP workflow as defined by the Metadata Working Group and implemented by IMatch and all major applications.
Note: You can protect unwritten metadata in IMatch from changes done in other applications via the options provided in Edit > Preferences > Metadata 2.

I'm not sure what you tried to solve with your apparently very special and cumbersome C1 XMP workflow. What I have so far seen from C1 and how it if fiddles with metadata is sub-optimal and did not impress me at all. IMatch, on the other hand, has very strong when it comes to working with metadata. You should try to update and manage your metadata only in IMatch from now on. That's the reason of having a DAM, after all.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

RobiWan

Quote from: Mario on December 07, 2017, 09:11:51 PM
Note: You can protect unwritten metadata in IMatch from changes done in other applications via the options provided in Edit > Preferences > Metadata 2.

I will check if this is the option I was missing.


Mario

This option is only for very special situations. It is off by default for good reasons because it has side-effects you need to be aware of.
IMatch will still import EXIF, IPTC, XMP and GPS. But fields modified in the IMatch database will not be replaced during the import. With the sole purpose to allow you to write them back so the other application can sync the changes. You should not try to juggle with different states of XMP data in different applications - especially not in C1.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

Jingo

Quote from: Mario on December 07, 2017, 09:11:51 PM
You should try to update and manage your metadata only in IMatch from now on. That's the reason of having a DAM, after all.

I think this is a point easily missed by many folks that try to use a RAW editor that includes any sort of image management along with a dedicated DAM like IMatch... best option - don't do ANYTHING in C1, LR, etc to alter metadata... no ratings, no labels, no keywords, etc.  Leave all "recipes" and "non-destructive edit" data in the RAW editor's catalog... no need to write it out to the file - it will be in the catalog and backed up via your normal backup programs.  No other program can read this data with accuracy anyway so there is no need to embed it.  Then, import in IMatch and do your thing..

RobiWan

#7
Quote from: Mario on December 07, 2017, 09:30:40 PM
You should not try to juggle with different states of XMP data in different applications - especially not in C1.

I think it was a bad example. Here's another example that better represents what's just going wrong here.
What happens if you accidentally delete XMP files on the disk or due to hardware errors they are suddenly faulty? In my experience, the data is also deleted in IMatch database. And sorry but this behavior is certainly not implemented by any serious DAM software and I know some of them.
This has nothing to do with C1, LR or other software.

Mario

If you happen to run into a disk problem which selectively deletes XMP files, nothing will happen. IMatch will not even detect that the XMP file is missing.

Only when you then modify the original image in a 3rd application which does not re-create the deleted XMP file, IMatch may get into trouble. Because when the image file changes, it re-creates the XMP data from the IPTC, EXIF, GPS (XMP) data in the image file, like it would have when it sees the file for the first time. Unless you have XMP protection enabled in IMatch etc.

But such a situation is so rare and obscure, I don't even consider this as remotely likely.
If you think that you may run into such a situation, I can only recommend a daily backup of all your data with a proper application like Macrium Reflext or TrueImage or the built-in backup in Windows. And keep the backups for several weeks so you can roll-back easily. Software like Reflect or TrueImage make this very easy.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

RobiWan

Quote from: Mario on December 08, 2017, 12:10:58 AM
If you happen to run into a disk problem which selectively deletes XMP files, nothing will happen. IMatch will not even detect that the XMP file is missing.

I can not confirm. I've deleted XMP files and all the data was gone in IMatch.


Mario

IMatch does not actively scan for missing XMP files.
Most likely you manipulated the image files as well, causing a change in the last modified file system timestamp. In that case, IMatch rescans the image image file. And since you've deleted the XMP file with the previously saved XMP data, IMatch re-creates the XMP from the data gathered from the original image file.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

thrinn

Quote from: RobiWan on December 08, 2017, 12:23:35 AM
Quote from: Mario on December 08, 2017, 12:10:58 AM
If you happen to run into a disk problem which selectively deletes XMP files, nothing will happen. IMatch will not even detect that the XMP file is missing.

I can not confirm. I've deleted XMP files and all the data was gone in IMatch.
You mentioned in your second post, that you:
Quotesynced the disk with the XMP files with my "master" Disk.
Does this mean that the RAW files already indexed by IMatch were replaced by a copy from your master disk? Thus changing the files IMatch knew about? This could explain why IMatch detected that something changed and had to reread the files. I ask because "syncing" can mean different things regarding sync direction, include/exclude filters and so on.
Thorsten
Win 10 / 64, IMatch 2018, IMA

RobiWan

Quote from: thrinn on December 08, 2017, 07:33:47 AM
Does this mean that the RAW files already indexed by IMatch were replaced by a copy from your master disk?

Not, because RAW files are not changed because the same state on both sides.
The only thing that has changed are the XMP files nothing else.
At the moment I am testing Photo Supreme which gives the user the decision to decide which side is more important (internal database or Files on disk). As is the case with large DAM systems, because mistakes can always happen


Mario

As I said, IMatch will not even detect missing XMP files unless you force a reload or you change the timestamp of the original image.
I wish you the best of luck with PhotoSupreme. Make sure you give it a thorough evaluation.

Quotedecision to decide which side is more important (internal database or Files on disk

IMatch allows you that too (Edit > Preferences > Metadata 2: "Protection"). Press <F1> in that dialog to learn more.

But if you run into a situation where you have two competing XMP records for the same file, you're in trouble either way.
This should never happen. Don't change the same XMP data in multiple applications without synchronizing the data between the applications properly.
You will always lose data otherwise if you replace data written by one application with data written by another application.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

RobiWan

Quote from: Mario on December 08, 2017, 02:29:25 PM
I wish you the best of luck with PhotoSupreme. Make sure you give it a thorough evaluation.

We do not need to be sarcastic. I wrote what I expect from a DAM software no more and no less. That's not all great at PhotoSupreme, I know myself

Quote from: Mario on December 08, 2017, 02:29:25 PM
IMatch allows you that too (Edit > Preferences > Metadata 2: "Protection"). Press <F1> in that dialog to learn more.

Now I have changed 2 Settings
Background Processing - to "OFF"
Protection - both entries I have set to "YES"

That seems to work.
How do I get IMatch to reread the XMP files for selected images from the disk and replace entries in the IMatch database?

Quote from: Mario on December 08, 2017, 02:29:25 PM
But if you run into a situation where you have two competing XMP records for the same file, you're in trouble either way.

Of course and that's why it's important to leave the decision to the user.

I think this is a big misunderstanding. I do not want to do IMatch bad. I know the software not long enough.

Robert

Mario

#15
Quote from: RobiWan on December 08, 2017, 03:30:17 PM
We do not need to be sarcastic. I wrote what I expect from a DAM software no more and no less. That's not all great at PhotoSupreme, I know myself
I was not being sarcastic. I just want to give you a heads up for your evaluation of this other product. I have never used PS and I really don't know much about it.

Make sure you understand these settings and what they mean. It appears to me you are deliberately trying to create different versions/content sets of XMP in multiple applications and then manually decide which one wins in the end. Dangerous. You can lose data easily that way.

QuoteHow do I get IMatch to reread the XMP files for selected images from the disk and replace entries in the IMatch database?

IMatch automatically rescans folders and files when it detects changes in the file system last modified timestamp, or when Windows sends out certain messages.
If you have deliberately disabled this you need to run a manual rescan of the folder us use Shift+Ctrl+F5 to control the scan en-detail for one or more selected files. This is all explained in the IMatch help, as usual.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

sinus

Quote from: RobiWan on December 07, 2017, 11:33:18 PM
Quote from: Mario on December 07, 2017, 09:30:40 PM
You should not try to juggle with different states of XMP data in different applications - especially not in C1.

I think it was a bad example. Here's another example that better represents what's just going wrong here.
What happens if you accidentally delete XMP files on the disk or due to hardware errors they are suddenly faulty? In my experience, the data is also deleted in IMatch database. And sorry but this behavior is certainly not implemented by any serious DAM software and I know some of them.
This has nothing to do with C1, LR or other software.

I tried this here on my system.
And in my case this is not true, what you are saying.

I just deleted one xmp from a raw ... all information is still here.
I could then simply create a new xmp with IMatch.

No problem.
And btw, before I do something with 45.000 images; I do a test with some files.
I have 250.000 files in IMatch and would not dare doing something without testing, except I am very sure.

Hope, you can manage your files again correct.
Best wishes from Switzerland! :-)
Markus

RobiWan

Quote from: sinus on December 08, 2017, 04:41:45 PM
And in my case this is not true, what you are saying.

Of course is it true!
You can't reproduce it, that is a big difference.

By the way - I have tried the same way with only 4 images and I can't reproduce it too, but I know what happened here. But I can not say why it happened.
- Maybe shit happens - I don't know

Quote from: sinus on December 08, 2017, 04:41:45 PM
Hope, you can manage your files again correct.

Everything is fine again right now, thanks to PhotoSupreme. Both applications have advantages but also disadvantages. I will deal with both with the settings.

sinus

Quote from: RobiWan on December 08, 2017, 06:30:24 PM
Quote from: sinus on December 08, 2017, 04:41:45 PM
And in my case this is not true, what you are saying.

Of course is it true!
You can't reproduce it, that is a big difference.

RobiWan, I wrote clearly:
I tried this here on my system.
And in my case this is not true, what you are saying.

8)

Quote from: RobiWan on December 08, 2017, 06:30:24 PM
Quote from: sinus on December 08, 2017, 04:41:45 PM
Hope, you can manage your files again correct.

Everything is fine again right now, thanks to PhotoSupreme. Both applications have advantages but also disadvantages. I will deal with both with the settings.

Fine, glad to hear that.
Best wishes from Switzerland! :-)
Markus