Why IM5 creates XMP files for TIF files ?

[jarraun]

Hi Users,

Every time I copy and paste a TIF file in a different folder or rename the TIF, find that IM5 generates a new XMP file for the new or renamed TIF. I must been missing something obvious but can´t find nothing in the forum. Please, find my settings for Buddy files and Metadata 2 attached.

Javier

[attachment deleted by admin]

[cytochrome]

Hello Jarraum,

- your buddy file definition does not concern tif/tiff  try ^(_*{name})[+\-_]*[0-9|a-z]*\.(jpg|jpeg|tif|tiff|dng|nef\.dxo|nef\.bib|nef\.xmp)$

- According to the help the default file definition for Tif/tiff is to embed xmp into the image file, so should not create an xmp sidecar.

- and be aware that f tif  and tiff are different file extension so different settings...

Francis

PS I made a quick test (don't use tif often) I moved a tif and there was no xmp side car created, and all my metadata from the original NEF are in the moved tif file.

Francis

[jarraun]

Thank you Francis,

In my workflow TIF files are not NEF Buddy Files, I don´t want them to be moved, copied or renamed along with the NEF file. I cannot imagine why this can happen as you can see my Buddy File settings are almost as default except for the this link expression:

Default: ^(_*{name})[+\-_]*[0-9|a-z]*\.(nks|jpg|jpeg|dng|nef\.dxo|nef\.bib|nef\.dop)$
Actual:   ^{name}\.(nks|jpg|jpeg|dng|nef\.dxo|nef\.bib|nef\.dop)$

as I don´t want file names with sufixes ( name_v1.jpg, name_web.jpg ) trated as Nef Buddy files.

Salud

Javier

[jarraun]

More findings. It is something about TIF files in the same folder as RAW files (NEF files in this case).

Test:
Buddy files and Metadata 2 Default settings.

• Create a new folder.
• Copy a TIF file and paste in the new folder -> OK, no XMP file generated.
• Copy the corresponding NEF file witth the very same name as the TIF file and also paste to the new folder -> OK, XMP file generated.
• Delete NEF file -> NOT OK, xmp file not deleted.
• Rename TIF file, say "name_v1" -> NOT OK, a new XMP file "name_v1.xmp" appears.

Please could some of you test this, because if this repeats could be a bug.

Jvier

[thrinn]

Delete NEF file -> NOT OK, xmp file not deleted.

A xmp file belongs by definition to all files with the same name. You might have a look at https://www.photools.com/community/index.php?topic=2753.msg17864#msg17864. I think ubacher had a similar problem as you.

[ovrevid]

More findings. It is something about TIF files in the same folder as RAW files (NEF files in this case).

See https://www.photools.com/community/index.php?topic=2793.msg18157#msg18157 for a discussion about this issue.

[jarraun]

Thank you both, Thorsten and Ovrevid,

As Mario stated:

By definition, an XMP file belongs to all files with the same name in the same folder. So if you have

bla.nef
bla.xmp
bla.jpg

IMatch does not delete the XMP because it is still linked to the jpg file.

As Erik says in his post:

This issue seems to becoming common (or at least it's been posted about a couple of times).

Richard covered why the problem occurs, but as a note this conflict with XMP files that happens when two files have the same name but different extensions can be dangerous to a user in cases such as this.

As mentioned in the original post, you end up with an XMP file that is only tied to a JPG, but JPGs rarely have XMP files, unless a user specifically sets it that way. There are many other conditions where XMP files could conflict if you were purposely only working with sidecars (such as orientation, develop settings, etc).

This is one of the reasons that I chose to make version files have unique names from the master files. 

Instead of having:

File1.raw
File1.jpg
File1.xmp

I use:

File1.raw
File1-v1.jpg
File1.xmp

The result is that when I delete or move the Raw file, the XMP is now only tied to the master file.  If the version were to have its own sidecar file

File1-v1.xmp

Then that file is unique to the version.  The added advantage is that I can easily track and manage multiple versions, but the primary reason I had started adding a suffix to the original file name for versions was to avoid conflicting xmp records, which was an issue for me back in IM3.6. 


--Anyway, I can see possible value to the request, but I can also see it getting quite complicated in cases where both similar named files legitimately need the XMP and having to keep track of the Metadata Settings closely to know when to delete an XMP file or not.  Ultimately, the issue can be remedied by a user with a bit of planning on their file naming convention.

I must rename my TIF files with a suffix to reach my goal.

Javier

[Ferdinand]

Or just don't ever put NEF & TIF files in the same folder.

[Mario]

Incidentally, I had a related enhancement on my to-do list for the 5.1.12 release.

Since IMatch now has a 'database' of which file formats take embedded XMP metadata (or for which file formats the user explicitly forces IMatch to use external XMP metadata) I thought I can use this data to handle XMP files a bit smarter when moving/copying/renaming/deleting files.

So far the "XMP belongs to all files with the same name in the folder" was always enforced. If you have a folder with a A.RAW A.XMP and A.JPG and you copy the A.JPG to another folder, IMatch always copies the XMP as well. The same rule is applied when you delete the A.RAW. Since there is still a A.JPG in the same folder, the A.XMP is not deleted. When the A.RAW is renamed, IMatch actually duplicates the XMP file so we end up with NEW.RAW NEW.XMP A.JPG and A.XMP.

For IMatch 5.1.12 I have changed this behavior. IMatch now uses the per-file format XMP settings database to determine if a file that is copied, moved, renamed or deleted actually requires an external XMP file. If this is not the case, existing XMP files are ignored. If you now copy the JPEG to another folder, the XMP file is not copied. If you rename the JPEG and XMP file with the same name is not renamed.

If a user enforces IMatch to use XMP sidecar files also for JPEG files (not recommended) the old behavior kicks in and IMatch keeps the JPEG and the XMP together as before.

I'm, not sure if I should just implement this, or add (yet another) option to allow users to disable this new behavior. 

[Carlo Didier]

One way to avoid the "one xmp for all" problem if you want to keep derived version in the same folder than the raw files is to simply change the names of derived files, for example by adding a suffix like "_a", "_b", ... to their name.

[jarraun]

Carlo,

You are right and I´m going to follow your suggestion, only remember that if someone renames his derived files (until Mario decides to implement the new behaviour) some XMP files will be produced and must be deleted. And before he had to have changed the default link expression for buddy files from:

Default: ^(_*{name})[+\-_]*[0-9|a-z]*\.(nks|jpg|jpeg|dng|nef\.dxo|nef\.bib|nef\.dop)$

to:

Actual:   ^{name}\.(nks|jpg|jpeg|dng|nef\.dxo|nef\.bib|nef\.dop)$

Please correct me if I´am wrong.

Mario,

I think the new implementation wold be very useful for people who have decide to keep master an derived files together.

[Ferdinand]

Once 5.1.12 is out, it seems to me that there is a legacy issue of how to find and deal with unwanted XMP files from past copies and moves of files.  As it happens, I have a script ... but I'm fairly sure that it's an IMatch 3.6 script ... but it shouldn't be hard to port.  Before I attempt this, is there any native functionality in IMatch 5 that would make this script redundant?  I ask because every time I think of a script to do something, I then find that it's not needed in IMatch 5 any more.

[Carlo Didier]

... remember that if someone renames his derived files (until Mario decides to implement the new behaviour) some XMP files will be produced and must be deleted.

So far, I have never seen any such "unwanted" xmp files on my system, but I haven't defined any automatic relations yet. We'll see what happens when I do.

[cytochrome]

There was already a discussion on removing unwanted xmp files here; https://www.photools.com/community/index.php?topic=2793.msg18599#msg18599

I posted a (slightly awkward) solution.

Francis

[Erik]

I have to say that I was under the impression that XMP files were essentially buddy files beyond the definitions presented in the File Relationships configuration.

For instance, if a user had no file relations set up (buddy files or versions), the XMP files were still bound to their "parent" files (with all the limitations and behaviors that have been described here and elsewhere).

If my understanding and recollection are right (and I don't have the ability to look at the help file at the moment, which I'm prefer clearly lays this out), would an XMP file show up as a buddy file through the scripting engine.

In other words, if you utilize the sample script that shows all the related files for a selected image, would it show an XMP file in its output?

If you attempt to find the master for an XMP file that is tied to two original files, would it show both files?

I don't really have XMP files to test, and my guess is that any script is going to have to work outside of the file relation classes and work directly with the files and file names to find orphaned files be it through a method such as Francis referred to from the linked thread or otherwise.