Writing back metadata to raw files without risk?

[grauschaf]

In IM3 writing matadata to raw files was not recommended (hope I remember right  . There was the risk of damaging the raw.
Is this also valid for IM5?

If I do a rating on a raw, the "unwritten metadata" symbol appears on the thumbnail. Is there a way to remove the icon without writing metadata back?

Gerd

[sinus]

In IM3 writing matadata to raw files was not recommended (hope I remember right  . There was the risk of damaging the raw.
Is this also valid for IM5?

If I do a rating on a raw, the "unwritten metadata" symbol appears on the thumbnail. Is there a way to remove the icon without writing metadata back?

Gerd

Hi Gerd
I believe, the risk is also still true. But of course, you will find people, who don't care and have some reasons for this.
And, as always, you can find people, who says the other way round.

In IM3, I wrote it into the raw.
Now (after a long hesitating) I do work with sidecars and hence write such infos into the sidecar (xmp). It gives me also a good feeling, because with the sidecar my raw-editing-work is also stored (unfortunately such informations are proprietary, I work with Lightroom).

Another way would be go with the DNG-line, but I decided against it.

Hope, you will find a good way.

[Mario]

1. Ask yourself, why do you want to write to RAW files?

2. Writing to RAW files is still considered dangerous because the camera vendors don't document their RAW formats, they can change the formats at any time etc. bla-bla.

3. IMatch 5 by default does not write to RAW files.
The exception is when existing EXIF/IPTC/GPS data needs to by synchronized with the XMP data. But that's the only way to not end up with different sets of data.

There is tons of info about all this in the Metadata and Write-back help topics. No need to reiterate all that here.

4. IMatch uses the renowned ExifTool library to safely write to all kinds of files. ExifTool is used by millions of users out there and considered the de-facto standard for reading and writing metadata.

5. If you accidentally change metadata, use the "Edit > undo" command immediately.

6. To throw away changes made to metadata, use the <Shift>+<Ctrl>+<F5> -> Reload Metadata command. This reloads the data from the file and removes the write-back indicator.

[cytochrome]

I do it all the time, with all my NEF and RW2. I did it also with MRW (Minolta A1 raw) and with these I lost some raw when I was experimenting with Exiftool. I suppose it was my fault.

This said, I have a backup, and jpg(s) for all raw I don't dump immediately. And I must admit I have NO raw that is really vital to me. If I was a pro I could decide differently...

Francis

[grauschaf]

Thanks all  for your support within minutes  .

1. Ask yourself, why do you want to write to RAW files?

@Mario: If I set and reset a rating or label, the pen icon will get active. Clicking the pen icon writes metadata back to the (raw) file and removes the pen icon.

6. To throw away changes made to metadata, use the <Shift>+<Ctrl>+<F5> -> Reload Metadata command. This reloads the data from the file and removes the write-back indicator.

@Mario: This is the information I was looking for. Thank you.

I believe, the risk is also still true. But of course, you will find people, who don't care and have some reasons for this.
And, as always, you can find people, who says the other way round.

In IM3, I wrote it into the raw.
Now (after a long hesitating) I do work with sidecars and hence write such infos into the sidecar (xmp). It gives me also a good feeling, because with the sidecar my raw-editing-work is also stored (unfortunately such informations are proprietary, I work with Lightroom).

I use DxO to develop the raws. DxO creates xmp files for each raw with the developing settings. Do you think it might be a problem if one sidecar file is used by different software?

[jch2103]

I use DxO to develop the raws. DxO creates xmp files for each raw with the developing settings. Do you think it might be a problem if one sidecar file is used by different software?

DxO creates its own flavor of sidecar file, .dop, for storing its development settings. It doesn't write to .xmp files (except for certain other metadata like Ratings). DxO works very smoothly with IMatch, and I'm not aware of any conflicts with other software.

[Mario]

XMP metadata was designed to be used by multiple applications. Not at the same time, meaning you don't should edit XMP data for the same file in two applications at the same time. But XMP is the de-facto metadata interchange format.

If your other application produces sidecar files, make sure you configure them as buddy files in Edit > Preferences > File Relations so IMatch can keep them together with the RAW files.

[sinus]

I believe, the risk is also still true. But of course, you will find people, who don't care and have some reasons for this.
And, as always, you can find people, who says the other way round.

In IM3, I wrote it into the raw.
Now (after a long hesitating) I do work with sidecars and hence write such infos into the sidecar (xmp). It gives me also a good feeling, because with the sidecar my raw-editing-work is also stored (unfortunately such informations are proprietary, I work with Lightroom).

I use DxO to develop the raws. DxO creates xmp files for each raw with the developing settings. Do you think it might be a problem if one sidecar file is used by different software?

I use Lightroom for editing RAWs (nefs) and LR and IMatch does use Sidecars (extension .xmp).
Lightroom does write its editing stuff into the xmp (while does not touch the other fields like description or copyright), and IMatch does write all other metadata (city, description, copyright ...) into the xmp-sidecar.

Until now this works great. The only thing, what I am looking for, is, that I do not touch the xmp from both applications in the same time.

This means for me. In IMatch I have deactivated automatic writing-back. Hence when I open LR and work with LR, IMatch does not disturb this writing of LR.

Once LR is finnished, I do close LR and then let IMatch read the changed metadata (Ctrl F5- read metadata).

What is important (I think) is, that you do carefully check your scenario with some files, and if you have found, that the two applications does not disturb each other, then I would make it always in the same manner and order. This means having a consistent workflow.
From time to time you can check, if all is still ok, so you can sleep very well. 

[Mario]

Some points:

IMatch works well with LR, CS and other major XMP-based software.

XMP was not designed by Adobe for concurrent use (multiple applications accessing XMP simultaneously). There are no locking mechanisms and thus two applications which work on the same XMP data simultaneously can wipe each other's changes out.

It is not always obvious when LR writes changes to XMP.
It seems that when you work with XMP in sidecar files (= fast) LR updates the XMP with every change you make in the LR UI - depending on your settings even with every brush stroke (if you use full history).

When IMatch is running at the time, it receives "folder contents changed" messages from Windows and rescans the folder automatically, bringing in the new XMP data, re-creating thumbnail and cache files (because the XMP data may affect how the images look or need to be cropped). If LR changes the XMP file 50 times, IMatch has to do this 50 times (!). But IMatch always waits for a certain amount of time to let things settle before it rescans a folder after a file system notification has been received. This reduces the number of file processing operations considerably and also deals with the "file save method" Adobe uses (write to a TEMP file, delete the original file, rename the TEMP file).

If IMatch is not running at the time you work with LR or CS in folders contained in your IMatch database, IMatch picks up the changes automatically when you start it, rescanning the folders containing new or updated files once.

If you make changes to XMP data in IMatch, LR may not pick them up until you explicitly ask it to reload the XMP data. LR seems not to check file time stamps.
The same can be said for other applications, e.g. Picasa (which does not even have a refresh command, you need to remove and re-add the files) and some other RAW processors. Cooperative behavior is apparently high on the list of many vendors out there. Adobe naturally assumes that you only use their software for all your files 

[grauschaf]

Thank you for your help and invaluable tips.
Improving my workflow and configuring IMatch to my needs will be the main tasks for me. Good to know, that there is no problem with buddy files used by different applications. And very good too, that IMatch covers (nearly) all use cases in a photographic workflow 

Gerd

[sinus]

Good to know, that there is no problem with buddy files used by different applications. And very good too, that IMatch covers (nearly) all use cases in a photographic workflow 
Gerd

Note, that (I believe) buddy files and sidecars is not the same thing.

[Ferdinand]

Note, that (I believe) buddy files and sidecars is not the same thing.

They can be.  The term sidecar is normally used to indicate a small file that a raw converter uses to store its settings.  Usually in the same folder as the raw file, but sometimes in a subfolder.  Sometimes more than one sidecar per raw file and sometimes whole layers of subfolders and sidecar files (Capture One). 

Imatch does not regard a file as a sidecar.  IMatch has buddy files.  When you rename, move, copy, delete a file, Match will do the same to the file's buddy files. 

So you would normally want to have IMatch treat your raw converter's sidecars as buddy files, in order to keep them sync with the raw.

IMatch does this automatically for Adobe sidecars, since they're just xmp files.  You need to define buddy relations for the others. 

[Richard]

Hi Ferdinand,

If I were grading posts, I would have to give you an A+ for that one.

[Ferdinand]

  or perhaps 

Thank you.  Best laugh I've had today!

[sinus]

or perhaps 

Thank you.  Best laugh I've had today!

   

... and finally all is in the help-file:

"Buddy files (sometimes called sidecar or sidekick files) are secondary files created by your camera, scanner or software. This includes files like the JPEG previews your camera produces or the settings files created by some applications.

IMatch uses buddy relations to keep buddy files and their master files together when moving, renaming, copying or deleting files.

XMP sidecar files are special and are automatically handled by IMatch. IMatch automatically keeps XMP files together with their associated master file, you don't need to setup XMP buddy files yourself."

[sinus]

Hi Ferdinand,

If I were grading posts, I would have to give you an A+ for that one.

or even a "Triple-A"