Didn't change default so metadata only in xmp files. How to get into JPG IPTC?

[Fellwalker]

Please can anyone help me overcome my lack of prior knowledge and stupidity?

I've just tagged my latest batch of a few hundred photos taken in RAW (RW2) and JPG, with buddy files hidden, and asked iMatch to process the metadata.  I did not realise then that iMatch 5 by default does not put the metadata into IPTC when you process while showing only the buddy RW2 file.  Up to today I'd only used iMatch5 with JPGs and no RW2.  When I tried to upload the JPGs to Flickr, there were no keywords.  If I try to upload both the JPG and the xmp Flickr tells me that the xmp is a bad file.

xmp is the right way to go for the RW2 file and for the future, but clearly Flickr does not use it yet.

I have changed

Code Select Expand

Edit>Preferences>Metadata2>Configure File Formats for the JPG file type to "Allow create IPTC/EXIF/GPS" set to Yes.  I added a TEST keyword, and

Code Select Expand

Commands>Metadata write-back>For selected files... but the jpg still does not hold any keywords.

I've tried unfiltered view so I can see both the JPG and RW2, copied the RW2 and pasted attributes into the JPG.  Nothing happens - ExifTool output shows Warning: No writable tags set from E:\My Documents\My Pictures\000 Import Folder\101_PANA\P1010758.RW2. Clearly the keywords are in the xmp, which is not visible in iMatch.

Solved that - I have to use the keywords panel, and click the copy icon, move to the jpg and click the paste icon, then go to commands and write the data back.  But it only works one file by one file, which is going to take an inordinate amount of time.

I cannot find an answer in the iMatch help or forums, so seek help.  How do I quickly copy the metadata, that I have spent the morning inputting, from the 130 or so xmp files into the IPTC within the JPG files where I need it for Flickr?

[Richard]

Your post was not a tip or a trick so I moved it.

[cytochrome]

Not sure I understand it right, but if the xmp is in the same folder than the jpg you might try (on a backup) the ECP with something like

-overwrite_original_in_place
-tagsfromfile
{File.PathS}{File.Name}.xmp
%d%f.XMP
{File.PathS}{File.Name}.jpg

I use this to copy xmp into RW2 (obviously the last line is {File.PathS}{File.Name}.rw2 ) It is probably not the most elegant and concise Exiftool command but it works, relatively fast on entire RW2 folders.

Francis

[joel23]

xmp is the right way to go for the RW2 file and for the future, but clearly Flickr does not use it yet.

XMP sidecars are meant for RAW and file types which don't allow embedded EXIF/IPTC. JPG per default allows those and also embedded XMP. No need for sidecars here.

I have changed

Code Select Expand

Edit>Preferences>Metadata2>Configure File Formats for the JPG file type to "Allow create IPTC/EXIF/GPS" set to Yes.  I added a TEST keyword, and

Code Select Expand

Commands>Metadata write-back>For selected files... but the jpg still does not hold any keywords.

But this should work. Add XMP keywords to them and an IPTC record is created and XMP -> IPTC keywords are synced.

I've tried unfiltered view so I can see both the JPG and RW2, copied the RW2 and pasted attributes into the JPG.  Nothing happens - ExifTool output shows Warning: No writable tags set from E:\My Documents\My Pictures\000 Import Folder\101_PANA\P1010758.RW2. Clearly the keywords are in the xmp, which is not visible in iMatch.

They should - why do you believe their XMP is not visible in IMatch? A sidecar don't have to be "visible" - but IMatch has to be configured to use them the right way. Check your Metadata2 settings for RW2 (Default/Force/Favor/Embed) .
Guess you tried to copy metadata which are not in the source file, like IPTC or this was before you allowed to create IPTC for JPGs.

I cannot find an answer in the iMatch help or forums, so seek help.  How do I quickly copy the metadata, that I have spent the morning inputting, from the 130 or so xmp files into the IPTC within the JPG files where I need it for Flickr?

There are several ways, one are the ECP commands cytochrome showed, which copies XMP from the sidecars to the JPGs, as you IMHO want it. From there IPTC is created and synced.
Another way is to use file relation and propagating XMP from RAW (respectively XMP sidecars) to the JPGs. Here as well IPTC records are created when you've configured it.

[Fellwalker]

Thank you to Joel and cytochrome for the ideas.
My solution (or rather what I initially thought was a solution) :
Unfilter buddy files. Select the RW2 AND the JPG of an image. In the Keywords panel, click the greyed out pencil icon next to the new keyword box, so that it becomes yellow signifying that the metadata has been changed and needs to be written back again.  Click commands/ metadata write-back/ for selected files, which transfers the yellow pencil flag to the individual files. A second time and the keywords are written to the database in iMatch for the JPG. Then for a third time commands/ metadata write-back/ for selected files to actually write back the metadata.  I tried this in a test database, then in the live one. Now all my JPG files hold the same keywords as the respective RW2 files. CARE this only works by selecting one pair of files at a time, as it copies ALL keywords from ANY file into ALL files.

Note: In the metadata2 tab of iMatch Preferences, if I leave JPG as default, when working with buddy files the keywords are written to the file but as XMP data. That works in Flickr, Photoshop elements 12 and even windows explorer, but not in Irfanview which I use a lot.  To get the keywords into IPTC where I can see them in all my programmes, I needed to set "Allow create IPTC/EXIF/GPS" to Yes.

xmp is the right way to go for the RW2 file and for the future, but clearly Flickr does not use it yet.

XMP sidecars are meant for RAW and file types which don't allow embedded EXIF/IPTC. JPG per default allows those and also embedded XMP. No need for sidecars here.

As there is an RW2 and a JPG of the same image, which are defined and checked as buddy files, does that not mean the RW2 needs a sidecar file, but the JPG should embed?  A lone JPG does embed into IPTC using the default iMatch settings.

I have changed

Code Select Expand

Edit>Preferences>Metadata2>Configure File Formats for the JPG file type to "Allow create IPTC/EXIF/GPS" set to Yes.  I added a TEST keyword, and

Code Select Expand

Commands>Metadata write-back>For selected files... but the jpg still does not hold any keywords.

But this should work. Add XMP keywords to them and an IPTC record is created and XMP -> IPTC keywords are synced.

Yes, I thought this too, but I have just repeated the excercise on several different pairs of files either as filtered buddies, or as unfiltered with both selected, and got the same result.

I've tried unfiltered view so I can see both the JPG and RW2, copied the RW2 and pasted attributes into the JPG.  Nothing happens - ExifTool output shows Warning: No writable tags set from E:\My Documents\My Pictures\000 Import Folder\101_PANA\P1010758.RW2. Clearly the keywords are in the xmp, which is not visible in iMatch.

They should - why do you believe their XMP is not visible in IMatch? A sidecar don't have to be "visible" - but IMatch has to be configured to use them the right way. Check your Metadata2 settings for RW2 (Default/Force/Favor/Embed) .
Guess you tried to copy metadata which are not in the source file, like IPTC or this was before you allowed to create IPTC for JPGs.

The metadata was in the XMP file. I've tried again. The processing queue is empty, and the RW2 has 3 keywords, but with paste attributes checked against categories and against keywords, they are not pasted into the JPG. I've also tried it with the "All metadata" option checked.
I looked again at the metadata2 tab.  I already had this configured for JPG so that Write IPTC and Write EXIF were "Yes".
For RW2, it is set as default, which means "Allow create IPTC/EXIF/GPS" is set as No. I've tried it set to Yes, it makes no difference to the above.  If I force it to create IPTC it does so only within the RW2.

I cannot find an answer in the iMatch help or forums, so seek help.  How do I quickly copy the metadata, that I have spent the morning inputting, from the 130 or so xmp files into the IPTC within the JPG files where I need it for Flickr?

There are several ways, one are the ECP commands cytochrome showed, which copies XMP from the sidecars to the JPGs, as you IMHO want it. From there IPTC is created and synced.
Another way is to use file relation and propagating XMP from RAW (respectively XMP sidecars) to the JPGs. Here as well IPTC records are created when you've configured it.

Not sure what you mean by this, but if it was what I did as shown at the top of this posting, then great.  Thank you for triggering that line of thought.

My concern for the future is :
My solution ONLY works when I have already got the keywords in the RAW file, and do not have files filtered, and am selecting both the JPG and RW2. It does NOT work when adding a keyword to a filtered buddy pair. 
What settings do I need in iMatch so that when I am only showing filtered buddy files, the keywords and categories are written to BOTH the RAW file 's xmp, and ALSO within the JPG?

[joel23]

I cannot find an answer in the iMatch help or forums, so seek help.  How do I quickly copy the metadata, that I have spent the morning inputting, from the 130 or so xmp files into the IPTC within the JPG files where I need it for Flickr?

There are several ways, one are the ECP commands cytochrome showed, which copies XMP from the sidecars to the JPGs, as you IMHO want it. From there IPTC is created and synced.
Another way is to use file relation and propagating XMP from RAW (respectively XMP sidecars) to the JPGs. Here as well IPTC records are created when you've configured it.

Not sure what you mean by this, but if it was what I did as shown at the top of this posting, then great.  Thank you for triggering that line of thought.

In "File Relation" - > "Detection" you define Buddy Files which are more or less good for keeping files together when moving, renaming, copying or deleting them. That's it.
And you also can define "Versions" in there by which (among other) automatically XMP data and categories can be copied from RAW (Master) to JPG or TIF (Versions).

My concern for the future is :
My solution ONLY works when I have already got the keywords in the RAW file, and do not have files filtered, and am selecting both the JPG and RW2. It does NOT work when adding a keyword to a filtered buddy pair. 
What settings do I need in iMatch so that when I am only showing filtered buddy files, the keywords and categories are written to BOTH the RAW file 's xmp, and ALSO within the JPG?

See above: Versioning.

But I made some tests: manually copying (CTRL-C -> SHIFT-CTRL-V) keywords or XMP data from RAW with a XMP sidecar to JPG should work - but it does not, when the RAW has no IPTC respectively no embedded XMP record.
Warning: No writable tags set from D:\RAW_CANON_450D.CR2
I would call that a bug. 

[Fellwalker]

In "File Relation" - > "Detection" you define Buddy Files which are more or less good for keeping files together when moving, renaming, copying or deleting them. That's it.
And you also can define "Versions" in there by which (among other) automatically XMP data and categories can be copied from RAW (Master) to JPG or TIF (Versions).

Ah-ha!  I had totally misunderstood buddy files.  I had always thought that what you did to one was done to its buddy. That felt to me to be sensible.  It explains why, thinking back to when I used an SLR and took RAW photos, the categories never got put into the JPG files.  I had understood that for categories because they are only within iMatch; but wrongly assumed that Keywords, which are for wide usage, would be propagated.

That said, I cannot get versioning to propagate Keywords (which are a sub-set of Categories).  I have taken 8 new RW2/JPG pairs, and used drag and drop to import into a new folder in iMatch.  I've set up the versioning, and confirmed (see screenshot) that the RAW file DOES see the JPG as its version.

I added a keyword to an RW2 file, and CTRL-ALT-S to process, three times just to be sure it got written to the xmp file.  Then Right Click on the RW2 file: "Versions/Propagate data to versions..." but JPG remains unchanged, with no Keywords.

JPG file format metadata options started as per attached screenshot, then I tried the whole thing again with a different pair of files, but options set to "Embed XMP in file".  No difference, I have gone back to default setting.

I added another keyword, and get this ExifTool Output:

Code Select Expand

-overwrite_original
-m
-use
MWG
-charset
ExifTool={PTETCHARSET}
-ex
-sep


-XMP-lr:HierarchicalSubject=
-XMP-lr:HierarchicalSubject=TEST from rw2 to jpg
-XMP-lr:HierarchicalSubject=TEST2 from rw2
-XMP-dc:Subject=
-XMP-dc:Subject=TEST from rw2 to jpg
-XMP-dc:Subject=TEST2 from rw2
-XMP:CreatorTool=photools.com IMatch 5.1.0.6 (Windows)

-xmp:InstanceID=xmp.iid:ec94764d-514b-4ac2-9d94-61b4a3e3cc9f

-MWG:ModifyDate=now
-XMP:MetadataDate=now
E:\My Documents\My Pictures\_TEST Versioning 2\P1010899.xmp
-execute9999

    1 image files updated

----- Runtime: 0.1 s.WHich suggests that all iMatch is doing is putting it in the xmp file (again).

If instead I add a keyword to one of the JPG files then CTRL-ALT-S, the output is :

Code Select Expand

----- Runtime: 0.0 s.
-overwrite_original
-m
-use
MWG
-charset
ExifTool={PTETCHARSET}
-ex
-tagsfromfile
E:\My Documents\My Pictures\_TEST Versioning 2\P1010900.JPG
-@
C:\Program Files (x86)\photools.com\IMatch5\arg_files\exif2xmp.args
--Exif:rating
-@
C:\Program Files (x86)\photools.com\IMatch5\arg_files\iptc2xmp.args
-@
C:\Program Files (x86)\photools.com\IMatch5\arg_files\gps2xmp.args
-sep


-Composite:Keywords=
-Composite:Keywords=TEST3
-IPTC:Keywords=
-IPTC:Keywords=TEST3
-XMP-lr:HierarchicalSubject=
-XMP-lr:HierarchicalSubject=TEST3
-XMP-dc:Subject=
-XMP-dc:Subject=TEST3
-XMP-xmp:Rating=0
-XMP:CreatorTool=photools.com IMatch 5.1.0.6 (Windows)

-xmp:DocumentID=xmp.did:881838b2-4546-44b4-bd50-0ddd3bd06022

-xmp:OriginalDocumentID=xmp.did:881838b2-4546-44b4-bd50-0ddd3bd06022

-xmp:InstanceID=xmp.iid:a565245e-4727-43b9-b6c2-c7c984dee881

-MWG:ModifyDate=now
-XMP:MetadataDate=now
E:\My Documents\My Pictures\_TEST Versioning 2\P1010900.JPG
-execute
-overwrite_original
-m
-EXIF:ImageDescription=
-EXIF:Software=
-EXIF:Copyright=
-EXIF:Artist=
-IPTC:By-line=
-IPTC:Caption-Abstract=
-IPTC:CopyrightNotice=
-IPTC:Keywords=
-IPTC:ObjectName=
-IPTC:By-lineTitle=
-IPTC:Writer-Editor=
-IPTC:Category=
-IPTC:City=
-IPTC:Country-PrimaryLocationName=
-IPTC:Credit=
-IPTC:DateCreated=
-IPTC:TimeCreated=
-IPTC:Headline=
-IPTC:SpecialInstructions=
-IPTC:Source=
-IPTC:Province-State=
-IPTC:SupplementalCategories=
-IPTC:OriginalTransmissionReference=
-IPTC:Urgency=
-IPTC:Country-PrimaryLocationCode=
-IPTC:Sub-location=
-IPTC:DigitalCreationDate=
-IPTC:DigitalCreationTime=
-GPS:all=
-tagsfromfile
E:\My Documents\My Pictures\_TEST Versioning 2\P1010900.JPG
-@
C:\Program Files (x86)\photools.com\IMatch5\arg_files\xmp2exif.args
-@
C:\Program Files (x86)\photools.com\IMatch5\arg_files\xmp2iptc.args
-@
C:\Program Files (x86)\photools.com\IMatch5\arg_files\xmp2gps.args
--XMP:ImageDescription
E:\My Documents\My Pictures\_TEST Versioning 2\P1010900.JPG
-execute9999

    1 image files updated

    1 image files updated

----- Runtime: 0.8 s.
Warning: Error converting value for ExifIFD:SubSecTimeOriginal (ValueConvInv) - E:\My Documents\My Pictures\_TEST Versioning 2\P1010898.JPG

and that puts the TEST3 keyword in both IPTC and XMP. It does this for other images whether the JPG file format is set to "Embed XMP in file" or "default".

However I try, I cannot put the keywords into one file and have them automatically propagated into the version/buddy. 

Whether I first put the keyword in the RAW file or the JPG, if I then select both, manually mark the files as having their keyword field modified, then (with appreciable delays between each time), update the metadata with CTRL-ALT-S, that is the only way I can find to get the keywords copied into the other file type.WRONG! That puts ALL of the Keywords from ANY of the files with Keywords into ALL of the files selected. Oh how wretched inconvenient. 

Am I doing something wrong?

[attachment deleted by admin]

[joel23]

[...]
That said, I cannot get versioning to propagate Keywords (which are a sub-set of Categories).

I am on the jump so I might be a bit short worded: "Categories" is a proprietary concept within IMatch, but they are not the classical keywords we find in IPTC or hierarchical keywords from XMP (even when they show up in the same tree in the categorie view)

In order to propagate categories you have to check the box in front of the categories you want to propagate (or @All) AND if you want to propagate classical keywords, you need to mark one of the XMP options (like "Don't copy XMP orientation") in the versioning menu. I just made a quick test: it works like expected.
You'd use XMP here because your RAW (XMP sidecars) don't have IPTC keywords.

You should consider reading a bit in the help file.

[Fellwalker]

Thanks Joel

I have read the help file, and find that the categories and Keywords are very confusing.  @Keywords is shown within iMatch Categories, but treated differently.  I know that.  I wish they were in a different hierarchy, as it is all too easy to allocate a keyword to a file when you intended to allocate a category. It makes me realise I need to make a suggestion that in the categories panel there should be a background colour to Keywords to highlight that they are different.

Versioning is new to me, and there is a lot to learn about it.  The help for Propagate / Categories shows in the screenshot that Categories includes @Keywords, whereas in the iMatch in front of me (5.1.6) it does not.  Things change so quickly, and iMatch improves significantly all the time, so I'm not surprised it takes time to update help.

As you suggest, if Versioning is set to included XMP (I chose XMP without LR/Camera Raw data), propagation DOES copy the Keywords from RW2 to JPG, in both IPTC and an new dataset of embedded XMP.  I'd thought I did NOT want raw because the JPG file needs IPTC keywords.  Phew.  And if I put in a new keyword to the RAW file, the propagation is now done automatically.  Yay! 

Jim

[Erik]

Thanks Joel

I have read the help file, and find that the categories and Keywords are very confusing.  @Keywords is shown within iMatch Categories, but treated differently.  I know that.  I wish they were in a different hierarchy, as it is all too easy to allocate a keyword to a file when you intended to allocate a category. It makes me realise I need to make a suggestion that in the categories panel there should be a background colour to Keywords to highlight that they are different.

You can do that on your own... Look at the help file for the category view and category properties.

Quickly:

1. Select the @Keywords category.
2. Look at the Properties panel below the category list.  Select the Color Coding Option and select one of the options.... If you don't want to see the colors on your actual image thumbnails, select the "Color coding active, hide from file window" option.
3. Choose a Text Color or Background Color per your liking.
4. Since @Keywords is data-driven, the sub-categories will get colored, too. 

The result will be that all the keywords categories will be uniquely colored.

[Fellwalker]

Excellent Erik.  I'd realised after I posted that they have a different icon to the normal categories, but this is much better for me.

Thanks as ever to the many people who make this forum helpful by responding to people like me who have just not read far enough in the help file, and obviously searched for the wrong thing.

Note to Mario - I am continually amazed by how much is in iMatch, and how wonderful you have made it. Thanks.