IPTC Keywords

[jack@vankinsbergen.com]

I have  for years used an iMatch 3.6 provided script to copy categories to jpg IPTC Keywords.  How can I do the same thing in iMatch 5?

In other words, I want to assign a batch of jpg images to selected categories, and when done batch process a conversion of these categories into IPTC keyword fields.   

I would be happy if I could get the original script converted but In the attempt , I have worked past a couple of problems but have now encountered an issue with:

    Dim iptc As New IPTCStream - what is the replacement for the IPTCStream object  in iMatch5?

Thank you for your help.

Jack


' This script transfers IMatch categories into IPTC categories
Option Explicit

Sub Main

    Dim db As Database
    'Set db = Application.ActiveDatabase
    Set db = Application.Database

   Dim i As File
   Dim iset As Files
   'Set iset = db.ActiveSelection
   Set iset = Application.GetFileWindow(imfwtCurrent).SelectedFiles


   Dim ListArray(1) As String
   Dim seltype As Integer
   Dim fieldname As String
   ListArray(0) = "Update Categories"
   ListArray(1) = "Update Keywords"

   Begin Dialog UserDialog 580,196,"Transfer IMatch Categories to IPTC" ' %GRID:10,7,1,1
      Text 10,7,560,56,"This script copies your IMatch categories into the IPTC category or keyword fields stored in the image. It uses a 'dot-notation' to transfer the complete IMatch hierarchy information to your files. This enables you to restore your IMatch categories from the IPTC record whenever needed.",.Text1
      CheckBox 20,105,470,14,"Keep existing IPTC categories / keywords",.CheckKeepExisting
      Text 40,119,530,21,"Check this option if you want to keep existing IPTC categories or keywords",.Text2
      OKButton 370,161,90,21
      CancelButton 470,161,90,21
      DropListBox 20,70,240,91,ListArray(),.DropListBoxType
   End Dialog
   Dim dlg As UserDialog
   'If Dialog(dlg) <> -1 Then
   '   Exit Sub
   'End If

   'If dlg.DropListBoxType = 0 Then
   '   fieldname = "Supplemental Category"
   'Else
      fieldname = "Keywords"
   'End If

   Dim iptc As New IPTCStream
   Dim res As IMIPTCResult
   Dim c As Category
   Dim cats() As Variant
   Dim ci As Integer
   Dim newcats As New IMDictionary

   Set iptc = New IPTCStream
   For Each i In iset

      If i.Categories.Count > 0 Then
         ' Get the existing IPTC record to check if the file supports IPTC
         ' and if it already has an IPTC record
         iptc.KeepFileDateTime = True
         res = iptc.ReadFile(i.FileName)
         If res = imiptcNoIPTCInfo Or res = imiptcSuccess Then

            ' Build an array in cats holding all categories of the current image
            ReDim cats(i.Categories.Count-1)
            ci = 0
            For Each c In i.Categories
               cats(ci) = c.FullName
               ci = ci + 1
            Next c

            newcats.Clear

            If dlg.CheckKeepExisting Then
               ' Get the existing categories from the file and merge them
               ' with the categories from the image to avoid duplicate entries
               ' We use a dictionary object to get only a single entry for each
               ' keyword

               ' Get existing categories
               Dim iptccats As Variant
               iptccats = iptc.Fields(fieldname)
               For ci = LBound(iptccats) To UBound(iptccats)
                  newcats.Item(CStr(iptccats(ci))) = 1
               Next ci

               ' Add the categories from IMatch, but avoid duplicates
               For ci = LBound(cats) To UBound(cats)
                  newcats.Item(CStr(cats(ci))) = 1
               Next ci

               ' Update the record in the file
               iptc.Fields(fieldname) = newcats.Keys
               iptc.WriteFile i.FileName

            Else
               ' Build a new categories array
               ReDim iptccats(UBound(cats))
               For ci = LBound(cats) To UBound(cats)
                  newcats.Item(CStr(cats(ci))) = 1
               Next ci

               iptc.Fields.Item(fieldname) = newcats.Keys
               iptc.WriteFile i.FileName
               db.RefreshImage i,False,True,True
            End If
         End If
      End If
   Next i

   Set newcats = Nothing
   Set iptc = Nothing

End Sub

[thrinn]

Citing from  the help topic on deprecated and retired methods: Use the Metadata classes instead.
But they are not a direct replacement, I would expect that parts of your script have to be rewritten.
If your goal is to mirror some of your categories in your files, one could think about using Keywords instead of native IMatch categories.

[jack@vankinsbergen.com]

       I have almost 150,000 images under management by iMatch 3.6.   They are organized by categories.  The workflow process includes writing the categories out to the jpg files as keywords so that the images carry such information in its metadata. 

       The reason for all this is:  The process I went through to migrate from my prior database when the vendor disappeared, was such a mess that I decided to create such data in the image itself in what I thought was an industry standard protocol.    The reason being that maybe, just maybe, the next vendor after Photools might be able to use the keywords in a conversion process.  Hopefully I never have to fall back on this.  But never say never!

      I am new to iMatch5 but get the impression that my "industry standard protocol" has been made obsolescent and I can't do what I have been doing in a transparent manner.  Am I to understand that:

     1. the keywords in the IPTC data for my 150,000 images are obsolete?
     2. I must learn a new object model and rewrite my script accordingly?
     3. the new saved structure would be incompatible with the history in the existing 150,000 images?
     4. to get a consistent history I would have to go back and change the existing 150,000 images?

    Sorry for all this, but I appreciate any insight you can provide.  If it is just a question of rewriting the script with the new Metadata classes and producing the same result as iMatch 3.6, then no problem.  I have 55 years of computer system development experience, and as time goes by I just have gotten more tired of these conversions.  If it is the mess that I am concerned about, I will probably just forget it and hope never to leave iMatch.    I am 77 years old, so the chances are that I will disappear before iMatch does anyway!  :>) :>)

Best Regards,

Jack

[JohnZeman]

I have an IMatch 3 script that did what yours does but after researching this same issue I retired it in IMatch 5 and started using @Keyword categories instead of regular categories.  Why?  Because IMatch 5 automatically writes those @keyword categories to each jpg file and stores them in the

Code Select Expand

{File.MD.XMP::Lightroom\hierarchicalSubject\HierarchicalSubject\0}

tag which replaces IPTC keywords.

As to your 150,000 existing images ExifTool should automatically copy them to the above Hierarchical Keywords tag for you, I know it did mine.

The one downside to doing it the way I do now, and it's a minor one, is after making any category changes I always have to writeback back the metadata (Menu > Commands > Metadata Writeback) so IMatch updates each image's affected @Keyword categories in each jpg file.

If you still want to do this with a script post again and I'll see if I can give you some guidance.

[jack@vankinsbergen.com]

John:

   Thank you. I will have to think about that.

Best Regards,

Jack

[Ferdinand]

@Keywords are certainly the best way to do this and are a major advance over the scripted approach in V3.6

The issue is how to migrate your categories to keywords.  Officially the best way to do his is using a metadata template.  Unofficially I have a script that does this and provides a few more options about how this is done.  My script could be used to migrate categories to keywords.  You could use either the script or a metadata template to continue to use the sort of workflow you used in V3.6, writing categories to keywords as needed, but in my view that would make for redundant metadata, as you would have the same items in both categories and @Keywords, with the potential for confusion.

[JohnZeman]

The issue is how to migrate your categories to keywords.  Officially the best way to do his is using a metadata template.  Unofficially I have a script that does this

Unofficially I simply used copy and paste to copy my categories into @Keywords.  You can't paste directly into the @Keywords parent but if you create an empty subcategory under @Keywords you can paste into that.

in my view that would make for redundant metadata, as you would have the same items in both categories and @Keywords, with the potential for confusion.

This was true for me before I switched to @Keyword categories.  I essentially had duplicates, one set in normal direct categories and an identical set under @Keywords.  That's when I decided to eliminate the direct categories and work only with @Keywords.  That was over a year ago and I haven't regretted it since.

[jack@vankinsbergen.com]

Ferdinand:

     I have already found your wonderful script yesterday, but I have been having ongoing general EXIFTool issues.  Finally resolved it, I think.  On my main system, EXIFTool seems to have to be installed on the C: drive to function.  I Have an SSD for my apps and iMatch is installed in  "E:\Win 8\Graphics\iMatch5".  I put a copy of EXIFTool in "C:\Program Files" and pointed your script at it and it seems to run fine. I have now installed iMatch5 in "C:\Program Files" as I don't see how to point iMatch5 to the copied EXIFTool.  It looks like my EXIFTool issues may be solved, but we will see. I am not happy about all this, but there it is.

     Now that it runs, I have to work with the script to see how close it is to what I need.

    Thank you.

Best Regards

[Mario]

As I said in my last post, most likely this is a security issue on your system. You deliberately installed IMatch into a different folder than the folder reserved for that purpose by the operating system. And this can cause all kinds of issues, from Windows protection to virus checkers. IMatch expects the exiftool.exe in the same folder in which IMatch5.exe is installed. This is required to prevent problems on systems where users run multiple versions of ExifTool, e.g. users with GeoSetter or ExifToolGUI etc.

[Ferdinand]

Maybe you should try using a metadata template and see if that does what you want.  My script will allow you to me more selective (I think), but you may not need this.

(I wrote the V3.6 version of this script while V5 was still in Alpha and not safe to be used on production data, as it enabled us to prepare for V5.  It's not really needed any more, but it's an option that some people may be more comfortable/familiar with.)

[lanerellis]

Unofficially I simply used copy and paste to copy my categories into @Keywords.  You can't paste directly into the @Keywords parent but if you create an empty subcategory under @Keywords you can paste into that.

in my view that would make for redundant metadata, as you would have the same items in both categories and @Keywords, with the potential for confusion.

This was true for me before I switched to @Keyword categories.  I essentially had duplicates, one set in normal direct categories and an identical set under @Keywords.  That's when I decided to eliminate the direct categories and work only with @Keywords.  That was over a year ago and I haven't regretted it since.

Hi JohnZeman and all,

John, I've used your method to duplicate my regular categories under @Keywords in the current latest version of IMatch 5.

At the moment I have the duplicates you mention, with each file listed under both regular and @Keywords categories.

If I now delete the regular categories, will my files somehow magically become full-fledged @Keywords files, and have their @Keywords categories written to the files as metadata?

You may recall some time ago in a series of postings here I discussed my options for moving from an IMatch 3 regular-category-only workflow in which no metadata was ever written to my files, into an IMatch 5 workflow that will write my categories into file metadata.

My goal seems simple and straightforward, but I can't yet figure out step three here:

1. My 90,000 files are categorized in regular internal IMatch categories
2. I've used your method to duplicate my internal categories under @Keywords
3. ? Mystery step... ?
4. @Keywords successfully writing metadata to my files

How does the IMatch 5 user like me who has only used internal categories switch to using the powerful @Keywords functionality?

When I duplicated my regular category tree under @Keywords using the methods you've used, John, I had hoped that my files would be assigned to those new @Keywords categories, but perhaps I'm missing a step.

I thought perhaps there was pending metadata write-back issue, but after multiple tries and various refreshes, none of the categories my files have already been assigned to are showing up as keywords in the file -- no metadata is yet being written.

Could you explain how you migrated from regular categories only to successfully using the @Keywords category?

Thank you.

Cheers,
Lane

[Ferdinand]

Lane, it sounds like you've done it.  I don't think there is a mystery step 3 for this to work.

@Keywords reflect what is actually contained in your files, provided that you've one all the requisite writebacks and hence none of these files have pending writebacks.  @Keywords is in effect a data-driven category based on the contents of your files (or sidecars).  You can confirm this by examining the files externally in something like ExifToolGUI or even in the ExifTool Command Processor using "All Keywords" preset.

If there is a mystery step now, it is simply to delete the categories that you have migrated to @Keywords to remove the duplication.  What I did, and what I would suggest you do as a precaution, is export the categories together with file assignments as a precautionary backup before you delete the categories.  You can do this using the Import and Export Panel.

From now on you simply assign keywords (rather than categories) and perform the writebacks and you will see the keywords under @Keywords and they will be inside your files.  When you assign a keyword you are instructing IMatch to write the keyword to the file and IMatch then reads what's in the file and displays it under @Keywords.

[Canay]

I *think* I love the idea of this. But just brain storming here. My @Keywords list is a conglomerate of every keyword I've ever used and is not nicely organized as my categories are. I need to think about how to handle keywords that are not things I use as categories. For example peoples names. I have a category called "my friends". But in the keywords I have added names of people. Would I just manually make all the existing @Keyword names a sub-keyword of "my friends"? Can you explain how you copied all your categories to the @Keywords. Would the list then be co-mingled with all the @keywords already there?

What I have been doing is adding my category, then using a metadata template to add that category into the keywords, then using the keyword panel to make any changes to the keywords I want, in my example to add the persons name.

Just to confirm, you are suggesting to eliminate the categories and just use the @Keywords like we used to use categories. Your system sounds much more streamlined. I just want to make sure I implement it properly and don't mess up years of organization.

[JohnZeman]

Lane I agree with Ferdinand, you should pretty much be there.  The one thing about moving your categories to @Keywords is you have to keep your data-driven (@Keywords) refreshed to see your newest category assignments.

Canay, I don't think you want to try to combine your regular keywords with categories that you might move to @Keywords, at least not before you give it a lot of thought.  Using @Keywords for categories works for me because I never use regular keywords.  In IMatch 3 I used a script to write my category assignments to keywords as a way to back up my categories to each file.

Anyway, in your situation I'd start out with a small test database with a few test images that already have keywords so you can work out the kinks of how you want your situation to work.

[lanerellis]

Hi Ferdinand and John,

Thanks for your input.

I've exported my categories and file assignments to have a backup as you recommended, Ferdinand. Now I suppose I'll try deleting my regular non-@Keywords categories, so I'll be left only with the duplicate set I now have under @Keywords.

Using several methods I see that none of my files have had metadata written to them yet, even after duplicating my regular categories under @Keywords. In my keywords panel no keywords show up for any file, and it notes that "The thesaurus contains no entries." The various metadata panel sections also show keywords to be empty. I used the ExifTool Command Processor as you suggested and see no keywords in my files there, and also used another method to test and found no keywords either.

To make sure there aren't any pending write-backs I've tried Command/Metadata Write-back for both individual files and all pending files. I've run database diagnostics and "compact and optimize" just to be thorough, and also checked my metadata and metadata2 preferences (under Keyword Export "write hierarchical keywords is checked". I've also tried refreshing the @All and @Keywords categories, and any other refresh options I could find.

In the category panel under "Current" when I activate the "show or hide @Keywords categories" to show all the categories a file is assigned to, it shows as being assigned to both my regular categories and the duplicate set I made under @Keywords, with seems to be what I want, yet no metadata has yet been written to my files. Perhaps once I delete my regular categories and am left with only @Keywords categories that will trigger write-back? That doesn't sound likely, but I can give it a try.

As far as I know I don't have any data-driven categories to worry about.

I have a sense of doom and fear that this is all caused by my files never having had any metadata written to them, since I've used only IMatch's excellent regular categories until this attempt to start using a @Keywords workflow.

I'm a bit stumped and perplexed at the moment, and have spent many hours researching this that I had hoped I could use organizing thousands of new photos in my queue. Any more help or tips are greatly appreciated. Thanks! :-)

Cheers,
Lane

[gunda]

No doubt the other more experienced users will respond shortly, but I for one would NOT delete the categories if you can't see any sign of the keywords in your files.  What you describe is most odd, i.e. you have entries under @Keywords and no pending writebacks, but you can't find the keywords in the files.  Either you're not looking in the right place or they're not there.  I would exercise caution and be patient at this point until you have resolved this paradox.

[lanerellis]

Hi Gunda,

Thanks for your message. I'll hold off a bit before deleting my regular categories in hopes that others will have other suggestions to try.

[JohnZeman]

As far as I know I don't have any data-driven categories to worry about.

@Keywords are data-driven categories Lane.  Have you refreshed them?

[lanerellis]

As far as I know I don't have any data-driven categories to worry about.

@Keywords are data-driven categories Lane.  Have you refreshed them?

Thanks, John. I should have been clearer ; I don't believe that I have any data-driven categories among the regular categories I've used over the years.

And as I mentioned in my earlier post, I have tried refreshing, to no avail:

I've also tried refreshing the @All and @Keywords categories, and any other refresh options I could find.

[Ferdinand]

I've exported my categories and file assignments to have a backup as you recommended, Ferdinand. Now I suppose I'll try deleting my regular non-@Keywords categories, so I'll be left only with the duplicate set I now have under @Keywords.

Using several methods I see that none of my files have had metadata written to them yet, even after duplicating my regular categories under @Keywords. In my keywords panel no keywords show up for any file, and it notes that "The thesaurus contains no entries." The various metadata panel sections also show keywords to be empty. I used the ExifTool Command Processor as you suggested and see no keywords in my files there, and also used another method to test and found no keywords either.

To make sure there aren't any pending write-backs I've tried Command/Metadata Write-back for both individual files and all pending files. I've run database diagnostics and "compact and optimize" just to be thorough, and also checked my metadata and metadata2 preferences (under Keyword Export "write hierarchical keywords is checked". I've also tried refreshing the @All and @Keywords categories, and any other refresh options I could find.

In the category panel under "Current" when I activate the "show or hide @Keywords categories" to show all the categories a file is assigned to, it shows as being assigned to both my regular categories and the duplicate set I made under @Keywords, with seems to be what I want, yet no metadata has yet been written to my files. Perhaps once I delete my regular categories and am left with only @Keywords categories that will trigger write-back? That doesn't sound likely, but I can give it a try.

As others have said, this is all rather odd and I don't understand what's going on either.  Something is not right.  @Keywords shows the keywords contained in the file, but the metadata panel says that there aren't any.  Please don't delete your categories just yet.

To may any sense of this we need more information. 

Are you able to post a small image which appears in @Keywords as being assigned to a keyword, but which the metadata panel shows as having no keywords?

Can you post a screen grab of your @Keywords list?  I.e. the top left corner of the category view?  Please do so with the category count turned on.  Perhaps with one of the nodes expanded, if that's possible.  If you can manage a screen grab with the example image shown that would be good as well.

Can you also post a screen grab of the metadata panel showing that this example image has no keywords.

The thesaurus is different.  It is not populated by default.   You either populate it by hand (tedious), or you populate it from the existing data in your files (easy).  Let's get the existence of the keywords sorted out first and then we can worry about populating the thesaurus.

[Ferdinand]

Can you explain how you copied all your categories to the @Keywords. Would the list then be co-mingled with all the @keywords already there?

What I have been doing is adding my category, then using a metadata template to add that category into the keywords, then using the keyword panel to make any changes to the keywords I want, in my example to add the persons name.

Just to confirm, you are suggesting to eliminate the categories and just use the @Keywords like we used to use categories. Your system sounds much more streamlined. I just want to make sure I implement it properly and don't mess up years of organization.

Several comments.  I'm not advocating migrating all your categories to keywords, only those that you want embedded in your files.  So I migrated things like "Who" and "Where", but not categories which indicated workflow, sales, version type, etc.  I haven't even migrated "What" because it's in a state of flux and prone to reorganisation, which gets hard once the hierarchy is embedded in the files.  My advice is to be selective and give some thought to what makes sense to migrate and embed.

What I did was wipe all existing embedded keywords before I migrated categories to keywords.  I have been helping my brother migrate to V5 and we've done the same thing in his DB as well.  In theory you could do this using the delete all keywords option in the keywords panel.  Sometimes I've found it easier to use the ExifTool Command Processor and wipe all keywords that way, and then get IMatch to recognise that they're gone by forcing a reload of the metadata (CRTL-SHFT-F5).  There is another thread running here at the moment about issues with deleting keywords, and the ECP seems to work in such situations. 

In terms of the writing out, have a look at this post and this thread:
https://www.photools.com/community/index.php?topic=3232.msg21440#msg21440
A lot of this post was about writing the correct flat keywords at the same time to avoid multiple rounds of writebacks.  If you wait for V5.2.6 then Mario has fixed this and if you write a hierarchical keyword using a metadata template like the ones I listed in that post, then IMatch will look at your metadata preferences and write the correct flat keywords automatically.

If you want a nice structure to your @Keywords, think about adding things like "Who"," Where", "What" as parent categories for your existing category branches.

(I used a script to do all this wiping and migration but it's no longer necessary.)

[Canay]

Hi Ferdinand, what you said makes sense. I realized yesterday, that I needed to take what bits of wisdom I find on the forum and adapt it to my own unique situation. So I had to ask myself why I started using IMatch and categorizing all my photos and why I started writing those categories to the metadata. Once I answered those questions, I came up with your first paragraph. It does not have to be an all or nothing approach. IMatch 5 has caused me to rethink my system a bit and clean some things up. I have a feeling that the category/@keyword issue will evolve for me over time. I'll check out your thread and gleam a bit more info. Thanks for taking the time to respond. I spent a lot of time with V5 last week and I find I need to go to help and re-read posts in the forum often. I would not be able to use the program efficiently if it weren't for everyone's help.

[lanerellis]

Are you able to post a small image which appears in @Keywords as being assigned to a keyword, but which the metadata panel shows as having no keywords?

Can you post a screen grab of your @Keywords list?  I.e. the top left corner of the category view?  Please do so with the category count turned on.  Perhaps with one of the nodes expanded, if that's possible.  If you can manage a screen grab with the example image shown that would be good as well.

Can you also post a screen grab of the metadata panel showing that this example image has no keywords.

Hi Ferdinand, and thanks for your time. Attached is a wide screenshot showing the items you mentioned. In the image I've also included a snapshot of the categories panel "current" tab for the selected image, showing the image with checkmarks next to both the @Keywords and regular categories.

Perhaps I'm missing something that you'll be able to see in my screenshot. Thanks again, Ferdinand! :-)

Cheers,
Lane

[attachment deleted by admin]

[Ferdinand]

I see your problem.  I really don't understand.

Can I ask that you download the custom metadata layout attached to this post:
     https://www.photools.com/community/index.php?topic=2380.msg15324#msg15324

You'll need to unpack it into C:\ProgramData\photools.com\IMatch5\presets then open the metadata edit layouts dialog and import (merge) it.  Can you then post a screen grab of what it shows for that image. 

If you can post an image that exhibits that behaviour it may also shed some light.  At least then we can see what is really in the file, and work out whether @Keywords or the metadata panel is right.  If you do then make sure you zip the jpg because the forum strips out the metadata that we're trying to assess.

[lanerellis]

Hi Ferdinand,

I'm away from my IMatch computer for a few days now, so I'll grab a screenshot using the layout you mentioned when I get back, and post a small sample image as well.

The situation showing in my screenshot occurs with every file I've checked from my roughly 90,000 file database (in testing this I've checked probably 100 random images and all have the same issue), and with neither IMatch, ExifTool, or the other test method I used showing any keyword metadata in my files, it would seem to me that the metadata panel is correct while the @Keywords assignments somehow are now.

Thanks again, Ferdinand.

[JohnZeman]

Lane I'm curious to know what the variable is in your screen shot that shows empty keywords.  If you right click over the empty keywords in your metadata panel and choose Copy as variable you can paste it into your reply here.  I realize you're away from your IMatch machine now, just throwing this out for consideration.

[Ferdinand]

You would think that since this is the default layout that it would be the correct field, but you never know.  That's why I asked him to install a known (to me) metadata layout.  What is more worrying / puzzling is that the ECP didn't show any keywords.

[lanerellis]

I'm using only the default stock unaltered layouts.

[Ferdinand]

I assumed that was the case, but perhaps when you're back at your PC you could try my and John's suggestions and see what they tell us.  At the moment we are all perplexed, and are grasping around for a clue.

I do think we need to see an image or two that exhibits this behaviour.  It's the only way we can be sure which part of IMatch is really telling us what's in the files.

It may also be useful for you tell us step-by-step precisely how you did the migration.  You said that you followed our instructions, but there may have been something that you omitted to do, although for the life of me I can't imagine what.

I suspect also that what we're going to have to do is take one image as an example and get you to try various things.  I will design some steps.

[lanerellis]

Hi Ferdinand and John,

Ferdinand, I've attached a screenshot using your custom metadata layout.

I've also attached a ZIP file of the example image.

I'm pretty certain there is no keyword metadata in the file, with IMatch showing none in either its default or your custom layout, nor ExifTool, and I've even looked using my old school hex editor. So I'd say the metadata panel is correct and @Keywords is showing categories not written in my files.

John, a "copy as variable" from the empty keywords field of Ferdinand's custom metadata panel produces this:

{File.MD.IPTC::ApplicationRecord\25\Keywords\0}

My migration process from IMatch 3 to 5:

1. Used Mario's conversion utility
2. Used John's method to copy my regular categories to @Keywords, as outlined here:

There is a much simpler way.

1.  Left click the parent category of your regular IMatch categories, then right click > clipboard > copy category
2.  Left click @keywords to select it then create a subcategory under it.
3.  Left click that newly created @keywords subcategory then right click and choose clipboard > paste category.

Clean up by drag and dropping as necessary then once you're satisfied delete your regular categories.

As I mentioned the other day, "I have a sense of doom and fear that this is all caused by my files never having had any metadata written to them, since I've used only IMatch's excellent regular categories until this attempt to start using a @Keywords workflow."

Let me know what you think after looking at the new screenshot and examining the attached image, and considering my migration process.

I appreciate your time and help, Ferdinand and John.

By the way, when I imported and merged the custom F P metadata layout, it seems to have replaced all other metadata layouts, and despite searching thoroughly I can't find the original .immdl files to restore. I only have the F P layout now. How can I get the detault layouts back?

Cheers,
Lane

[attachment deleted by admin]

[Ferdinand]

OK, I'm starting to understand what is going on.  I don't think John's Zenman's method works, at least not in the latest version of IMatch.  It may have in earlier versions, but as I don't recall trying it I can't be sure.

By the way, when I imported and merged the custom F P metadata layout, it seems to have replaced all other metadata layouts, and despite searching thoroughly I can't find the original .immdl files to restore. I only have the F P layout now. How can I get the detault layouts back?

The file you want is C:\ProgramData\photools.com\IMatch5\presets\system.immdl and this is the folder for storing such things, and where you'll find the system defaults.

You weren't offered both a merge option and a replace option when you imported my layout?  The trick is to use the merge option.

I've also attached a ZIP file of the example image.

I'm pretty certain there is no keyword metadata in the file, with IMatch showing none in either its default or your custom layout, nor ExifTool, and I've even looked using my old school hex editor. So I'd say the metadata panel is correct and @Keywords is showing categories not written in my files.

This file has virtually no metadata of any kind - no EXIF, IPTC, or XMP.  None.  This is a scan, right?    I imported it into a test DB and wrote a keyword to it using the keywords panel and IMatch wrote the hierarchical keyword to the file no problems. So there's no problem with the file itself.

John, a "copy as variable" from the empty keywords field of Ferdinand's custom metadata panel produces this:
{File.MD.IPTC::ApplicationRecord\25\Keywords\0}

There are in fact three keywords fields in that template.  The most important is the first one
{File.MD.XMP::Lightroom\hierarchicalSubject\HierarchicalSubject\0}
{File.MD.XMP::dc\subject\Subject\0}
{File.MD.IPTC::ApplicationRecord\25\Keywords\0}

In IMatch 5 when we say "keywords" we usually mean hierarchical keywords rather than IPTC (flat) keywords. (I realise that this is a little confusing, but normally it's possible to understand what we mean from the context.)

My migration process from IMatch 3 to 5:

1. Used Mario's conversion utility
2. Used John's method to copy my regular categories to @Keywords, as outlined here:

This is where I think the problem is.  I used this workflow and got your exact symptoms.  I see the pasted category under @Keywords but the files are unchanged.  If you right-click on @Keywords and choose the refresh option, I'll bet all those keywords will disappear, since they're not actually in the files.  This is what happened to me.

John - can you try to replicate this in IMatch 5.2.6?  I can certainly copy and paste a category into @Keywords, but nothing is written to the file and the keywords vanish at the next refresh of @Keywords.  I think that this is a bug - either you can't do this, or you're supposed to be able to but something has gone wrong in the rewrite of the metadata code for IMatch 5.2

As I mentioned the other day, "I have a sense of doom and fear that this is all caused by my files never having had any metadata written to them, since I've used only IMatch's excellent regular categories until this attempt to start using a @Keywords workflow."

That usually isn't a problem and I don't think it's the problem here.

Here is my latest suggestion.  Can you take one image and one only, and try using the metadata template outlined in this new FAQ of mine:
https://www.photools.com/community/index.php?topic=3428
Just ignore all the Group and Exclude stuff for the moment.

Please report back what happens.

[JohnZeman]

John - can you try to replicate this in IMatch 5.2.6?  I can certainly copy and paste a category into @Keywords, but nothing is written to the file and the keywords vanish at the next refresh of @Keywords.  I think that this is a bug - either you can't do this, or you're supposed to be able to but something has gone wrong in the rewrite of the metadata code for IMatch 5.2

Ferdinand I can collaborate your latest observations regarding the lack of metadata in Lane's image however for me it still works when I copy a direct category to the clipboard and then paste it into a subcategory of @Keywords (which I just now did).  I then assigned that new @Keywords category to Lane's test image and did a metadata writeback.  Used the ExifTool panel to view the metadata and even ran an external command line script to generate a text file containing the embedded metadata, and that new category was there along with a few others I had also assigned to Lane's image.

It would be tempting to suggest Lane is doing something wrong however if you're having the same results as he then I'm rather stumped at the moment.

[Ferdinand]

One p.s. :  As the test file Lane attached doesn't have an IPTC block, when IMatch writes XMP it doesn't also write the corresponding IPTC field.  I forget how to fix this.  It may or may not be an issue for you.

It would be tempting to suggest Lane is doing something wrong however if you're having the same results as he then I'm rather stumped at the moment.

Isn't that odd?  I wonder why that is.  Is it a different setting or a glitch in one of the databases?

Mario - are you listening to this?  Why is it that John Z can copy a category and paste it into @Keywords, and have writeback triggered, whereas Lane and I do this and writeback is not triggered?  In our cases @Keywords is showing something that isn't in the files, with no pending writebacks, and the metadata and keywords panels also show that the files have no keywords.  If I do a refresh of @Keywords these "phantom" keywords vanish.

[jch2103]

By the way, when I imported and merged the custom F P metadata layout, it seems to have replaced all other metadata layouts, and despite searching thoroughly I can't find the original .immdl files to restore. I only have the F P layout now. How can I get the detault layouts back?

Look in C:\ProgramData\photools.com\IMatch5\presets for system.immdl. When you open it, you'll have a choice of replacing the existing or merging with the existing. I assume you'll want to merge.

John (H)

[Ferdinand]

I posted a bug report.  John Z - you may care to post a log.

Ferdinand-not-John-or-even-Jean

[lanerellis]

 Hi Ferdinand and all. I've been away from IMatch for the past month, as this has been my busiest work period of the year, so my apologies for not getting back to you until now.

I've read your message and the other replies, and also the bug report thread you filed.

This file has virtually no metadata of any kind - no EXIF, IPTC, or XMP.  None.  This is a scan, right?    I imported it into a test DB and wrote a keyword to it using the keywords panel and IMatch wrote the hierarchical keyword to the file no problems. So there's no problem with the file itself.

Yes, it's a scan, as are roughly half of my 90,000 images -- old photos scanned for my genealogy research. I too was fairly confident the issue wasn't with my files but with IMatch reporting my files in @Keywords but not writing the information to my files.

My migration process from IMatch 3 to 5:

1. Used Mario's conversion utility
2. Used John's method to copy my regular categories to @Keywords, as outlined here:

This is where I think the problem is.  I used this workflow and got your exact symptoms.  I see the pasted category under @Keywords but the files are unchanged.  If you right-click on @Keywords and choose the refresh option, I'll bet all those keywords will disappear, since they're not actually in the files.  This is what happened to me.

Thanks, Ferdinand. As I mentioned earlier, I too immediately tried every type of refresh I could think of:

I've also tried refreshing the @All and @Keywords categories, and any other refresh options I could find.

Unlike you, however, when I tried refreshing @Keywords via right-click and "Refresh Data," my @Keywords categories remain.

I've just updated from 5.2.4 to 5.2.6, and a refresh of @Keywords still results in my fully populated categories remaining under @Keywords, yet no metadata in the files or pending write-backs. It's frustrating to see them there but not have them working. :-(

Here is my latest suggestion.  Can you take one image and one only, and try using the metadata template outlined in this new FAQ of mine:
https://www.photools.com/community/index.php?topic=3428
Just ignore all the Group and Exclude stuff for the moment.

Please report back what happens.

I'll give it a try and get back to you. I'll first need to study your "How to migrate categories to @Keywords with a metadata template" thread.

I have to admit that the amount of time I've spent trying to write the category data I've spent years working on into my files with IMatch 5 has been frustratingly high, causing me to slowly grow disenchanted with the program, and I now fear that although John Zeman's method worked for him to quickly convert his regular categories to @Keywords, I may now have to spent tedious days or weeks trying to do the same thing using other methods, especially if Mario has made changes to not allow for the easy method to work any longer.

I'd like to be spending my time organizing my photos in IMatch using regular categories and @Keywords, with my work being written to metadata, not researching difficult and time-consuming strategies to make this work -- it seems to me something that should be quite simple.

I've invested so many years in IMatch that I suppose I'll have to keep spending time until I can accomplish my seemingly simple task, but if I can't use the method that worked for John, or another method that's equally quick and easy, I may continue to drift away from IMatch 5.

I'll also post a brief reply in the bug report thread you posted, Ferdinand.

On another ancillary matter:

By the way, when I imported and merged the custom F P metadata layout, it seems to have replaced all other metadata layouts, and despite searching thoroughly I can't find the original .immdl files to restore. I only have the F P layout now. How can I get the detault layouts back?

The file you want is C:\ProgramData\photools.com\IMatch5\presets\system.immdl and this is the folder for storing such things, and where you'll find the system defaults.

You weren't offered both a merge option and a replace option when you imported my layout?  The trick is to use the merge option.

As I mentioned, I did an import and chose to merge, yet it replaced all other metadata layouts, and now I have no original .immdl files to restore. Despite doing a merge, my presets directory now contains only F_P XMP Input.immdl. I'd hoped that perhaps updating to 5.2.6 would restore any missing default files, but this does not appear to be the case, as system.immdl no longer exists on my setup, either in presets or anyplace else. I suppose I'll have to see if I can find the file in the full download installation file, perhaps, unless there's another way. I like your layout, but would like to at least have the option of using the default ones too.

Thanks again, Ferdinand, John, Mario, and all.

Cheers,
Lane

[Ferdinand]

Lane

That's a long post and there are several as well in the bug report thread.  In short, you want to migrate some categories to @Keywords, is that right?  Let's focus on that.

The short version is that John Z's method won't work.  It may work for a few people at the moment, but not us it seems, and it won't work for anyone once 5.2.8 is out.  So we may as well forget that approach.

But a metadata template will work.  You've found my thread How to migrate categories to @Keywords with a metadata template and so I suggest you focus any further questions on that approach.

I don't know why you're missing that system file.  I've attached it here, so you can extract it and save it as
C:\ProgramData\photools.com\IMatch5\presets\system.immdl
I'll delete this file once you've downloaded it, since I don't think it's good practice to have system files posted here, as they may change.  Actually, a "repair" of IMatch in the Windows control panel should also restore that file. 

[file deleted]

[lanerellis]

Lane

That's a long post and there are several as well in the bug report thread.  In short, you want to migrate some categories to @Keywords, is that right?  Let's focus on that.

The short version is that John Z's method won't work.  It may work for a few people at the moment, but not us it seems, and it won't work for anyone once 5.2.8 is out.  So we may as well forget that approach.

But a metadata template will work.  You've found my thread How to migrate categories to @Keywords with a metadata template and so I suggest you focus any further questions on that approach.

I don't know why you're missing that system file.  I've attached it here, so you can extract it and save it as
C:\ProgramData\photools.com\IMatch5\presets\system.immdl
I'll delete this file once you've downloaded it, since I don't think it's good practice to have system files posted here, as they may change.  Actually, a "repair" of IMatch in the Windows control panel should also restore that file.

Hi Ferdinand,

Thanks -- I've snagged the file you attached, so feel free to delete it now. I appreciate your help.

I'll go back to experimenting with metadata templates to copy my regular categories to @Keywords, as I did a few months back. I spent many hours reading help, looking at examples, and testing, trying to get the result John Zeman got in just a few minutes, but I wasn't able to find a method that worked, most likely because it's been many years since I've done any programming. I posted some of what I was trying is this post:

https://www.photools.com/community/index.php?topic=2286.msg14638#msg14638

I'll study the thread you mentioned and focus on metadata templates, as you suggest.

Thanks again.

Cheers,
Lane