Migration path from Adoibe Elements PSE

[hesmith1029]

I have logged better than 25 hours in the last 5 days trying to plan a transition from Adobe Elements Organizer to Imatch.   At this time I am not even close.  There is tool for going from Lightroom to Imatch,  so doing the same for PSE should be somewhat easier.  I believe there are a lot of people out there that want to get away from PSE, but when you have thousands and, more than likely, tens of thousands of photos you don't have the time to redo all that work.   Imatch looks to be extremely powerful and flexible.  But that is also it's biggest drawback because the learning curve is so steep and high.

I will be more than happy to work on the project.  I can research the Adobe structure and to testing.  I have looked at the Adobe database, it is an SQLite database.  In  few hours I worked out the basic relationships with in it.   Information is also store in XMP files associated with the database, not the photos.  The database points to them.  There is "Face Analysis" folder with .json files in it,  I suspect this where the face recognition stuff is.   I will be more than happy to dig in and the exact details that will be needed, if it will help out.   I also have a good number of older versions available to me going back to PSE 4. 

And I can do a lot testing.   My collection is over 125,000 files which include lots scanned photos and digital photos.  Lots of originals of different formats and lots of edited versions.   Digital photos from a good half dozen sources

Thanks

Herb

[Mario]

What is the problem you want to solve?

You can just index the same folders you have indexed in PSE into IMatch.
IMatch automatically extracts all metadata from your files and XMP sidecar files. This includes keywords, descriptions, titles, face regions and face tags etc.

Have you tried this? What data is missing?

The Lr Importer App was written for the sole purpose of importing certain metadata that Lr stores only inside its catalog database (flags, collections, keyword lists). And to make it easier to migrate from Lr to IMatch. I had several requests at that time. I don't recall requests from PSE user. IMHO the PSE user base is not a typical target audience for standalone DAM software like IMatch. Their migration path is more likely from PSE to Lr.

Did you see this recent thread from a user moving from PSE to IMatch? https://www.photools.com/community/index.php?topic=7987.msg56126#msg56126

[hesmith1029]

I have taken the last few days off from trying to learn IMATCH,  both to catch up on yard work, and to think through what my alternatives are.  ACDSee, DigiKam, FastStone, Zoner among others mentioned in reading countless Google searches yield nothing useful.

I looked at Lightroom years ago and discarded because I don't like their concept of "Non Destructive Editing".   What it really means is that Lightroom has all your edits locked up in their database.  If it goes there goes all your edits. Yes I could add steps to work flow, but configuring a program to disable its big selling feature is never a good idea.   Disk space has always been fairly cheap, and today a down right bargain.  PSE exports your file back to the disk after editing where it is available to any all programs.  And with version sets (nothing more than specialized stacking or grouping) with the latest edit on top it works just great.   It takes 10 minutes of reading to understand how to configure so it NEVER overwrites your original (be it raw, jpg, Tif, or whatever your scanner produced).   Yes it eats up disk space, but as I said that is cheap.

But I digress,  since Imatch will load the Lightroom database I decided to look again at Lightroom.  Installed the last stand alone version and had it import one of my old PSE catalogs.  I was absolutely shocked and extremely disappointed.   None of the organization that is key to the PSE Organizer came across.  Sure the files had the data in it, but not organized or accessible.   

But I now understand where IMATCH is coming from.   The quick first impression of Lightroom and IMATCH is that they look the same.   Lightroom from all sources I read is the Gold Standard of photo organization and management.   So that may explain a lot why IMATCH is the way it is.

Attached is a screen shot of the organization panel in PSE.   The 4 top levels are installed by default and I doubt can be removed.  I expanded the People Tag to give an idea the way I setup that hierarchy.   The 6 people at the root are the ones I use the most.   There are well over 200 people in the database.

The PSE design and philosophy at the top level is solid and very simple and straight forward.  PSE breaks down in the implementation.  They pushed me over the edge when I upgraded to the latest version.  I still have lots old scans in psd and jpg format.  I work on editing them and saving as tifs as I can.  But they were all classified by people, place, event a long time ago.   Now when I save the edit, it does not bring forward the people tags in the original photo.  One more easy to forget step.  From what I find on the forums this was deliberate and will not be corrected.   They did something similar with the Place tags about 5 years back when they introduced the placing of the photos on a map.  (It also slowed things down considerably and required a new hierarchy and no support for moving from old to new ).  They had concept of UNKOWN time in early versions.  I know the date but not time of most photos in my Moms stuff.  That sounded very useful, so I adopted it.  Well they dropped support for it.  Any photo on 12/1/72 with unknown time becomes 11/30/72 11:00 am.  I don't know how many hours of rework this has cost me.  They simply do things that break your old data and don't care.  I have seen more than enough posts various place to know I am not the only one frustrated by Abode's behavior.   I would think there is nice market for a programs which gives people an easy way off PSE to something that just works.

IMATCH looks to be very powerful.   Some things I read lead me to believe I could build out the my people and event hierarchies in the Thesaurus.  Then on import (index as IMATCH calls it) it would read all the IPTC data populate things in something that resembled what I had.   In tests on a small sample I got it to just go one level.   Grandpa and Grandpa Smith IPTC data just stayed as entries under @all.    In fact only some of the photos with Herb in IPTC went under Person|Herb.   Some of them stayed under @all under Herb.  I have no clue what is difference about them.

A few weeks back I used some free tools to access the SQLite database that PSE uses.   In an afternoon a I had good idea of their schema and most of the relationships within the database.  So recreating the hierarchies in another programs becomes a programing problem, not an understand or intellectual one.  I haven't written programs in since DOS and Windows 3.1 days.  Yet another solution that has a very large learning curve to get what I need.

I have put more hours into reading and testing on this than anything ever done for personal use before.   I am disappointed and frustrated.   It seems the simple viewers have all the organizational power one needs.   But of course searching and retrieving by reading the metadata of 100,000+ files just to filter so I see only Mom photos is just not practical.

Herb

[Jingo]

I would say it really depends on what you are after in a DAM product.  I must correct you though on 2 points - LR is NOT the gold standard for organization.. in fact - their Library module is considered the weakest part of their software!  Also - non-destructive editing has little to do with image organization... the beauty of non-destructive RAW (and even non-RAW) editing is that you are applying a set of instructions without altering the underlying image data.  You want to export a file to use in another software - simple - just export the file as a layered TIFF or PSD.  Export as a JPG and your edits are saved forever in the final image.   Some software can even "interpret" LR's edits and attempt to recreate these edits right in the RAW file (so long as you are writing the XMP out to the file or sidecar).  This is not limited to LR... most software offer non-destructive editing and save this data in their own unique format.  No one is forced to use it - just export the file as a JPG and you never need worry about those edits again!  Anyway.. just thought I would share that info.

A dedicated DAM like IMatch does so much more than simply group your images so I think you need to ask yourself what is most important and how best your wish to organize your images.. an all-in-one editor and management program like PSE, LR, ACDsee, etc... or in a powerful DAM program.  The integrated options are sometimes all you might need... I don't use 70% of the power of IMatch.. but I chose a dedicated DAM because I enjoy the process of organizing my metadata in a pure XMP and catalog form..  it is a complex and powerful program - and does take awhile to learn and get the hang of.  Once you are there - there really isn't much on the market that can match it's capabilities (and support!).

Once you have your metadata from your images loaded into the catalog - you can organize in a myriad of different ways, create shortcuts to enable quick keyword addition, create synonyms and relationships between people and create metadata templates for adding hundreds of keywords (if you wish) all at once.  Your example of finding Mom photos across 100,000 images can be done in IMatch with a few keystrokes in about 10 seconds.. and frequent or complex searches can be setup and saved as data-driven categories for future one-click retrieval. So much power and so many options - and this may be part of struggle for new users.  But - I always say to start with a small database - 100 images - and play with the software, read the excellent help files and learn how to use the software.  This is my approach with any DAM program (and I've tried and used just about all of them!).

Good luck with your decision... I hope we can help you along the way!

[Mario]

Attached is a screen shot of the organization panel in PSE.   The 4 top levels are installed by default and I doubt can be removed.  I expanded the People Tag to give an idea the way I setup that hierarchy.   The 6 people at the root are the ones I use the most.   There are well over 200 people in the database.

You did not really reply to my questions so I'm non the wiser and cannot help you.

I see "Keywords" at the top of your PSE screen shot. I assume that these are standard flat or hierarchical keywords and that PSE stores these in your image files? If this is the case, you will see all your keywords in IMatch in the Category Panel, under the @Keywords top-level category. Is this the case?

The "People Tags" can be a PSE-only feature. Usually, "people tags" are words you assign to a "face" you have somehow marked in an application. The standard for modern DAM applications is to store this as part of the XMP metadata inside the image. This allows other applications (like IMatch) to read this data.

Does IMatch show face annotations when you look at one of your files in the Viewer?
If not, check in IMatch under Edit > Preferences > Metadata 2 that the option Import XMP face tags into Annotations is enabled. This is the default but better to check.

An XMP face region can have an associated "name" (tag). IMatch automatically imports these. If you have a face region tagged with "Bob", IMatch by default maps this to the keyword WHO|person|Bob. And this also means that all Bob photos show in the Category View under the @Keywords Category > WHO|person|Bob.

Is this the case? If PSE has written proper XMP face regions with tags, Match will not only show the faces in the Viewer, but also the tags below each face. Did you check that?

IMatch also supports hierarchical face tags. If you assign the tag Smith Familiy|Bob to a face in the IMatch Viewer (or the face region is imported by IMatch from another application with such a tag), IMatch will by default assign the keyword WHO|person|Smith Family|Bob. And this gives you a nice hierarchy in the IMatch Category View.

You can change the default import hierarchy (WHO|person) to anything you like under Edit > Preferences > Metadata 2. IMatch is flexible and adapts to your preferences.

Did you make PSE write all metadata into the image? Maybe PSE holds all your data only inside its proprietary database and hence makes the data inaccessible for other applications.
You can attach a ZIPped photo so we can see the metadata PSE has written to it. Or send a photo to me via support email address.

Without answers to these questions and a sample image, we cannot provide better support.
I just have never used PSE and I don't know what it does with your data or where it stores the information you have entered. PSE is a program used mostly by amateurs and SOHO users, and may not support all or even any of the standards and metadata formats we use in the modern DAM world.

[hesmith1029]

Actually all of the "tags" in the screen shot show up in the IPTC Keywords field.

[Mario]

Actually all of the "tags" in the screen shot show up in the IPTC Keywords field.

Legacy (IIM) IPTC or XMP IPTC? Very different.

In any way, if the face tags exist as keywords, they will be imported automatically by IMatch and mapped into XMP subject (keywords) and XMP hierarchical keywords. And hence they will show up in @Keywords.

Please read my questions above again and try to answer each one. We cannot help you if you don't answer questions which tell us what data is your files. You did so far not supply a sample image, which would explain a lot.

Don't forget to read the post I linked into my first reply:


...moving from PSE to IMatch? https://www.photools.com/community/index.php?topic=7987.msg56126#msg56126

[hesmith1029]

I wrote the entry you are referring to.

My files are all too large to meet the 2 meg limit on files even if compressed.   Alternatives?

[hesmith1029]

'What is the problem you want to solve?"

I would like to migrate all of my PSE data into Imatch,  and have it accessible for retrieval by simple searches.  See Capture 4.jpg  This is search for all photos with Mom in them that do NOT have Karen and not in California.   Good as far it goes as it does not support OR,  but only AND.  (Another example of Adobe with good concept, POORLY executed).    I would like to be able to get the reorganization done in under 100 hours of work.  Another words I don't want to start from scratch.

"Have I tried this?"   See file Capture.jpg.   This was for 12 files.   They all had geo data.   Only 5 came in.  The Thesaurus only caught some of the photos and only one level deep.  (Persons|Herb.....)  (See Capture 5.jpg).   Not every instance of Herb got mapped to Person|Herb.   These 12 files average 60mb, not less than 50mb so can't upload.

PSE data, just like lightroom, has all its data either in the SQLite database or in support files linked to the database.   PSE offers the ability to write it back to the Metadata in the file.  See photo supplied in the thread you directed me to.   I am sure Lightroom does also.

The path to usable system for filtering and retrieving photos based on who, what, where, and when  from having all the raw data in the ITPC keywords file is not at all clear.   And that is ultimately what I am attempting to accomplish

[Mario]

See my initial post with the address you can send files to. support email address

IMatch does no fill your thesaurus when you import files.
But it will map all keywords in the file into proper XMP hierarchical keywords.

If you say that some keywords were not imported, are these keywords in the image?
You can always see the native data in your files using the ExifTool Command Processor (Tools menu) in IMatch, and the "List Metadata" preset.

If PSE has not written the entire hierarchy of keywords, you first need to setup your IMatch thesaurus to match whatever you had created in PSE.

If you had a keyword "Family|Smith|Bob" and PSE has only written the keyword "Bob", you need to setup

Family
|- Smith
   |- Bob

in IMatch first. Then it can map the "Bob" keyword when it ingests your files.

I don't know a thing about PSE of how it stores data. The common metadata standards are all supported by IMatch (and Lr) but not necessarily by an entry-level software like PSE. This also may depend on the version of PSE you use, how you have configured it etc.

Experience tells us that the initial phase of migrating an image collection from "toy software" to a real DAM with proper metadata can be painful. Users always learn to late that they rely on some proprietary piece of software or some uncooperative tool that tucks away precious metadata in some proprietary database.

This is of course not a fault of IMatch. IMatch is all open standards, and you won't find better metadata quality and flexibility elsewhere.

[hesmith1029]

"IMatch does no fill your thesaurus when you import files.
But it will map all keywords in the file into proper XMP hierarchical keywords."

Never said anything about Imatch filling the thesaurus.   But after manually entering items in the Thesaurus it does not appear to do particularly good job of mapping those words into proper XMP hierarchical keywords.

"If you say that some keywords were not imported, are these keywords in the image? "   They are in the @all listing,  so they must have been in the image.   See capture jpg.  Mom is appears twice.

"If you had a keyword "Family|Smith|Bob" and PSE has only written the keyword "Bob", you need to setup

Family
|- Smith
   |- Bob"

See Capture 5.jpg  for what was entered in the Thesaurus.

You are concentrating too much on the back end data.   The issues are in the frontend presentation of the data to the user.   The user interface.

All references to jpg's are to screen shoots posted with prior post.

[Mario]

We are talking in circles. I need at least one sample file so I can see what PSE has actually written to the file.
Your screen shots don't tell my much without having seen the actual metadata in one of your image.

We continue after you have provided a sample image.


Note: The Thesaurus and the @Keywords category are different features.
@Keywords reflects the actual keywords in your files, the thesaurus allows you to use a controlled vocabulary for assigning keywords and other metadata.