Iphone6 - Some pics show Metadata in IMATCH - Some Do Not

[kiwilink]

I shot several pictures with my new Iphone6.  When I loaded them into IMATCH and opened the Metadata panel I noticed that some pictures show the Metadata and some do not.  I rescanned the folder with "force Metadata update" and some of the pictures were populated with Metadata and some were not.  I ran the EXIFtool Command Processor on two pictures (One that is populated and one that is not).

Any idea why some load the data and others do not?    I am running the latest version (5.2.12) of IMATCH.

Thanks for helping

Michael

[attachment deleted by admin]

[Mario]

Log file of a session where it did not work?
Sample images?

[PaulS]

I'm not able to currently provide image files or a log, but I have an iPhone 6 and experienced this identical behavior including the results from ExifTool.

[kiwilink]

Mario:

Attached is the log file.  What is happening is that I shoot the pics with the Iphone 6 and load them into IMATCH.  See the Catnometadata attachment.  This is all I see.  I then have to right click on the images and the hold down CTRL and choose scan rescan and then choose "Reload Metadata".  Once I do this, the Metadata is populated (see attachment CatafterforceMetadata). 

Is this related to my Metadata Preferences?  I captured them in the last attachment.

Thanks Mario and thanks PaulS for the response!

Michael

[attachment deleted by admin]

[kiwilink]

Mario:

I know you have not yet read the post I left above but I wanted to add one more thing.  As I mentioned in the 1st post, sometimes even when I right click and hit CNTRL/Rescan and choose "Reload Metadata" it does not occur.  Sometimes it works and sometimes it doesn't.  I just tried it on a single image (IMG_3789) and it didn't work and I immediately copied the log.  You can see this at 11:15 18:04:53 (log is attached).  Because the size of the pic may be too large I will send this via email to you.  Maybe this will help.

Please take a look at my post above also!

Thanks!

Michael

[attachment deleted by admin]

[Mario]

This first attached log file contains a number or warnings: The system is running low on memory while creating cache images. How large (dimensions) are these files?

Then ExifTool complains about invalid/unknown maker notes for several of the files. ExifTool also reports that the MicrosoftPhoto namespace uses a wrong URL. I don't know what impact this has on the metadata being imported for these files.

There is no error reported, and the average number of metadata tags imported from each file is about 120.
Which files failed, exactly? The file names would help making some more sense from the info in the log.

For example, the file IMG_3668.JPG used in one of the screen shots: I can see that IMatch has created a cache image for it, but I see no report from ExifTool importing data from that file. This may mean that the file has been skipped (locked, unknown format, failure loading the file etc.)

The second log file also contains ExifTool warnings about incorrect namespace names, invalid maker notes, attempts to fix maker notes by shifting file offsets etc.

Please try to identify a failed file or two and send them to me. I'll have a look and may forward them to Phil so he can analyze the metadata mess in these files. Unknown maker notes may just indicate that ExifTool does not know about them. Invalid maker notes are a different thing, and a wrong namespace for MicrosotPhoto (which software do you use to process these files?) is also uncommon.

At least I can see that ExifTool imports all files for which it gets data.
Note that what you see with your arguments in the ECP does not match how and which data IMatch extracts. IMatch uses different arguments and also has to apply MWG rules and argument files in order to produce the 'rich' metadata needed.

[kiwilink]

Hi Mario!

Thanks for the quick and thorough response.  I will try to answer all your questions as best I can.  I am running a very high powered I7 3770 processor with 32GB of Memory so I don't think memory is my issue (however, what I think caused this "low on memory" is that I took all of my Iphone pictures that I had taken (2,771 images) and brought them into one folder and did a scan/force Metadata update on them.  I then noticed that the directory of 2,771 images included .MOV files from my phone and also the new Iphone 6 "Panoramic" images.  Most images were from my old Iphone 4 and images were averaging 1,870KB in size.  The new Iphone 6 images are a little large (2,252KB) but there were about 5 of the new "Panoramic" images that were averaging 15,757KB.

To make it simpler for you to debug I just turned on my system this morning and only did one thing (highlighted an image that I can't get to load the Metadata into IMATCH).  I right clicked Image IMG_3789 and held down CTRL to rescan the image with "Reload Metadata".  Then I immediately copied the log for you.  This Image was just one of dozens of new pics I took with the Iphone 6.  I just hook up my phone and drag these from the physical phones directory to the IMATCH Iphone directory that has 2,771 images in it using Windows explorer.  This image is only 1,485KB and the properties look like all of the others (its not a panoramic or anything out of the ordinary)

AS for the ECP, I only used the EXIF Command Processor to show you that I see lots of Metadata associated with the image.

I don't know what the MicrosoftPhoto namespace is (maybe this has something to do with the Iphone 6 metadata - over my head here).  I am not running any special software except Windows 7 and IMATCH for managing my photos.  This image I am sending you was not manipulated with any software - just imported from the phone using Windows Explorer.

I am sending you an email with the Image I mentioned above (IMG_3789) which won't load the Metadata and also the Image you mentioned in your post (IMG_3668).  If you need any further information please let me know. 

Also, note that when I right click on this image and hit CTRL/scan and choose "reload Metadata" I receive a dialog box that says "Some Metadata is Protected, you have configured options to protect unwritten metadata and/or rating and label ........". Does this have anything to do with what is occurring

Thanks Mario for your help!  P.S.  I just shot 6 new pictures with my Iphone 6 and copied them to the Iphone directory in IMATCH.  Under Properties it said a rescan was needed so I did one and the same thing happened - no Metadata is showing.  I even did a "Force Update" and still no Metadata.

Mike

[attachment deleted by admin]

[Mario]

Send these six files as well.
I could not find any problem loading metadata in your log files, maybe ExifTool just skipped the files silently.
If you can reproduce this with the six files, maybe we have a valid test case. I'll look into this, but it may take a while.

[kiwilink]

Mario:

OK - thanks for looking at these!  I just sent you via email 6 more images I just shot that don't load the Metadata.

Michael

[Mario]

Please also add a bug report. Posts in the general discussions are not tracked and I will forget.

[kiwilink]

Mario:

Thanks for the reply.   Do I copy every thread to the Bug report or just the first one?  Sorry for the question but I want to make sure I don't do the wrong thing and duplicate the entire thread to the Bug Report and the General Discussion area.

Thanks for working on this.

Mike

[Mario]

Thanks for sending in the images.
This is a problem in ExifTool. The XML output produced by ExifTool for these files is invalid and thus IMatch cannot import the data. Phil accidentally produces XML node names which are invalid. I will contact him about this.

Do I have your permission to send him your two image files so he can check for himself?

[Ted]

Mario,
I just got back from a one week vacation.  Yesterday I started processing my photos and found that those I had taken with my iPhone 5 - contained no metadata at all.  When I look at them with ExifToolGUI I do see metadata.  Probably the same problem as the OP.  If you want logs or files, please let me know.

[Mario]

Did you check in Metadata Browser?
It depends on which data the iPhone has written. Some tags cause ExifTool to produce an invalid tag name when exporting to XML and hence IMatch cannot read the data. This is also recored in the the log file.

I have already reported this glitch in the ExifTool Forums and I'm sure Phil will soon react.

[Ted]

Did you check in Metadata Browser?
It depends on which data the iPhone has written. Some tags cause ExifTool to produce an invalid tag name when exporting to XML and hence IMatch cannot read the data. This is also recored in the the log file.

I have already reported this glitch in the ExifTool Forums and I'm sure Phil will soon react.

I have checked the Metadata Browser.  Only File Extension, File Folder, File Format, File Name, and Lifetime ID.
That said I did a Force Update and then a Reload Metadata.  If I do the entire folder, nothing happens.  I did several individual photos and gain the metadata - but still there are several (half) photos that show no metadata.  I'm attaching a log file.

[attachment deleted by admin]

[Mario]

From the log file I would say that IMatch is importing data just fine. No warnings or errors.
When you search for ET-Extract in Notepad, you'll see the file names data was imported from, how many tags were imported etc.
Which file (name) failed to import data? Knowing a file name would make it possible to search for it in the log file...

[Ted]

Ok, so I did a read metadata on one file (IMG_0918.JPG) and metadata was extracted.
I then highlighted four files (21,22,25 and 26) and did a read metadata - nothing happened.
Did a RM on 21, 22, 25 (one at a time) and data was extracted
Did a RM on 26 and nothing happened.

Attaching the log file.


[attachment deleted by admin]

[Mario]

Hm, strange. No errors or warnings. Each of the files you have scanned is accounted for in the log file. All files show up with about 140 metadata tags each, data is imported and the database is updated. For example:

IMG_0921.JPG 140 tags
IMG_0922.JPG 148 tags
...

no problems at all. And no metadata is shown for these files? Not even when you switch to Browser mode?

[Ted]

Hm, strange. No errors or warnings. Each of the files you have scanned is accounted for in the log file. All files show up with about 140 metadata tags each, data is imported and the database is updated. For example:

IMG_0921.JPG 140 tags
IMG_0922.JPG 148 tags
...

no problems at all. And no metadata is shown for these files? Not even when you switch to Browser mode?

Mario,
You misread my statement - Metadata was extracted from IMG_0921, 22 and 25 -- but, NOT from IMG_0926.  When I looked at the log, I saw no reference to file IMG_0926 at all.

I examined IMG_0926 with ExifToolGUI and it shows lots of metadata.  I ran diagnostics on my database (no errors) and and rescanned (Force) the image and reloaded metadata -- still no metadata shows with IM5.

I'm attaching the log - and the file for your review.

[attachment deleted by admin]

[Mario]

I would need that image file then for some tests.

[Ted]

I would need that image file then for some tests.

The image file was in my last post.

[Mario]

The image file was in my last post.

I see. Thanks.

This file is affected by the same problem. It contains proprietary Apple metadata which causes invalid node names when ExifTool is exporting the XML data that IMatch imports. IMatch cannot import this data and skips the entire file record / batch (if multiple files were imported at the same time).

Phil could reproduce the problem and will ship an update soon. I will include this update in the next IMatch release.

[kiwilink]

Mario:

Thanks for working this!

Michael

[kiwilink]

Hi Mario:

I tested the fix for this in 5.2.16 and it works great!

Thanks a lot for doing this!!!!!

Michael