Importing CSV file problem

[PaulT]

Hello,
I have just downloaded the trial 5.5.
My need for 5.5 is to use the CSV file import facility.

I've been trying various ways, reading the help and messages on this board for 4 hours but have run out of ideas - so was wondering if there was something simple I've missed.

I write my captions/keywords/Location/Copyright on an Excel sheet. I then need to embed these into jpgs.
I have saved my excel file as a CSV file

When I run the "import CSV file module". it seems happy and says "import completed".
But nothing appears in the metadata panel.

I have read the help "importing CSV files" and believe I have the import module set up correctly. (I have tried all the other field delimiter setting as well)

The only thing I can think of is from the example on the sample file on the help:
It has the full path to the file on the c:
I presume this is not necessary as Imatch has already catalogued the files. In case it is necessary I manually put the path in on the csv file but that didn't help.

On the CSV file I notice that here are no "" in some places between fields so I manually added all "" between fields. That didn't help

So I have run out of things to try.

In theory should I be able to convert an Excel file to csv and run the import module?

Thank you for any help.
Regards, Paul

PS on the image file name I have included one line which has .jpg - just in case Imatch wants that included

[thrinn]

Specifying the full path including the extension is necessary because the file name alone is not unique: There might be two different files with the same name in different folders. For IMatch to know which file the data should be added to, always use the full path, as mentioned in the help.

It would be helpful if you could attach a screenshot of your import configuration (like the one in the help, to be found under "Using the Import Module") and also an example of the csv file you are trying to import.

[Mario]

The import module cannot find your files. Your CSV file only has the file name, but no path.
As explained in the help, the script requires a fully qualified file name, like c:\images\beach\me.jpg.

[PaulT]

Thank you Mario & Thrinn for your help in this - its appreciated

I have added the full path.

I make the excel file > save as CSV file

Everything works OK and all data is entered - except the keywords are not written.

I have opened the csv file as a word file and see there could be a problem with the "" (or lack of "") generated by the csv file but cannot see how to make it generate what the "Imatch import system requires.
I've tried separating the file fields manually with "," but been unsuccessful in getting the system to recognise the keywords.

(I am attaching a zipped Excel and csv file plus png of the database)

[Mario]

1. It seems you are importing keywords into XMP flat keywords (dc:subject). Don't.
IMatch internally works with hierarchical keywords only.

2. In your screen shot, you have switched the "Repeatable tags" option to "Not split"
This means the import will not try to split values which you import into repeatable tags like keywords.
If your file has multiple values in a column (e.g., separated by , or whatever) you need to tell the import that.

See the help file for the CSV file for details.

[PaulT]

Thank you once again for your reply Mario.

I have changed the "Repeatable tags" option to ","  I have also changed it to every other option, but can't get it to work for the keywords.

"you are importing keywords into XMP flat keywords (dc:subject). Don't"
IMatch internally works with hierarchical keywords only.
See the help file for the CSV file for details."

I have read and re-read the help but just don't understand what to change. Is my Excel file not suitable for hierarchical keywords? when you say "Don't import flat keywords" I just don't know what to change. I don't know what (dc:subject) means.
I simply have a list of keywords to embed in an image.

I sent a copy of the excel sheet in the last post. Are you saying with Imatch I cannot embed a series of keywords into an image with an Excel file?

I've been on this 6 hours today and 4 hours yesterday and am nearly there but it looks like I'm going to fail with the keywords (which is my most important bit)
Could I just ask if it is possible with Imatch 5.5 to get these keywords embedded with a csv file from an excel file - I just can't see what I can do to the excel file to make them hierarchical and not flat.

For many years I've upgraded to the latest Imatch. Imatch 3.6 embeds my keywords perfectly but with a complicated script someone wrote for me - I'd now really like to upgrade to 5.5 and use this csv import if possible.

[Mario]

with a complicated script someone wrote for me

Of course. A purpose-built script which has been written to understand the special CSV format you are using can work.
But your script is for IMatch 3 and no longer runs on your new PC because it seems to miss a reference or component no longer available on your new machine.

A CSV file, by definition, has one value per column. Columns are separated by a special character, usually a comma, semicolon or tab stop.
Everything that goes beyond that is "custom" and non-standard. Like your CSV file. Which has multiple values in one of the columns.
Good that I did consider such things when I wrote that CSV import module three years ago.

This is how a row looks in your CSV file:

C:\Users\Paul7910\Videos\new database\161107_01.jpg,Skyline,"buildings, Florida",Tallahassee,Florida,USA,Paul T,

It uses comma to separate the columns. Then there is one column which is wrapped in " and " seems to contain two keywords. But these are also separated by a comma. Extra complicated.

Why not just different characters to separate columns and multiple values within a column?
That's how this is usually done. The " and " are always tricky and mean different things to different applications.

Anyway. I've download your file, set the options right and it worked immediately from what I can tell.

This is what I used:



And this is how one of the files looks in IMatch:

[PaulT]

Thank you Mario.

I've changed things as per your import details. And the keywords went in.

I've written the keywords back to the files and opened a file in Photoshop and had a look in File > Info (screen grab attached)

It has written the keywords three times and included "" so I'll do some more testing to see if it's anything I've done or whether it's an intrinsic hierarchical thing.
If a comma , makes things more complicated between keywords - I can make it a semi-colon ; or anything else if it would be better or easier.

Now the full image path has to be written in the Excel file; I'm busy googling how to enter that path at the beginning of 13,000 cells, plus then how to add ".jpg" at the end of those cells 

I'm leaving the computer now so it can cool off and will continue the effort to get it right on Monday.

Thank you (and to Thrinn) for answering so quickly - particularly on a Saturday.

[Mario]

1. Did you run the import several times? Without clearing the keywords in your files?
I'm not sure that the CSV import script checks for this case.
Just select all files before you import again and use the "Remove all keywords" command the Keyword Panel.

2. In my tests, I did not get the " or duplicate keywords.
Duplicate keywords have nothing to do with hierarchical keywords. IMatch maps hierarchical keywords into flat keywords automatically and does not allow dupes.

3. Get rid of the " in the file. Separate the columns with ; and the keywords with ,
Much easier.

Now the full image path has to be written in the Excel file; I'm busy googling how to enter that path at the beginning of 13,000 cells, plus then how to add ".jpg" at the end of those cells 

This is a simple search and replace for any text editor out there. In Excel it probably requires a script. Or may you can also do it using search and replace.
I'm sorry, but I don't do free Excel consulting.

[PaulT]

I've done some tests and it looks promising. The duplicate keywords were indeed me running the import a few times.

I am checking out the possibilities of changing the Excel file parameters - I wasn't suggesting or hoping you give free advice on this, I think I mentioned I was googling it.

I have noticed the keywords are re-arranged into alphabetical order, except for the very first keyword - which is placed at the end.

I presume there's no way of getting Imatch to preserve my keyword order? I have searched "Keywords" in your help but see no mention of it.

[Mario]

Which keywords did you look at?

As I said, importing keywords into IMatch from a CSV file produced by Excel is something maybe 5 people in the entire world will ever need.
You can duplicate the CSV import script and adapt it in any way you want it.

Note that Basic scripting is a thing of the past and will be removed from IMatch in a couple of months.
Then IMatch will use modern JavaScript and HTML for scripting purpose. But there are not that many people around anymore who ever would need that.

[PaulT]

Note that Basic scripting is a thing of the past and will be removed from IMatch in a couple of months.
Then IMatch will use modern JavaScript and HTML for scripting purpose. But there are not that many people around anymore who ever would need that.

Are you saying that in a few months Imatch will be a better fit for what I need to do? Or will it be more difficult?
I'm just trying decide if I should pay for the present Imatch or should I wait till the new one comes out which will use modern JavaScript and HTML for scripting purpose?

[Mario]

I doubt that the new implementation of the CSV import (either built-in in IMatch or a re-implementation of the current script) will have additional features.
Nobody ever requested more, which is a good indicator that this script is not used by many.

[PaulT]

Ok, thanks Mario.

I think you've given me enough info for me work on this so it does what I need.

However from what you have said (only 5 people in the entire world do it the way I'm doing it) I am going to have a look round and see what everyone else is doing which is so much better.

I presume these are only the two choices:
One can either embed keywords/caption/description/headline/author etc by using an Excel file (then CSV) as I do or a hierarchical method directly in a proprietary software like Imatch and Lightroom.

I keyword 1,000 of images and simply copy&paste/fill down/fill series then autofill/search/replace/spellcheck and a lot of other Excel features over the file. I can easily assess 100s of lines of metadata in seconds, everything is visible at all times - nothing happening behind the scenes that I can't see.
Once I'm ready convert to csv file (and Imatch) will embed it all in seconds.

If I ever need to change anything I can simply pass the script back over the relevant images. I am secure in knowing I have my Excel sheet and csv separate from any proprietary software.

I have never used hierarchical keywords but am of the view (which may be wrong) that images have to be done individually and/or synced to other identical images then altered as necessary. The hierarchical keyword paths have to written or prepared in advance (I presume bought software doesn't know the attributes of a weasel or bungalow or sausage and even if it did, would they be the keywords I would want applying? I know that Lightoom (I use LR for processing RAWS) can have templates add any metadata to images on import. All this is too rigid and pre-planned for me.

A very clever German friend of mine introduced me to Imatch many years ago and let me use his script - The system has been a joy to use, super efficient and Imatch has always worked beautifully. Through this method, no image (or video with help from Bridge) leaves my computer without full metadata.

But I am always willing to see if there's a better way - so I'll make inquiries. I'm not asking for advice    But will always listen to it if offered.

If anybody else would like to add or repute what I have said - please do, I'm open and eager for opinions. I don't need to be right or fear being wrong - I just need to get done in the best possible way that that needs doing.

Cheers, Paul

[Mario]

(...) see what everyone else is doing which is so much better.
(...) One can either embed keywords/caption/description/headline/author etc by using an Excel file (then CSV) as I do or a hierarchical method directly in a proprietary software like Imatch and Lightroom.

There seems some misconception on your side. Lr is foremost a RAW processor.  IMatch is a proper DAM.

The primary task for a DAM are managing your files and metadata input and display. Excel does almost never come into play with image management.
IMatch has super-flexible and poweful built-in editors for metadata and keywords. You can do things with these tools Excel cannot even come close. Why somebody would even try to manage digital asset metadata in something like Excel escapes me entirely.

There is also nothing proprietary at IMatch. When you edit your metadata and keywords in IMatch, IMatch stores this data in your files. It the best possible quality. From this moment on, your metadata os persistent, compatible with all image editing and digital asset management applications out there. Every major application will be immediately able to see and work with the metadata you have edited in IMatch.


Did you even try out the Metadata Panel and Keyword Panel yet?
The Thesaurus?
Metadata Templates?
Favorites?

You can make IMatch fill in metadata and keywords into thousands of files with a single mouse-click. The thesaurus avoids typos and typing and improves metadata quality and consistency massively. Favorites allow you to assign any number of keywords with a mouse click or keyboard shortcut. Synonyms save a lot of typing and make sure that all keywords are in place. Visual guides allow you to see immediately if you have forgotten keywords, if all keyword groups you maintain are considered.

I could go on like this for a long while.
Excel is a superb application for spread sheet purposes and for managing lists. It stinks at digital asset management.

PS.: If Excel is involved in a DAM workflow it is as a recipient of metadata. Once you have added metadata and keywords in IMatch, you can use the Text Export module in IMatch to export CSV files you can then look at or maybe print out in Excel. Not the other way round.

[PaulT]

There seems some misconception on your side. Lr is foremost a RAW processor.  IMatch is a proper DAM.

No, I thought I mentioned to you earlier that I use LR for processing RAWs. I just mentioned LR here vis-a-vis entering keywords. It was never designed as a DAM software even though some photographers try to make it so.

Excel does almost never come into play with image management.
You can do things with these tools Excel cannot even come close. Why somebody would even try to manage digital asset metadata in something like Excel escapes me entirely.

Now I understand why you have spoken to me with a little disdain in your tone.
In this whole thread I have only wanted, and asked, to know how to get keywords from a csv file into my images using 5.5 - I never mentioned that I then use my DAM software to manage my images because I use a competitor of yours, and I didn't think it was correct to mention your competitors as this is the Imatch forum.
You assumed I use Excel to manage images - I can see why you think I'm a little stupid. Please don't assume - just ask.

Did you even try out the Metadata Panel and Keyword Panel yet?
The Thesaurus?
Metadata Templates?
Favorites?

No I haven't but I'm sure they work really well - though not as well as dedicated word software like Excel 

The thesaurus avoids typos and typing and improves metadata quality and consistency massively. Favorites allow you to assign any number of keywords with a mouse click or keyboard shortcut. Synonyms save a lot of typing and make sure that all keywords are in place. Visual guides allow you to see immediately if you have forgotten keywords, if all keyword groups you maintain are considered.

This all sounds pretty good and indeed what I'll probably do if my present workflow is not possible anymore.

I could go on like this for a long while.

It's always interesting to here the advantages of various software - particularly by the guy who wrote it.

Excel is a superb application for spread sheet purposes and for managing lists.

That's why I use it to write my keywords and all the other columns I need

It stinks at digital asset management.

That's why I use DAM software.

Anyway all this is getting side-tracked from the reason of the thread; which was Importing with a CSV file
As I said, I want to continue to use Imatch for this and it looks as though I can, as you wrote Imatch 5.5 to have that facility. Which is great.

[Mario]

So, basically, you use another DAM software.
And only wanted to use IMatch to get your keywords out of Excel into the metadata in your files?
I recommend you make yourself aquatinted with ExifTool which can do that without the intermediate step of using IMatch.

No I haven't but I'm sure they work really well - though not as well as dedicated word software like Excel

I disagree. Excel is neither a "word software" nor can it keep up with IMatch when it comes to metadata management. No thesaurus. No synonyms. No features to improve quality and consistency. No ways to import pre-made controlled vocabularies. All you manage in Excel are lists of words and then you struggle with getting these words into shape and keywords. What kind of workflow is that?

If you already use a another DAM and you still need Excel, you may re-consider your choice of your DAM.
I wish you luck with your other DAM software.

[sinus]

No I haven't but I'm sure they work really well - though not as well as dedicated word software like Excel 

Hi Paul

I am not sure, do you mean this really or do you make only a joke? 
I am really not sure.

Because I do not know Excel very good, though I use it from time to time, but it is hard for me, to believe, that Excel is as good as a DAM-software.
I think, Excel is very good for calculations and Word is very good for texts.

And a DAM is very good to manage all the files, what we have.

I am not sure, how your workflow really is.
But I would try to use only one DAM for your images (or files), if you like Excel, then use it.
If you like your other DAM, use it and if you like IMatch, use it.

But take several software for managing files, is not a good idea from my point of view.

I worked quite a long time with Access (Microsoft) to manage my files. Although I think, with this DB it is good possible, to manage files, it means a lot of work and knowledge.
At the end I gave up (several years ago), because for solve some specific problems I had not the knowledge with Access.
Hence I changed to another DAM (Portfolio), but after a while I discovered IMatch and since then I never changed again and I am happy until now. 

I hope, you can solve your problems.

[PaulT]

Hi Markus,

Thanks for your comments.

I do use quite a lot of different software and pick the bits I want from each. As you say, that's not ideal, it's expensive and not as efficient in general terms. But it's a little unusual to have one piece of software that's better than all the others for everything.

My workflow is actually quite good and works well but it evolved for various reasons that are unimportant now. What is of interest is the future.
As I mentioned earlier; I'm totally open to suggestions and ideas but I have to keep my workflow uninterrupted so any changes have to happen alongside present workflow.

I'm reading an article right now (whilst Premiere Pro renders some videos) to get a further idea on how this could work for me. I'm also seeing a lot about Lightroom keyword lists so trying to figure out where Lightroom comes into this.
https://www.photools.com/4588/free-controlled-vocabularies-imatch/
http://workshops.bigsunphotography.com/news/28-free-lightroom-keyword-library-download

Cheers, Paul

[sinus]

But it's a little unusual to have one piece of software that's better than all the others for everything.

Of course you are rigt.
That is why I use also different software.

Like InDesign for Laouts.
Word for text.
Excel for calculations.
Premiere Pro for videos creating and rendering.
Photoshop RAW for raw-processing.
Photoshop for editing.

And well ... IMatch for manage files and put keywords in it (controlled vocabulary).
I think, IMatch wants not be "one for all", but it want be one of the best DAM-solutions at least for say about until 400.000 files.
And in my opinion for this IMatch is something of the best.

I have now about 330.000 files, managed by IMatch.
I would even not try to do that with LR.
I have read a lot about LR, it is a very good DAM ... but (so far I have read this several times), if I have more then, say, 100.000 files, then the problems starts with for example slowness.
But I have not tried it.

I know your sent articles, thanks, they are interesting, or course.

Good luck, hopefully you will find something, what is good for you.

[Mario]

I'm also seeing a lot about Lightroom keyword lists so trying to figure out where Lightroom comes into this.

Lr keyword lists are a reduced variant of the universal thesaurus approach used by IMatch.

For more info about how Lr and IMatch work together

Information about controlled vocabularies and how IMatch can utilize them.  In this knowledge-base article you'll also find download links for free controlled vocabularies and links to web sites which compare them.

Note that IMatch of course can import the "keyword list" format used by Lr.

I suggest you read about the Universal Thesaurus and the Keyword Panel in the IMatch help. It seems you are missing a lot and make your workflow a lot harder and slower than it should  be.