csv import: formated description

[tomzappa]

Sorry to ask about Metadata again. I know, you are not amused about Metadata-Questions.
I have to write formated Data into the XMP::dc\description\Description\0 tag
(it's not my idea - it's the customers choice - specialy for an XNView IPTC-Viewer)
Something like:

Fotoausschnitt: km 123.4 bis km 567.8
Position der Kamera: rechte Seite
Blickrichtung: Südost
Aufnahmekoordinaten: 3404998.33,5560485.83
Zielkoordinaten:          3405202.50,5560223.06
Horizontalstrecke:333.00m
Flughöhe: 123.00m
Azimuth: 153 Grad

All those Text is written in the XMP::dc\description\Description\0 - but without linebreaks - and without ö,ä,ü - (but i'm working with utf-8).

Years ago i have realised this with a basictool, written by myself, in IMatch 3.4 but this is no more availiable :-(. There i was able to get the linefeed with inserting CRLF. Now i have checked out the CSV-Import App with "|" (which works for Keywords perfectly!), ";", ".", ",", "{clrf}","\n","\r" and many other exotic Symbols.
And yes, i have read the Helptext, the guidiance of the Metadata Working Group, all the Copmmunity- Articles about CSV and XMP::dc\description\Description\0,
I have switched and tested all nessesary Options like IIM IPTC/EXIF/GPS neu erstellen etc....

I know, a lot has changed in XMP and Metadatasurrounding.
Tell me, it's not possible and i will buy some other Software. But i have to deliver - no Chance. I was able to do this for years- with the 3.4
Actually i try IMATCH 2018.8.7.

My Question:
Is it possible to write formated Text by CSV or Textfile into the XMP::dc\description\Description\0 tag? How must the CSV look like in this spoecial case?
Normaly i use the ";" as tag-limiter and "|" as repeatable tag like keywords. This works fine.

Thanks & best regards
tom

[Mario]

(...) specialy for an XNView IPTC-Viewer

You can of course input any amount of text, with or without line breaks and special characters like German Umlauts directly in the Metadata Panel in IMatch. Maybe use Copy/Paste if the data is stored somewhere in Excel or whatever.

If you prepare the data in CSV files for some reason, you can import data from CSV files into metadata using the CSV Import App in the Import & Export panel (View menu > Panels > Import & Export).

The App maps fields from CSV to any metadata tag you like. One of the fields in the CSV must be the fully qualified (with path) file name. The other fields can be mapped to one or more metadata tags.
The app can handle regular and repeatable tags (e.g. keywords).

But what you need is really special. This sounds like you have to combine multiple columns from your CSV file in a special way and then store the result in a metadata tag. This is quite special and goes beyond what the CSV Import App can or needs to do.

If you can prepare your CSV file so that it already contains all this data combined into one column, you can use the CSV Import App straight away.
But the CSV Importer has no feature to 'combine' multiple columns from the CSV file into one metadata tag, with formatting.

Luckily IMatch is one of the few solutions on the market which can be extended to handle such special tasks. This makes it so versatile.

It is fairly easy to write a small app which does what you need.
The embedded IMWS in IMatch makes it easy to read text files. JavaScript has powerful features to parse text. There are also functions in IMWS to quickly locate files by their name and other methods to write metadata. The CSV Import App does all this, and comes in source code, with detailed comments. There is also the Metadata sample App which demonstrates how to read and write metadata.

[tomzappa]

Hello Mario,
Thanks for your reply.

For me now it is allmost shure, that the CSV-Import App will not fix my Problem. I have found out, that i need to fill in a chr$(13)+chr$(10)-tag between the tags.
This was possible within my basicscript in IMatch 3.6 - but not in the CSV. Bad Luck. For the BasicTools in 3.6 i guess i have lost some of the references - and now i will receive only type mismatch :-(  No Idea.
Doing by Hand is also no Solution: I have about 100- 1500 Pics per Job to handle - once or twice a year. The Data comes out of an GIS - and every Pic has different Text. Hurra.

I have read about the asynchronos processing, new apps, javascripts and have understood nothing. Maybe i have to grow up and learn something new?

Thank you
tom

[Mario]

chr$(13)+chr$(10) was used in Basic to insert a carriage return and linefeed. Of course there are similar was to do this in JavaScript.

But the CSV Import App does not have the features you have apparently implemented in a BASIC script for IMatch 3. But BASIC is dead and IMatch no longer supports the old Sax Basic or WinWrap basic interfaces. This has been dropped for good finally with IMatch 2017. And good riddance.

IMatch now uses an integrated web service to provide programmers with access to the IMatch database. This gives programmers a free choice of programming languages. For apps running inside IMatch JavaScript and HTML are used. But you can also access the embedded web server in IMatch from other modern programming languages like Python, C#, Go, Java, Koitlin, whatever. IMWS is language agnostic and any language which can access web services will do.

It is fairly easy to write a Python script or even an IMatch app which does what you need (unless there are other issues, like mass data handling or complex rules).
Any programmer with JavaScript experience could bring this together in a couple of hours. Or maybe in Python or similar.
If you use this for commercial purposes paying somebody to write this for you should be feasible. In that case you can also contact me via support email address and we can talk about time frame and cost. I occasionally write such apps for commercial clients or help their developers.

Or maybe one of the script geeks in the Scripting & Apps board has some time and can help you out.
It's surely fun to write a script or app that does what you need. But it will be useful only for you and of very little benefit for other IMatch users.

[tomzappa]

Good Morning Mario,
i have a little Expierence in python for GIS-Efforts and so i will do my best. The first steps will be a bit difficult.
But java is completely strange for me and i'm not a programmer but a User with some special Jobs.

On the other Hand i guess , it is not so special to have formated Text in the description-Tag. It's a Question of readebility.
Have a nice weekend

tom

[Mario]

But java is completely strange for me and i'm not a programmer but a User with some special Jobs.

IMatch uses by default JavaScript, which is the #1 web language. Very common, easy to get help and samples for everthing.

On the other Hand i guess , it is not so special to have formated Text in the description-Tag.

The description tag takes any kind of text. You can edit your descriptions directly in IMatch in the Metadata panel. Very comfortable editor, even with spell-checking. Or you use the Notes App. Or copy/paste it from other applications.

But you need to import text from a CSV file and combining multiple columns in that process to form a description with a specific formatting.
This is rather special and I doubt that many users will ever need something like that.

But IMatch has been designed to be extensible and to support scripts and apps written by users. As I said, for a JavaScript programmer or a Python programmer with a bit of experience, writing a script that does this is easy to do.

There is even a XMP Toolkit for Python which allows you to write XMP metadata directly. So you can write a small self-sustained Python script which reads your CSV data (there are Python libs for that too) and then writens whatever you need into XMP. Do this before you index your files in IMatch and all will be nice and shiny.

[hluxem]

Hello Tom,

It's not really clear to me what exactly you want to do, but a while ago I thought I need to write an app and started on it. At some point I realized that I maybe able to use the export and import app and was surprised how flexible these function are. I found a solution for my problem in less than one hour. There are a lot of functions in Excel to manipulate, replace or substitute text. Most of them unknown to me, but there is a ton of information and help available online.

I just did a fast test, and was able to import your full sample text into the xmp description. What is not clear to me is, in what form you get this data and if you want to have them with the line breaks,ö,ä,ü or without. In general, I think whatever formatting you need, it's best done in Excel. Even if the data is already in the file, it maybe worth to export to Excel, manipulate and reimport.
I copied your sample formatted text in an excel sheet. There are 2 ways of doing this, if you just highlight a cell you get multiple rows which is not useful for this task, if you highlight a cell and click F2 or click in the field on top to edit the cell and then paste the content, everything is copied to this cell.
To use the cvs Import app you need the file names and path, I use the text export app first with this argument {File.FullName}. That gives you a file you can open in Excel.
Then I copied your sample text in the second column. Not sure if needed for this, but I usually just highlight the cells needed, create a new file and copy the data. I then save the file as CSV UTF-8. Next I run the csv importer app and select this file. I choose "," as column delimiter, CRLF as record delimiter and say don't splitt for repeatable tags. I suggest to use column names in the first row, just make sure the checkmark is set correctly. Then select where you want to import, I choose Metadata tags. If the first field contains the full filename, set the checkmark.
Select the metadata tag for the other fiel, I choose XMP::dc\description\Description\0 as that seems to be what you need.

If you post some sample data you need to start with and a sample file how you want it in the description field I can see if I'm able to create a template you can use.

Heiner

[tomzappa]

Hello Heiner,
thanks for your reply.
The Idea is, to have a formated Outprint of some special Data in, and this is a mustbe, the XMP::dc\description\Description\0 -Tag (see the Attachment - i hope it has worked).
The Data results out of some complex calculations and modifications inside GIS-System and must be easily readable - which means: every single Information in an extra Line. The Targetsystem is XNView but i guess, most of the IPTC-Viewer have more or less the same size.

I have tried the CSV-Importer with the "|" as quasi linefeed (as i did it for the Keywords) and all the data was written to the Metadatatags more or less without Problem - in the "description" with the "|" as part of the Text - and without formating. I wonder why there is a linfeed-Handler for the Keywords - but not for the Discription. It's maybe a Question of internal dataformat? For me as a GIS-Specialist it is normal to have less special cases as possible and so every Tag should work in the same way. Maybe i should start working with the MetaData Working Group ;-)

But actually i have the solution:

I have found the Metadata.bas in IMatch 5, which was easy to modify - just a completly other direction than the CSV-Importer. In the GIS i was able to write Textfiles per Image, read it with the modified Metadata.bas in a loop and write it directly inside the Metadata-Tags. For the XMP::dc\description\Description\0 -Tag i was easily able to change the "|" into Chr$(13)+Chr$(10) and there where no more Problems with ä,ö,ü- Letters. Until now i don't know what the Problem with the utf-8 in the CSV-Importer was? But this was also in the IMatch 5 Version and not in IMatch 2018. My Problem was, to write something like Chr$(13)+Chr$(10) in a CSV, which is directly interpreted as linefeed and in fact as a new Datastring. So the context was damaged...

On this Point i will thank you and Mario for the Input.
When i have some time left, i will start checking out the actuall possibilities in IMatch 2018. In 2010 i have started working with IMatch because of the BasicScripter and this was for shure the only reason. I'm no professionel Photographer but Ingenieur with a lot of pics which have to be prepaired for people, far away from Photographical Efforts, who need the sourounding Informations in an easy Way.

Have a good time
tom

[Mario]

The | is interpreted for hierarchical keywords as a level separator. It has nothing to do with line feeds or anything. Not sure what you mean by that.

IMatch of course interprets carriage return / linefeed (0x13/0x10) in tags and wraps the lines in the Metadata Panel and elsewhere.
The description tag already uses a multi-line field in the Default Metadata Panel layout. It will show line breaks and also scroll bars as needed.

Inserting a | in an imported text of course does not do anything for line wrapping. The | has no special meaning, except for hierarchical keywords. For all other tags it is just normal text.
And you cannot include CRLF in CSV files, because the CSV standard uses CRLF as the record delimiter. Or only LF on Linux/Mac.


When I understand you correctly, your CSV files already has the entire text you want to import into the description tag in one column. And the only problem is the formatting with line breaks.

This can be easily handled.

To make the generic CSV import in IMatch work for you, just fork it.
Close IMatch. Then create a copy of the app in a new folder below the user scripts folder: C:\ProgramData\photools.com\IMatch6\webroot\user.
Open the app.json of the copy in your text editor (I recommend the free Visual Studio Code from Microsoft) and change the "id" to a unique id and also the "name" and "description". Maybe also change the icon file to something different. This allows you to tell which CSV Importer is which.

When you now open the Import & Export panel, your custom CSV Import will show up in the panel. It will work exactly as the original CSV Importer. Now, to customize your CSV importer:

The only code change you need to make is in line 1034:

Code Select Expand


// This is the value
value : cols[f.index]


Here the value sent to the server for the metadata update is filled from the data in a column imported from the CSV.
If you, for example, include | in your text and want this to be translated to a CRLF, change the line to

Code Select Expand


value : cols[f.index].replace(/\|/g,'\r\n')


This replaces | in the imported text with CRLF before sending importing it into the metadata.
The complex syntax /\|/g is just the way for JavaScript to do a global replace in a string. And since | is a special character in a regular expression, it needs to be escaped with a \ to give it its real meaning b back.

If you use another character in your CSV to indicate a line feed (e.g., __crlf__ the replace would look like:

Code Select Expand


value : cols[f.index].replace(/__crlf__/g,'\r\n')


If you need to make multiple replacements, e.g. you also want to replace __tab__ to a tabulator, you can chain multiple replaces:

value : cols[f.index].replace(/__crlf__/g,'\r\n').replace(/__tab__/g,'\t')

This should take only a few minutes and you can then directly import your CSV files into IMatch.
Note: DO NOT modify the original CSV import. Your changes will be wiped out when the next IMatch update is installed.

[tomzappa]

This sounds realy great. I will test it.
tom