[WF] CSV Import

[cg]

The CSV import does not seem to work for filenames with the ~ character in the name.

I tried quoting the filename in the CSV file, tried escaping the ~, but I couldn't get the import to work for those files.

Otherwise, the CSV import works great, and is lightning fast, even with over 1000 files at a time.

Thanks!

[Mario]

Works just fine here. I can use ~ in both the folder and the file name, import works.
I've exported file name, title, description with the text exporter.
Wiped the data from the files.
Imported with CSV Importer. All data back. File names with ~ at the start or middle, folder name with a ~ at the start.
I've used "" to wrap values with the export like so:

Code Select Expand

Name:"{File.FullName}"
Title:"{File.MD.title}"
Desc:"{File.MD.description}"
Any other details?

[thrinn]

Is it possible that the OP refers to the old 8.3 "short filenames" in Windows? Would the CSV import work if the CSV contained 8.3 filenames? I think IMatch always uses the "real" (long) filename, right?

[Mario]

Is it possible that the OP refers to the old 8.3 "short filenames" in Windows? Would the CSV import work if the CSV contained 8.3 filenames? I think IMatch always uses the "real" (long) filename, right?

Good point!

IMatch does not perform conversions of the shortened 8.3 file names in the CSV importer. Modern Windows versions also don't support the 8.3 shortening anymore by default.

[cg]

I'm running the CSV import using the "Run" button in the Import & Export panel:

Column delimiter: ;
Record Delimiter: CRLF
Split Repeatable Tags: Don't split

Import into: Attribute Set 'Content' (Per-File)

Column 1: File name
Column 2: <selected from pull-down attribute menu>

Code Select Expand

H:\CG Picture Library 2\2022\from kevin 2022\PXL_20220220_010253541~2.jpg;The image shows a man standing in front of a rocky hillside, wearing a blue shirt, black jacket, and glasses. The sky is a mix of blue and white, and there are a few trees in the background. 
H:\CG Picture Library 2\2022\from kevin 2022\PXL_20220220_010253541-2.jpg;The image shows a man standing in front of a rocky hillside, wearing a blue shirt, black jacket, and glasses. The sky is a mix of blue and white, and there are a few trees in the background. 
H:\CG Picture Library 2\2022\from kevin 2022\PXL_20220220_0102535412.jpg;The image shows a man standing in front of a rocky hillside, wearing a blue shirt, black jacket, and glasses. The sky is a mix of blue and white, and there are a few trees in the background. 
H:\CG Picture Library 2\2022\from kevin 2023\PXL_20220220010253541-2.jpg;The image shows a man standing in front of a rocky hillside, wearing a blue shirt, black jacket, and glasses. The sky is a mix of blue and white, and there are a few trees in the background. 
The attribute is assigned to files 2-4 above, but not the first one. I've tested this with other filenames and files with a ~ don't seem to work, though the results say it successfully imported all of the attributes.

Please let me know if you need any other information.
Thank you!

[Mario]

I've tested this.

Ive used the four rows provided and saved them to a CSV file.
Then I've recreated your folder structure on my E: disk and added 4 image files with the names from your CSV.
Then I imported the file.
4 records were imported, 4 images now have a description.
Works perfectly.

Right-click the attachment and use "open link in new window" to see the attachment in full-size.

[thrinn]

The CSV Import seems to behave differently if the data is imported into attributes instead of metadata tags (which is what the OP did!). I used the CSV cg provided and added another line with a file in my test database.
So my CSV file looked like this:

Code Select Expand

H:\CG Picture Library 2\2022\from kevin 2022\PXL_20220220_010253541~2.jpg;The image shows a man standing in front of a rocky hillside, wearing a blue shirt, black jacket, and glasses. The sky is a mix of blue and white, and there are a few trees in the background.
H:\CG Picture Library 2\2022\from kevin 2022\PXL_20220220_010253541-2.jpg;The image shows a man standing in front of a rocky hillside, wearing a blue shirt, black jacket, and glasses. The sky is a mix of blue and white, and there are a few trees in the background.
H:\CG Picture Library 2\2022\from kevin 2022\PXL_20220220_0102535412.jpg;The image shows a man standing in front of a rocky hillside, wearing a blue shirt, black jacket, and glasses. The sky is a mix of blue and white, and there are a few trees in the background.
H:\CG Picture Library 2\2022\from kevin 2023\PXL_20220220010253541-2.jpg;The image shows a man standing in front of a rocky hillside, wearing a blue shirt, black jacket, and glasses. The sky is a mix of blue and white, and there are a few trees in the background. 
Z:\TEST\TEST ECP\Mit Tilde ~.JPG";The image shows a man standing in front of a rocky hillside, wearing a blue shirt, black jacket, and glasses. The sky is a mix of blue and white, and there are a few trees in the background. 
Note that I did not change the first 4 lines, therefore the CSV Import should show a "File not found" error for these. The last line, on the other hand, refers to an existing file and should work.

Now, when I imported this CSV into "Metadata tags", everything works as expected: 4 files not found, 1 imported correctly. Description is filled.



Now I tried to import into an attribute set (file based).
Result says 5 rows imported (note there is no complain about the non-existing files!), but in fact nothing is imported.


Hope this helps.

[Mario]

I've missed that Attribute bit, sorry.

Will look into this again next week when I'm back.

Until then I recommend to just remove the ~ from the file name and input CSV, This was always a "dangerous" character to use in file name since Windows used / used it when shortening file names to the legacy 8.3 format required by some older applications.

[Mario]

I see what the problem is.
The ~ is used as the escape parameter to tokenize string parameters in post and form data.
This is why the path cannot work.

This is a "Won't fix", sorry.

From the IMWS documentation of the path parameter (used by the Attribute import):

Code Select Expand

One or more Fully qualified paths for the files to retrieve. Separated by comma.
If path contains a comma, it must be escaped with a tilde ~ => ~, . It neither id nor path are given, all files in the database are returned.
To fix "your" bug I would have to change all that. Sorry, not worth the time and money.
Change the affected file names  in the database (easy to do with a replace step in the Renamer) and do a replace also in your CSV fiile.

I have added a "File names must not contain ~" to the CSV Import help tropic for the next release.

~ is a dangerous character to use in file names (because of the 8.3 feature in Windows) and hence ~ was chosen as the path separator in IMWS.

[Tveloso]

Could the tilde itself also be escaped, in the same manner as the comma?...(so "~," for paths containing a comma, and "~~" for paths containing a tilde)?...

[cg]

@Tveloso
Yes! That worked! Before posting, I had previously tried escaping the ~ with \ and /, to no avail. But apparently the ~ is itself the escape character in this context. Thanks!

@Mario
Thanks for checking into this. I now have two options, either escape the ~ or rename the files to remove it. Perhaps the help files could be updated to mention escaping it, instead.

Thank you both!

Edit: and thanks to @thrinn too!

[Mario]

Perhaps the help files could be updated to mention escaping it, instead.

Using ~ in file names is really not something that should be done. Windows used (and still uses) the ~ when it transforms a long file name (> 8 characters) into a short file name (8.3). And the ~ is used to indicate such shortened file names.

I don't know why you use ~ in your file names, or why escaping ~ with ~~ is better than just removing the ~.
I've added a not that file names should not use ~ to the CSV help.

I can extend that to also mention the ~~ escape trick in the CSV file. But given that ~ should not be used and since 2016 when the CSV import was added to IMatch you're the first user ever that ran into this problem, I don't think that there is a real need.

[sinus]

@Tveloso
Yes! That worked! Before posting, I had previously tried escaping the ~ with \ and /, to no avail. But apparently the ~ is itself the escape character in this context. Thanks!

@Mario
Thanks for checking into this. I now have two options, either escape the ~ or rename the files to remove it. Perhaps the help files could be updated to mention escaping it, instead.

Thank you both!

Edit: and thanks to @thrinn too!

I would clearly change the filename. Using the special character ~ in a filename, is not a good idea. From my point of view.