File Rename Tool and TimeZone/DST

[leifel]

IMatch version 2023.4.6 (released yesterday I think).

Using Tools->Renamer... to rename a number of image files (Nikon D300 NEF-files) from July 2010.
Main variable in target filename template is "Date in YYY-MM-DD Format".

But, - the filenames created ends up being one hour (+1) offset from the correct time in the image metadata!? I have tested around in my image database and have concluded that this error happens on images that at taken during DST (Daylight Saving Time).

I have tried to play with the different options in "Preferences->Metadata2->Date and Time->File.DateTime Maping Mode" but none of them seem to have any effect on the rename-tool.

I have used this tool to rename image files many times during the years, also with NEF-files from the same camera, but I have never experienced this problem until now. However, it is a while since I did it previously. Probably at least 5-6 months, maybe more. I therefore suspect it might be a bug introduced in one of the more recent IMatch versions. Don't know which one.

Anyone else experiencing this problem? Or is it something in my settings that I need to adjust to make it work correctly?

[Mario]

I have tried to play with the different options in "Preferences->Metadata2->Date and Time->File.DateTime Maping Mode" but none of them seem to have any effect on the rename-tool.

This settings affect how IMatch maps the date and time found in the file to UTC when the image file has no time time-zone offsets. This is unrelated to the Renamer. The Renamer uses the resulting global File.DateTime, like all features in IMatch which work with date and time.

For example: A file reports these values for File.DateTime after import:

Code Select Expand

UTC: 29.09.2020 10:47:58
Local: 29.09.2020 12:47:58+02:00
Original: 29.09.2020 15:47:58+05:00
The LocalDate is calculated based on UTC and the date of the image (applying local DST at that date). That makes +02:00 at my location (Germany). If the file would have been taken e.g. in November, the local time would be displayed as

Local: 29.11.2020 11:47:58+01:00

(since DST off and Germany is at UTC +01:00).

Renaming the file in the Renamer using Date in YY-MM-DD format and Time in HH-MM-SS format uses the local date and time and thus gives:

20-09-29 12-47-58

which matches all outputs using the local time for display (File Window layouts with the Date & Time (Local) format etc.
The Metadata Panel displays the original date & time: 29.09.2020 15:47:58+05:00 and so does the generic {File.DateTime} variable.

We've had very loooong discussions about date & time and the many things involved in the months before IMatch 2023 was released in June. Since 2023.1 IMatch stores File.DateTime in UTC and provides additional features and variables to access both the original date & time, local date & time, the time zone offset used to calculate UTC from the time stamps found in the file, how to apply DST etc. IMatch e.g. uses the Windows time zone database to convert from UTC to local time, applying the local DST.

IMatch does not yet perform cloud-based lookups of DST based on GPS coordinates and/or country recorded in the image. This appears a bit of overkill except for the deeply involved users and there are ZIP requests (but a Python script that does this is available in the Apps area, I believe).

See How IMatch uses Date and Time Information for more detailed info.

[thrinn]

Main variable in target filename template is "Date in YYY-MM-DD Format".

I suppose with "variable" you mean in fact  the step in the processing instructions?
You might consider to use a "Text and variables" step instead of "Date in YYYY-MM-DD Format". This step type is much more flexible. You can use one of the many available options to choose between UTC and local time or whatever. And you can format the date and time to your requirements.
See especially this release note and also the IMatch 2023 migration topic in the help.

[Mario]

I pondered to add additional sets of Date & Time step for Original Date, Original Time, UTC Date, UTC Time plus various combinations of with/without time zone offset to the Renamer for a while while building IMatch 2023.

I feared that this would probably become overwhelming for users, given that we already have 27 (!) Date & Time steps with various formats. Scrolling down 81 Date & Time steps would probably not make a good UX. So I've decided to stick the what we had before, the "Local" Date & Time steps, since this is what most users need.

For all else, a Text & Variables step with the desired date & time variant (original, UTC, local) and the desired formatting will do the trick just nicely. Including things like calculations (math) timediff, 1/100 seconds etc.

In my experience, if a user needs "more" than the Date & Time steps and formats provided by the Renamer, using a variable is the easiest way to go. From neutral to country-specific formats, partial dates/times to explicit output of time zone offsets, all is possible.

[leifel]

Thank you for the answers!

I go for the "Text and Variables" with "{File.DateTime}". Then I get the raw original timestamp in the format "YYYY.MM.DD HH_MM_SS". But I want it in the format "YYY-MM-DD HH-MM-SS" and so I add two "Replace Text" steps. One with "Replace Text '.' with '-'" and one with "Replace Text '_' with '-'. However, only the first Replace Text step seem to work. The second one refuse to replace the underscore characters. I have tested with replacing other characters, and with "Regular Expression" checked. It works nice for all except underscores. How am I supposed to replace the underscores so I get the preferred time-stamp format?

Another question: I am a bit confused why the images takes in summertime gets the +1 hour offset when converting to "local time". I mean, my computer local time zone is the same as in which the images are taken, so why is it offset by one hour? Do this mean that if I do the same rename operation on the same image-files in the summer-time (instead of as now which is winter-time) I will not get the +1 hour offset? In that case I think there must be something wrong in how IMatch convert the time to local time. And if not, I don't understand why I get the +1 offset in the first place, since my local time zone is the same as of the image-files. Please explain

[Mario]

I recommend you just use the format function to format your date & time variable.
Very easy, many examples in the help: Date and Time Format
You can include separators and delimiters as you wish.

However, only the first Replace Text step seem to work.

Does not work is not really helpful. We need more info:

1. The steps you use and their order
2. The file name
3. The file date and time

Usually, the Renamer is right and the specific combination of file name, variable, steps and step order is causing the specific unexpected outcome.

Another question: I am a bit confused why the images takes in summertime gets the +1 hour offset when converting to "local time"

As I tried to explain in my post and in the How IMatch uses Date and Time Information (important read) extensive topic on this subject, IMatch does not "add" or "subtracts" anything by itself.
IMatch imports the time stamp from the image and converts it into UTC. When local time is requested, it converts it from UTC back into your local time, considering time zone offsets and DST at the date the image was taken. Depending on the date, this might add an hour or 30 minutes, depending on your DST settings.
If you don't like the local time, use UTC or the original time. IMatch gives you these options.

[leifel]

Very easy, many examples in the help: Date and Time Format

Reading that help got me to identify the problem with converting underscores. What happens is that IMatch get the time format as specified in my Windows Locale settings which is "YYYY.MM.DD HH:MM:SS". But since colons are invalid filename characters I guess IMatch convert this into "YYYY.MM.DD HH_MM_SS". When I changed my "Replace Text '_' with '-'" to instead change ':' to '-' it works! This probably means that there might be a bug in your code so the user specified text replacement is done on the time formatted by Windows instead of on the time formatted with you code that already replaced the colons with underscores. I assume this is not intentional.

[Mario]

The File.DateTime variable by default formats date and time using the local format as configured by the user.

Again, without knowing your original file name, the date of the file, the date & time format your system uses, all the steps and their order, we cannot analyze or suggest anything.

This probably means that there might be a bug

Always a possibility but we don't have enough info from you to do anything.

The Renamer processes steps in the order in which they appear, and each step sees the "output" of the previous step.
The Renamer automatically sanitizes file names according to the rules given in the tech spec Naming Files, Paths, and Namespaces published by Microsoft after processing all steps.

[leifel]

Here are some screen dumps that shows all you need, I think.
First, this is my locale settings on Windows:



And here is the IMatch Renamer-tool as of when I am unable to convert underscore characters (the problematic step is the selected/third one):
{REPEATING PROBLEMS WITH YOUR FORUM-SERVER TELLING ME THAT MY POST HERE IS IN ERROR BECAUSE IT EXCEEDS THE MAXIMUM ALLOWED LENGTH OF 60000 CHARACTERS - so I give up and put this and the next image in a separate post. This is image A}

However, adjusting the third step to convert from ':' instead of '_' makes it work, even though the Source File contains no colons in the first place. My guess is that the colons comes from the time stamp as formatted by Windows according to my Windows Locale settings (see the first screenshot):
{See image B in separate post}

Am I clear now? Let me know if you need anything more to be able to test and reproduce.

[leifel]

My Windows locale settings:

[leifel]

No, it looks like your forum server does not accept any images. So I give up. Have a nice weekend.

[Mario]

No, it looks like your forum server does not accept any images. So I give up. Have a nice weekend.

Use the click or drag files here... attach option at the bottom of the post to attach JPG files or ZIP files.
If you use the Quick Reply, note that the attach option becomes available when you click on the Preview button.

After processing your steps, the Renamer replaces characters not allowed in file names like : | \ etc. with _
When I use a variable and format the time as HH:MM:SS and then add a replace step that replaces : with # I get



which is the expected result. IMatch sanitizes the file name after processing each step so the : are available for the replace text step and end up as #.

{REPEATING PROBLEMS WITH YOUR FORUM-SERVER TELLING ME THAT MY POST HERE IS IN ERROR BECAUSE IT EXCEEDS THE MAXIMUM ALLOWED LENGTH OF 60000 CHARACTERS

Sound you tried to drag large amounts of text or images directly into the post perhaps?
This is not accepted.