Allow relocating to a file already in the database, or detect the move correctly

[janb83]

If I move a file from one indexed folder to another indexed folder outside of IMatch (yes, yes, I know, frowned upon, but there are scenarios where this might be useful, necessary, or just happen by accident), IMatch does not handle this situation well at all:

Current behavior: The moved file is indexed as if it was a new file, with new application of indexing Metadata templates and whatever else is configured for indexing. The original file location is marked as off-line, but if you use the relocate command, you are not able to do this:

You cannot view this attachment.

I understand WHY this message appears, but there is no way out of this problem other than removing the indexed new location from the database first and then use the relocate command before IMatch indexes the new file again.

I would like to see either one of two behaviors, or maybe even both:

1) Instead of this error, IF the file was detected as a binary duplicate, offer the option to treat this as a move. The newly indexed file would be removed from the database, and the old one relocated. So, the cataloging data in IMatch of the OLD off-line file "wins". If a user wants the new cataloging etc. to "win", he could just remove the off-line file and be done. So in this dialog, only having the older catalog data win makes sense.

2) Ideally, the move would just be detected when indexing the moved file in the new location. To be honest, this is what I expected and I was surprised this is not the case. The files are binary equivalent after all.

[ubacher]

I tried this. But I moved, inside of IM, the (externally moved) file back to its original location and that worked.
I did not try the relocate.

So the solution I see is not to use the relocate command but to "undo" the external move within IM and then
move the file inside of IM. Will that solve you problem?

[Mario]

If I move a file from one indexed folder to another indexed folder outside of IMatch (yes, yes, I know, frowned upon, but there are scenarios where this might be useful, necessary, or just happen by accident), IMatch does not handle this situation well at all:

When you deliberately move a file from a folder indexed by IMatch to a folder not indexed by IMatch (if I catch your meaning correctly) you move it out of the database.

IMatch keeps the existing file record and data unless you manually perform a rescan of the containing folder.

This allows you to relocate the file to the new location if you moved the file intentionally and you want to include the new (previously unindexed) folder in your database.

If you made a mistake, you can just use Undo in Windows Explorer or manually move the file back in Windows Explorer.

After a short time (or during a manually triggered folder rescan) IMatch will detect the file and find the associated file record in the database. It will determine that the file is unchanged by looking at the last modified file system timestamp and, after performing some other checks for cache file, XMP sidecar file etc., it will bring the file back online without reloading metadata or rescanning.

In general, workflows like this should be avoided. Deliberately moving files outside of the folder hierarchy indexing by IMatch, moving or renaming files outside of IMatch etc. Having to use relocate should be an exception, e.g. after replacing a disk or when a SAMBA server in a NAS accidentally changes the media serial number of a share.

If you work with software that moves or renames files, it is better to index these files in IMatch when the other software is done with renaming and moving, e.g. when the files are final and ready to be archived.

[janb83]

If I move a file from one indexed folder to another indexed folder outside of IMatch (yes, yes, I know, frowned upon, but there are scenarios where this might be useful, necessary, or just happen by accident), IMatch does not handle this situation well at all:

When you deliberately move a file from a folder indexed by IMatch to a folder not indexed by IMatch (if I catch your meaning correctly) you move it out of the database.

No, unfortunately you did not understand it correctly. Sorry about that, but I think I was quite clear, to quote myself "If I move a file from one indexed folder to another indexed folder". ubacher also understood it correctly.

In general, workflows like this should be avoided.

I find statements like this completely pointless. I am writing a feature request based on a need that exists because of an existing workflow. Replying, to paraphrase, "feature request makes no sense because your workflow is not what I had in mind when I programmed it, please change your workflow instead" is not helpful. People and institutions have workflows, that were not created out of thin air, but due to external conditions beyond the scope of the DAM alone. You can't ask them to change them, which might be technically impossible, as a default reply to everything. Please try to simply assess a feature request or bug report on its own merits.

No matter how the situation I described in the request was created, what is a fact is that IMatch ends up in a situation that cannot be resolved without several manual steps, and probably not at all if advanced file system monitoring is on.

Here's one scenario where this thing will occur, which might be common enough to warrant your time: Imagine the whole file archive is synced somewhere or resides on an external HDD, and you have a folder of photos in there which is a mess and needs sorting, on file level. Not tagging etc., just sorting into some subfolders or whatever. This needs to be done by someone else, or even yourself, but on a different PC than the IMatch installation. That's a perfectly normal need, and it can be done easily on the file system level. However, once you sync back, or reattach the HDD, IMatch will end up in the situation I described. The same file will exist twice in the DB. That's still somewhat OK, because IMatch, even for binary identical files, cannot really know if the added file is supposed to be a move or a copy. But, once the user actively tries to relocate to the new location, resolving the situation should be possible. I don't think this is an unusual or exotic expectation.

So the solution I see is not to use the relocate command but to "undo" the external move within IM and then
move the file inside of IM. Will that solve you problem?

Thanks. I will give that a try as it would at least help in situations affecting files within just one or a few folders. If, for some reason, it affects files across lots of different folders it will be hard to do.

When you tested this, did you check what happens in regards to IMatch metadata? Because with your solution I would expect it to use the data of the "new" file, unfortunately
Thanks. I will give it a try, but

[Mario]

If you move a file in IMatch from one folder to another, it will not become offline.
This is why I misunderstood your question. I apologize.

Now I think you mean that you moved a file from one indexed folder to another index folder outside of IMatch, e.g. in Windows Explorer. This is not a good workflow. You should always use the powerful File Management features in IMatch to perform such tasks. This way IMatch can keep the database up-to-date automatically.

Let me know if I still didn't get it.

If you use a IMatch and still perform file operations like moving or renaming files in Windows Explorer or other software, you are effectively working against your DAM and you make your life unnecessarily hard - like having to use relocate afterwards.

I find statements like this completely pointless.

I see.
I was only trying to be helpful and give a new user some advice. Typically this is received well and users may reconsider their workflows and usage habits after switching to a DAM workflow. If you don't want advice or are offended by it, that's cool to.
I will remember this for the future.

[janb83]

I'm sorry, but you misunderstood me. I am not against advice at all. However, "change your workflow to circumvent problems with my software" is not really advice. It's essentially like Microsoft saying, well, use Linux if you don't like that Explorer has no dual-pane view. As I mentioned in the other topic, workflows do not exist in a vacuum, they are built around existing external conditions and it might be impossible to implement a change you suggest. Suggesting to change a workflow is not a solution to a user feature request, it's a convenient way to politely ignore it. But it's still ignoring it. Stating that one is the first user to request it is also meaningless, as every new feature request by definition is done the first time. A meaningful reply to a request that you have to decline is not "weird workflow" but an explanation of why this might be hard or impossible to achieve, pointing out logical errors in the request, offering an alternative way to solve the issue within IMatch(!) or even just say "let see if others also are interested" in this.

I fully understand that touching files outside a DAM at all is not ideal. However, as explained, there might simply be situations where one is forced to do it. This feature request is about avoiding a usability deadlock in these situations, which I think is a reasonable expectation. IMatch right now blocks itself from allowing the user to solve the issue by denying the relocate. I am curious: How is the relocate of a single file (NOT a folder) supposed to work? As we just discussed, one cannot relocate to a file in a folder already indexed. So the only option is to relocate to a folder not yet known to IMatch. However, what happens if I do that for a single file? IMatch is folder-based, so I assume it has to add and index the whole folder? So, let's say this affects two files out of 100 in the "old" folder. Once I relocate the first one, will IMatch add and index the "new" folder, thus creating the same unsolvable problem again for the second file?

Btw, just a sidenote. I'm not new to DAM and I try to avoid file-level operations outside of IMatch for any indexed files as much as I can. I have to be honest though, that is often painful, mainly for two reasons

1) No proper edit-mode sharing and collaboration. If I want my wife or whomever to sort some images, maybe delete some, rename some, because she took the photos, I either have to give her access to my Windows user and IMatch (and she has to understand it well enough), or she has to do stuff externally.

This is NOT a problem for me, as long as IMatch can properly handle these outside changes. Which it does, except for the move from one indexed folder to another indexed folder that I described. This is especially what makes this so annoying. Everything is there for a nice peaceful coexistence of DAM workflow and some external work... except for this particular issue.

2) No dual-pane view or anything similar. I get it, professional photographers maintaining their project work, they work a long time in the same folder, even on individual files, so it's not an issue for them. But if you have any kind of workflow that requires moving files regularly, this is a major pain point. It simply is much much faster to do this outside of IMatch unfortunately. Which is why I wrote that feature request for a movable Result Window.

One example I had just yesterday: I got a batch of new scans of photos which I (partially) already had 20-year-old scans of. So, I want to check which old files I can delete. For this, I need a side-by-side view
either of both folders with thumbnails, or of a full-size side-by-side view like the File Viewer for marking the duplicates one by one. Neither is possible within IMatch. I have to open one of the two folders in a second application. Simply using to Windows Explorer windows would honestly have been the fastest way, but I forced myself to at least use IMatch for the one side of the side-by-side.

[Mario]

Let's see how many Likes your feature request gets. If this is a problem faced by many users, I'll look into this.

[Mario]

I need a side-by-side view either of both folders with thumbnails, or of a full-size side-by-side view like the File Viewer for marking the duplicates one by one. Neither is possible within IMatch.

Maybe add the folder with the potential copies to IMatch.
Select both folders in the Media & Folders View to display all the files.
Use the "Visually Similar" sort profile which sorts files by their similarity.
This should show you dupe files next to each other.
Use a layout that displays date and file size or whatever you need to decide which file to keep.
Select all files and open the Viewer with <Enter>. Switch the Viewer to a layout with 2, 4 etc. files side-by-side to compare the files. Use <Del> to mark the files you don't want to keep.

If the folders are apart and you cannot select them easily in IMatch, assign the files to a category and then perform the other steps.

This allows you to compare file data, metadata and the files visually side-by-side in the Viewer to judge quality etc.

[sinus]

...
One example I had just yesterday: I got a batch of new scans of photos which I (partially) already had 20-year-old scans of. So, I want to check which old files I can delete. For this, I need a side-by-side view
either of both folders with thumbnails, or of a full-size side-by-side view like the File Viewer for marking the duplicates one by one. Neither is possible within IMatch. I have to open one of the two folders in a second application. Simply using to Windows Explorer windows would honestly have been the fastest way, but I forced myself to at least use IMatch for the one side of the side-by-side.

Why is this not possible with IMatch? 
Import the new scans, and then you have both (the old and the new one) and you can complare them easy with the IMatch-viewer. 

[janb83]

Thanks Mario, that is a very interesting approach, despite being limited to a single monitor. I will give it a try next time.

@sinus

As you can see by Mario's workaround, it doesn't seem to be simple to achieve anything like this. His approach could/should work, but it's not exactly a classical side-by-side. It might even work better, we will see. Why do you think this is easy with the viewer, can you explain how you would do this?

I think this is just a symptom of IMatch being heavily optimized for professional photographer project workflows, which I can understand, they are willing to purchase software. With additions like face recognition and events though, private users with "family albums" could become more important to IMatch than years ago I guess.

[sinus]

I do this simply by assigning the files to a category, this is very easy.
Then I go to the category and can compare the files in the Viewer, side by side.

Say, I have 20 files inside IMatch.
I assigne them to a category.

Then I import the new scanned files in IMatch.
I assigne them  to the same category.

Then I go to the category and select the files, what I want compare.

Maybe I so not understand something correct, but at least I do it on this way. 

[Mario]

but it's not exactly a classical side-by-side.

What do you mean by that?
The Viewer can display up to 8 images side-by-side, see Controlling the Number of Files to Display

Or do you refer to a side-by-side file system folder comparison like some File Managers offer?
For this IMatch cannot help.

But bringing both folders into the database and select them both, or assigning the files to compare to a category should work as well.

[janb83]

I do this simply by assigning the files to a category, this is very easy.
Then I go to the category and can compare the files in the Viewer, side by side.

Say, I have 20 files inside IMatch.
I assigne them to a category.

Then I import the new scanned files in IMatch.
I assigne them  to the same category.

Then I go to the category and select the files, what I want compare.

Maybe I so not understand something correct, but at least I do it on this way.

Yeah, this is similar to what Mario described, but his idea to sort by Visual Similarity is what could make this work. Without it, in your approach, unless Capture Dates or Filenames allow sorting the potential copies next to each other, it would not work for a lot of files.

Mario, yes, with classical side-by-side I mean exactly what you describe, a folder in one window, another folder in another window, the two windows side-by-side in roughly equal size and screen use. Ideally, one Window on the second monitor, the other on the main. I know IMatch does not support this, that's why I added the Feature Request about the Result Window being a free moving panel. That would allow exactly this side-by-side, and also lots of other interesting uses, better drag & drop etc. That such a view does not exist shows that file-system operations have not been of huge importance for IMatch. Which is fine, but then its hard to say at the same time that people should not do file-system operations outside of IMatch

Another kind of side-by-side which would also be useful is having two File Viewers side-by-side. Careful, this is not the same as having one viewer with a 2-file layout, for two reasons:
1) With two viewers side by side, you can put them on different monitors, allowing much larger view of the two images.
2) With two viewers side by side, you can navigate through their files independently. This allows plenty of different things, for example, comparing two files that would not be next to each other in a single viewer, no matter how you sort them.

Both of these modes of operation are very common and easy outside of IMatch, which makes it harder than necessary to only work within the DAM.

[Mario]

1) With two viewers side by side, you can put them on different monitors, allowing much larger view of the two images.

You can use the Viewer on monitor A and the Quick View panel on monitor B. The Quick View panel can be undocked like any other panel and you can maximize it to full screen via the dedicated button in the caption.

With two viewers side by side, you can put them on different monitors, allowing much larger view of the two images.

Not sure if this is a thing many users will need. I recommend to make a separate feature request so users can like this individually.

Setting the Viewer to show 2 files side-by-side and using dive-zoom or sync-zoom I can compare details of the two files at up to 800% zoom ratio instantly, with synchronized zoom and pan. Or even 3, 4 or 8 images. That's usually good enough for culling. I use 4K screens with my workstation PC and a 2560 pixel screen for my laptop. Works well on both.

[janb83]

You can use the Viewer on monitor A and the Quick View panel on monitor B. The Quick View panel can be undocked like any other panel and you can maximize it to full screen via the dedicated button in the caption.

Interesting. I ruled this option out already because it seemed to me that the Quick View panel was always in sync with the File/Folder View and the File Viewer. It seems this is wrong, somewhat. I am indeed able to navigate two separate sets of files in one Quick View and one File View ... mostly. There are some problems, unfortunately:
1) You first need to open the File Viewer, because if you first open the Quick View you still need to navigate in the Folder View and this will change the Quick View to the wrong file (and folder!).
2) It only works if I put the Quick View on the same monitor as the main window, and the File View on the other. Otherwise using the Quick View will put the main window in focus, hiding the File View.
3) The Quick View is noticeably slower navigating between files
4) The Quick View has no context menu etc. to set flags or bookmarks, and do other things.

But yes, for certain tasks this could work, so thanks for the idea.

[ubacher]

I consider the issue of comparing files within as sufficiently explained.

Back to the problem of recovering from files which were moved externally:

The following will help if all file have been moved to the same directory:

1. In IM display all the files which have become off-line (filter: File Properties)
Give them a bookmark so you can find them later!
2. Select these files and, using Text Export, create the following windows batch files

Code Select Expand

$MOVE <Dir where moved files are>{File.NameExt} {File.FullName}
Name it, for instance, MoveBack.bat

Execute the batch file by double-clicking it.

Before the next step Force a rescan of the directory where the files were.

Now, in IM,  you can select the files you bookmarked and move them from within IM.

If you hav moved files into many different directories you are, pardon the slang, "shit out of luck"

NB: I assume basic familiarity with windows batch files. I did not try the above code so there might
be small changes required.