Trouble updating files in a folder

[Damit]

If you have a folder that you have imported into IMatch and you edit a photo or video and whant to import that edit into IMatch, what is the best way to do it?
I rotated a video making another copy.  I asked I match to rescan the folder, but it would not import the new file.  So, I reimported the folder, which was inside of another folder, and the whole parent folder was copied again in the folder tree, so now I had two copies of the parent folder, on with the new file and one without it.

I thought rescanning was the proper way to bring in new files.  IMatch was even showing that the file needed updating, even after I asked it to rescan it.

So I reimported the folder, actually twice, as I did not realize that the parent folder was duplicated and that folder had the child folder with the new file and I somehow created the condition where all the files were duplicated, but only within IMatch.  I rescanned that folder and the duplicates remain. 

I sent a screen shot to support email.

[Mario]

So, I reimported the folder, which was inside of another folder, and the whole parent folder was copied again in the folder tree, s

This cannot happen. You cannot add the same folder twice into IMatch, unless you

a) change the media serial number of the media containing the foldert

or

a) You rename the folder outside of IMatch and then
b) instead or using relocate to tell IMatch that you have renamed the folder you add the folder again with its new name.

That being said, when you create new files in a folder managed by IMatch or edit files in such a folder in an external application, IMatch will pick up the new and modified files after about 10 seconds the last file system operation in that folder stopped. This delay deals with the fact that many applications save files into a temporary file in the folder and when this worked, delete the original file and rename the temporary file to the original file name.

This works flawlessly for all other users and myself, since we all edit our files in other applications and create versions or derivative images and files, which IMatch automatically picks up, versions and processes. I don't see why this should not work for you.

Always include a logfile (log file) since this will show us exactly what you did, which folders you added, if IMatch detected new and updated files in the file systems, if Windows sent messages about changes you did to the file system and what else operations you performed that caused all this mess.

Check under Edit > Preferences > Background Processing that the Indexing option is set to the default  "Import new folders and files automatically" and that it was not accidentally changed.

I've offered you a refund for IMatch in another thread because I think that IMatch is not the right software for you and that you would be happier with another DAM. You did not comment on that so I offer it again.

[Damit]

Thank you for the refund offer and I will take it if I decide that this software is broken. I would rather not have to request one, as a refund is a pittance compared to the amount of time and effort I have invested in learning and troubleshooting issues which have arisen using this program. That is why I am still trying to work with it. It took so long to learn everything, and the program has so much potential, if it can do what it claims; I would rather things just work. I am not doing anything that any software should be able to do. I was just trying to bring in a new file. I am working on a response to your comment on that other thread addressing issues you brought up.
I wish you would stop claiming things cannot happen when I have done and proven to you, by your own admission, things that you said are not possible with this program, the duplicate @keywords is a prime example. I wish I took a screen shot of the file and media tree showing the same exact folder listed twice before I corrected it, because they were both there. I tried to replicate the problem and IMatch did not duplicate the folders again, which, I guess, is good news. I am sending (too big to attach) the log file which I believe was in debugging mode. I do see some entries relating to what I was doing. Hopefully that will bring some insight.
However, I did send you a screen shot showing the file window with one level of hierarchy showing. You can clearly see the same files, with the same file names repeated. I have attached it here again. There is no difference I can discern from the files. I assume this falls into the category of things IMatch cannot do, maybe I am wrong Please answer: How is it possible to have two thumbnails pointing to the same file in the same folder?

[Mario]

Thank you for the refund offer and I will take it if I decide that this software is broken.

IMatch is not broken. Nonsense.
Why should it work so well for everybody except for you?

as a refund is a pittance compared to the amount of time and effort I have invested in learning and troubleshooting issues which have arisen using this program.

Sigh. Then why not cut your losses and save time.
You run continuously into problems no other IMatch user has ever experienced or reported. This causes a lot of expensive support cost.  Uninstall IMatch, ask my distributor for the refund. Start over will a new DAM that will hopefully work out of the box for you.

Actually, after looking at your customer record, I think have to withdraw my offer for a refund.
I thought you were a new user, working for IMatch for a couple of weeks.
But you have purchased a license for IMatch 2021 in November 2022 and upgraded to IMatch 2023 ion February 2024.

So you are using IMatch for for about 18 months now.
If IMatch is so bad and costs you so much time, why did you use it for so long, I wonder?

But if it is so unbearable for you for work with IMatch, I still stand to my suggestion.
Uninstall IMatch, contact my distributor for a refund of your IMatch 2023 upgrade and I will grant it.
I prefer to have only happy users. Better for my nerves and budet. And also for the users of course.

Please answer: How is it possible to have two thumbnails pointing to the same file in the same folder?

I can do that, since it has happened a small number of times over the past decade.
The only known reason for this to happen is:

1. The user moves or renames a folder outside of IMatch.

2. The user does not use relocate to tell IMatch the new folder name / location but instead adds the folder a second time to his database.

3. Recognizing his mistake, he relocates the newly added folder to the existing folder, or vice-versa.
This basically folds both folders into one.

The result of this series of operations is exactly what you show us in your screen shot: multiple entries for the same files in one folder.

To fix your mistake: Remove the folder from the database. Right-click and use Remove from database from the context menu. Do NOT confuse this with Delete folder. Write-back before when needed.
Then add the folder again.

[Damit]

Duplicate entries in the folder tree were created once again, and not in the set of circumstances you outlined. I have attached screenshots showing the duplicate entries in the folder tree. So it can happen, as I asserted. This time I had a folder containing video images named HD Video within a folder, named 20XX-08-XX-Jorge & XXXXXXXX 50th Anniversary (Using X here to replace identifying characters). I selected all the files and dragged them into the parent folder, all within IMatch, not using Mcrosoft's file explorer at all. IMatch then created a duplicate folder holding all the images and the HD Video. I have sent the log, which is too big to attach, by email already.

Please explain why this is happening and how to avoid this in the future.

IMatch is not broken. Nonsense.
Why should it work so well for everybody except for you?

That is what I am hoping and what I am requesting your support to figure out. It would be great if you provide solutions that resolve the matter. If you think I am doing something that challenges your program to work correctly, please let me know. AFAIK, I am not doing anything strange or special. I have not worked with these files in any other program except one (Shutter Encoder) to rotate videos. I have not used Lightroom in almost two weeks while trying to resolve issues here.

[Mario]

The folder names are unreadable and I cannot tell if they are the same.

IMatch cannot create "new" folders (unless you use the "New Folder" command). You can copy a folder under another folder using drag & drop (this creates a copy of the dragged folder as a child of the target folder) or you can move a folder under a new parent folder using drag & drop.

You say you selected files and dragged them to a folder? This does not create a new folder either. IMatch moves or copies the selected files to the folder you select as the target, depending on whether you hold down Ctrl or not.

The log file will show that, especially when you run it in debug mode and attach it. No log file, no way to know what happened during that session.

The first folder in your screen shot reports 453 files. The second reports 473.
If they are copies as you state, why would they have different file counts? What if one of the letters you wiped out actually differs, making the folders totally different folders...?

I cannot tell the names of these folders either I can only tell that they have a different number of files.

[sinus]

Such screenshots are not very helpful, when so many things are not viewable. 
And I wonder really, how can it be, to create a new folder, if you "dragged them into parent folder". 

I am not able to do something like this. 

[Damit]

The folder names are unreadable and I cannot tell if they are the same.
I cannot tell the names of these folders either I can only tell that they have a different number of files....The first folder in your screen shot reports 453 files. The second reports 473.

OK, I have sent the unmarked files to your email. You will see that they are the same folder, the duplicate one created by IMatch with the video files, and the original folder without the video files (since I moved them using drag and drop). You have all the information to show that the highlightd picture in each folder is the same exact one.  Luckily they at least have a different fileID, but other than that they are the same file.

Again, to detail specifically what I did (which should be in the log file, which I already sent to you before even posting this morning at 9:51am Eastern Standard Time. I think, from the size it is a debugging log):
In IMatch (not windows file explorer), I went into the HD Video folder, I selected all the files by pressing ctrl+A, and then I dragged the files to the parent folder. I did NOT hold Ctrl while doing so. I then deleted the Video HD folder as it had no files within it. When I went into the parent folder to work on the files, there were no video files therein. This caused concern that they were somehow deleted, but I then noticed that IMatch created the duplicate file right below it with the addition 20 video files. That is why one is 453 and the other 473. So, unfortunately, IMatch did create a new folder in a manner which you state is impossible.
Please explain why this happened and how I can avoid it happening in the future.

[Damit]

Such screenshots are not very helpful, when so many things are not viewable.
And I wonder really, how can it be, to create a new folder, if you "dragged them into parent folder".

I am not able to do something like this.

I really wish I could say the same.  I really wish I did not have these problems to report, but they exist.
Unfortunately, after considering it, I don't know if I want to share screenshots with so much information publicly.  I have no problem if Mario wants to share them with you. I was going to send them to you, but I don't think I can in a PM. But I am 100% sure they are the same file and that IMatch created it.  I don't say I am 100% sure about many things but this time I am sure.

I have created a set of videos. The view is limited but I believe shows enough to document the issue. Video one shows a series of photos being moved to its parent folder and you can see that the parent folder file count does not change, and that the child folder's file count does change. If I go into the parent folder, the files are not there.  BUT the parent folder now shows it needs refreshing. 
The second video shows me refreshing the folder.  After a delay in processing, you can see me scroll to the folder and you will see the duplicate folder that IMatch just created. There is proof that this is happening, and it is quite concerning.
The third video shows that this new folder does allow you to move the files normally between parent and child with no issues. I don't know why IMatch cannot do it to the original folder imported just days ago to IMatch and why it can do it with the new one but I highly suspect this has to do with how IMatch is set up to protect and keep existing data, because most of my issues seem to lead back to that, BUT I could be wrong.
Video 1
Video 2
Video 3

[Mario]

I have no idea how you managed what you did. I cannot reproduce it, no other user has ever reported something similar.

Right-click on both "dupe" folders and use the "Open in Windows Explorer..." command.
Which folders are opened?
Why do they have different number of files?

If you really managed to duplicate a folder without relocating, provide exact steps or, better, a screen recording (Windows has built-in tools for that, see https://www.microsoft.com/en-us/windows/learning-center/how-to-record-screen-windows-11) so we can exactly see what you do to produce this rather unique effect.

I see one MoveFiles operation caused by a drop. That's all.
I see one call folder AddFolder operation for the N:/[] Personal/...50th Anniversary/ from your screen shot. IMatch reports that this is a NEW folder. So no folder in the database.

Maybe this issue is caused by your unusual folder naming with the [] at front?
I'm not sure if this is even legal across networks or boundaries? Can you rename the folders to not start with [] and see if this fixes your problem? I have never seen this before, so who knows where inbetween the NAS, SAMBA, Windows networking, IMatch etc. this causes problems.

I've made some tests with folders on my NAS and Windows Server starting with [] but they work just fine.
No issues with dragging and dropping folders or files  from/to these folders.

Also, check the "media label" and "serial number" for these folders in the properties panel below the Media & Folders tree.
Are they identical? Sometimes SAMBA on NAS boxes changes the drive serial number for unknown reasons.
Since no other user ever reported this, chances are that the only PC affected by this is yours.

[Damit]

I have no idea how you managed what you did. I cannot reproduce it, no other user has ever reported something similar.

I am NOT trying to be snarky, but do you realize you have said this to me 3-4 times already, each with a different issue? I must be the most unlucky person when it comes to this program. But perhaps you may now accept that there is something seriously wrong that needs to be address and that all these issues may be related.

Right-click on both "dupe" folders and use the "Open in Windows Explorer..." command.
Which folders are opened?
Why do they have different number of files?

They both open the same exact folder in Windows Explorer. It does not even open another window, it goes straight to the same window because they ae duplicate entries pointing to the same folder.

If you really managed to duplicate a folder without relocating, provide exact steps or, better, a screen recording (Windows has built-in tools for that, see https://www.microsoft.com/en-us/windows/learning-center/how-to-record-screen-windows-11) so we can exactly see what you do to produce this rather unique effect.

I am ahead of you. I already posted the videos in the comment above, very shortly before you responded. If you need a full screenshot, I will try to find another folder and send it to you because, as I reported above, after the duplicate is created it behaves normally.

IMatch reports that this is a NEW folder. So no folder in the database.

Weird because it is there, right?

Maybe this issue is caused by your unusual folder naming with the [] at front?
I'm not sure if this is even legal across networks or boundaries?

It is legal and works; I have used it in networks for years. It is possible, though, that it may be the cause and like I said, I am willing to anything, including change this, to try to help resolve all my issues, even though this has been part of my file structure for decades. I just do this to keep certain folders grouped up top. It has worked for me, but I do admit it is unconventional.

Can you rename the folders to not start with [] and see if this fixes your problem? I have never seen this before, so who knows where inbetween the NAS, SAMBA, Windows networking, IMatch etc. this causes problems.
I've made some tests with folders on my NAS and Windows Server starting with [] but they work just fine.
No issues with dragging and dropping folders or files  from/to these folders.

Yes! Like I said, I am willing to do anything, within reason, even reinstall windows, to resolve this because this program has so much potential and I just want to enjoy using it. Hopefully we can figure this out together in cooperation. That is all I ask. I don't want a refund; I don't want to leave IMatch. I am trying to do things correctly and by the book as much as I can. I REALLY want this to work. I do not want to be reporting problems just singing the praises of this program, but unfortunately, I have a poor experience up to now. Hopefully this is the start of the resolution.

So, how do you want me to do this?  Within IMatch? Please be specific, I want to make sure I do this correctly.

[Mario]

Rename a folder on the server (remove the []) and then relocate in IMatch so IMatch knows about the name change.
What does the diagnosis report for this database?
When Windows opens the same folder, why is the file count in IMatch different?
What happens if you rescan the folder? Do the file counts update?
Since you work with IMatch for almost two years, why now so many issues? What has changed?
Nothing in how IMatch does drag and drop for folders has changed since IMatch 2021.
I shall look into your video somewhen next week, when a new time slot opens. Busy weekend ahead.

[Damit]

Rename a folder on the server (remove the []) and then relocate in IMatch so IMatch knows about the name change. Will do!
What does the diagnosis report for this database? Attached. I did not have errors or warnings in the previous database diagnosis prior to moving files. I have received and fixed similar warnings previously. The entity problem is usually fixed by removing and adding back the keywords associated with the person. I did this and the problems were fixed and now I have no errors. The second one is also attached.
When Windows opens the same folder, why is the file count in IMatch different?It is different for the 1st folder, which does not have the files because IMatch moved them to the second duplicate folder, so those files are omitted from the file count, and it is lower.  The second, duplicate folder shows the same number of files as Windows Explorer.
What happens if you rescan the folder? Do the file counts update? I am sending you a full screen recording, so you can see what happens. It is when you rescan the original folder that the second one appears. It acts like it is re-importing a folder, perhaps because it is because for some reason, it thinks the folder is not already in IMatch even though I moved the files within IMatch itself. If I rescan that original folder again, nothing happens.  The file number remains the same, with the moved files missing from the file count.  The file is still showing that it needs to be rescanned even thogh I have done that 4-5 times now.
Since you work with IMatch for almost two years, why now so many issues? What has changed?
Nothing in how IMatch does drag and drop for folders has changed since IMatch 2021.I don't know if I had dragged and dropped before the last couple of days, when I had these issues, but I cannot be sure. I think I was just adding folders, face tagging and adding keywords. I have had problems since the beginning, but I thought it was something I was doing, and I did not realize the breadth of the problem until I came back to my photo project after taking 6-8 months off to attend to other matters. But these issues, and similar ones, have been present since the start. I really wish I had changed something. I would love to pin point the problem
I shall look into your video somewhen next week, when a new time slot opens. Busy weekend ahead.No problem at all! I am just glad that you will take a look into it. THANK YOU!! I really hope we can resolve these issues.
 Update: I just realized, when updating the folders, that there is a difference between now and the last time I used I match.  Previously, I had imported the whole folder [] Personal (which is now being renamed to Person), and all the files within were listed under that parent folder. But, since I was having issues with IMatch and @Keywords I started anew, and I was adding folder by folder.  As such, for some reason, IMatch was not listing the folders under []Personal, but rather under N:\ which is the drive letter.  I don't know why IMatch did this.  I did think it was weird, but I figured that was the status quo, but perhaps not.  Could this be the cause of the problem?

[Damit]

Update 2: I renamed every folder that had a [] in it and relocated the folders.  IMatch did not behave as before, when it acted like it was bringing in new files and the process was rather quick. In addition, folders now behave as they should, and I can drag and drop without creating duplicates. There was much rejoicing!
However, all the folders under "Personal" are not listed under Personal, but still under N:\. This is fine, but I would much rather them represent what is in file explorer and have the files listed under Personal, if possible, without screwing anything up.

[Tveloso]

However, all the folders under "Personal" are not listed under Personal, but still under N:\. This is fine, but I would much rather them represent what is in file explorer and have the files listed under Personal, if possible, without screwing anything up.

When you index a folder in IMatch, that has one or more parent folders, you are expressly telling IMatch to exclude that Parent Structure, so that newly indexed folder will appear under the "root" of the drive node in the Media and Folders View.  That behavior is by design.

If that Personal folder is off the root of N, and contains only the folders that you have already indexed individually, adding that Parent folder to IMatch, should do nothing more than "insert" it into it's place in the M&F tree (mirroring the arrangement of the FileSystem).  You'll likely need to uncheck the option to include subFolders though, if there are other folders inside Personal, that you have not yet indexed, that you intend to cary on indexing on a folder-by-folder basis.

But it would be best to wait for Mario to give the definitive answer on this, and advise on the correct approach...

[Mario]

Tveloso is correct.

When you have a physical folder structure like

Personal
 |- images
 |- bla

and you drag "images" into IMatch, IMatch will only index "images" (and it's sub folders). This behavior is by design and enables users with complex folder structures to selectively pick the folders they want to manage in IMatch, even when they are part of a nested hierarchy.

IMatch did not behave as before, when it acted like it was bringing in new files and the process was rather quick. In addition, folders now behave as they should, and I can drag and drop without creating duplicates. There was much rejoicing!

Are you sure that the SAMBA instance running on your NAS has not changed the drive label / serial number of the simulated drives? Because that sometimes happens (depends on the NAS vendor, Linux version, SAMBA version) and then folders are suddenly offline or rescanning a folder causes IMatch to bring in the folder as new, since it comes from a "new" media.
That' why I've asked to check for the media label / serial number.

And, PLEASE, use the <Enter> key occasionally when posting.
Look at your last post on a smart phone screen to get the meaning of what I call wall of text. Unreadable posts make it much less likely that another community member will read it and spend time to help you or make suggestions.

[Damit]

Thank you, Tveloso for your suggestion and explanation. I will back up and try to add personal and see if I can add it back to the folder structure.

And, PLEASE, use the <Enter> key occasionally when posting.
Look at your last post on a smart phone screen to get the meaning of what I call wall of text. Unreadable posts make it much less likely that another community member will read it and spend time to help you or make suggestions.

Sorry, I was trying to make the posts shorter in screen length on a computer. I did not realize that it was making it unreadable. I will hit enter more.

Are you sure that the SAMBA instance running on your NAS has not changed the drive label / serial number of the simulated drives? Because that sometimes happens (depends on the NAS vendor, Linux version, SAMBA version) and then folders are suddenly offline or rescanning a folder causes IMatch to bring in the folder as new, since it comes from a "new" media.
That' why I've asked to check for the media label / serial number.

The files are not on a NAS, they are housed in the computer directly connected to the motherboard in the Nvme drive slots.  I have three 4TB Nvmes in a Windows Managed JBOD Raid configuration holding all my files for quick access and transfer times. I have the operating system on another Nvme and the database on another Nvme attached to a Pcie 4.0 x 4 slot.