This is how I just lost a database (for a short while)

Started by Mario, April 16, 2025, 08:21:46 PM

Previous topic - Next topic

Mario

New external SSD. I stored an IMatch database on it.

Opened the database from the SSD and worked with it in IMatch.
While IMatch was in full-flight indexing new files, the new SSD suddenly disconnected. Whut?!
And, of course, Window was just writing data to the SSD when this happened. Worst case. And, by Murphy's law, most likely case.

After plugging the SSD out and in again, IMatch complained about database damage when trying to open the database.



Windows flushing file system buffers while a device is disconnecting or a power failure happens is the #1 recipe for database damage. And this is what happened here.

After researching and solving the problem (apparently, the SSD is not 'compatible' with Windows's power saving features for USB-connected devices), I replaced the database with the recent backup and was up and running again.

So, the lesson learned is: DO backups. OFTEN. KEEP THEM for a while.
(Thankfully, I've learned this lesson many years ago).
Even in 2025, (brand new) hardware can fail at inconvenient times for no reason. Just because.

IMatch databases are super-robust. But Windows unable to finish writing changes due to some external effect can damage the database file. Or, some hardware defect happens.

I'm still using databases initially created with IMatch 3 (back in 2010, 2012 or so?) and having backups saved me multiple times.

rolandgifford

Quote from: Mario on April 16, 2025, 08:21:46 PMWindows flushing file system buffers while a device is disconnecting or a power failure happens is the #1 recipe for database damage. And this is what happened here.

A more common cause of database "damage" is the accidental update. Removing/changing some details from a bunch of images and the like. This type of corruption can take a very long time to spot and can be very difficult to recover from even with backups.

Yes, regular backups kept for a long time are essential

Mario


QuoteA more common cause of database "damage" is the accidental update. Removing/changing some details from a bunch of images and the like. This type of corruption can take a very long time to spot and can be very difficult to recover from even with backups.
What? I have never seen this and no user ever reported this. Do you have any proof that this happened?
The database system used by IMatch is also used by companies like Apple, Microsoft, Google, Adobe and others. It is really, really robust.

Tveloso

I think rolandgifford may simply be highlighting that a User Error, that has gone unnoticed, will make it onto our backups, so recent backups will be no help there...and it's the older backups that can save us:
Quote from: rolandgifford on April 16, 2025, 08:45:04 PMYes, regular backups kept for a long time are essential

It's the question of which of our backups is the most valuable.

In the case of a drive failure, the most recent backup will of course be the most valuable one.  But in order to recover from a "far reaching" unintended update, that has effectively "corrupted" our data/files (even outside the context of IMatch), and has gone unnoticed for a period of time, a much older backup is where we must turn, to recover from our mistake.
--Tony