Sync IMA database to IM database

[ben]

Hi Mario,

so far i am using IM on my laptop exclusively (images and database on SSD).
I would like to keep that. But i would also like to start enabling my family to watch the images.
So, i am thinking about setting up a NAS plus some kind of WIN computing stick to run IMA on it.

The question is, how do i keep the images and the database on my master device (=laptop) synchronized to the NAS?
I could set up some sync process on my laptop to sync all my images and the database as well to the NAS.
Would that work if i use the same folder structure?

Long term it would be perfekt to have some kind of synchronization feature. Then i could even do smaller changes on the database on the NAS (=tagging, descriptions, ..) and then sync it back to my master device (=laptop).

Thanks for your thoughts,
Ben

[Mario]

I would look into the RoboCopy command, which is part of Windows. This is a very powerful synchronization tool. You can use it to copy new and updated images and your database and settings from you laptop to your NAS with it.  Write a batch file or a PowerShell script to automate this completely. You can even use Powershell to shut-down IMatch WebServices to replace the database, then start it again.

Setup your IMatch database to use Portability Options so it automatically relocates the folders in the database to the folders used on the NAS.

If possible (for performance reasons) keep only the images on the NAS, but the database on the same device you are running IMatch WebServices on. The Stick PC or whatever. IMWS caches a lot so it does not hurt as much when the database has to be pulled over the network (from NAS storage). But the next generation of IMatch can also update metadata and categories in the browser, and then the database storage becomes more important again (performance-wise). IMatch databases are small and fit easily even on a tiny stick PC.

[Jingo]

Hi Ben - if you do a quick search, I have posted some info on here about how I use IMA with my TV and a small NUC PC hooked up to it.  It works pretty well!

Quick background Info:
 

• Imatch and IMA DB's are stored on my main computer.... I use SyncBackSE to copy my Imatch DB to a new IMA DB every evening at 2am... this is automated via the scheduler. 
• Images are also stored on the main computer but using a NAS would work well tool.
• Wireless NUC (Beelink M1) hooked up via HDMI to large screen LCD with Kuman wireless/trackpad combo.

If I used the setup more often.. then I would recommend Mario's suggestion of moving the DB to the device for speed... I tested wireless vs wired connection and suprisingly, it doesn't seem to make much of a difference.  I do need to let the previews / thumbs "preload" before starting any sort of viewing session... but overall, it works well.  Videos are not great for viewing this way... they just take too long to load (not an IMA fault... probably just a fault of the setup).  For movies, I just load them into a shared folder on my PC which my TV can find directly and display or I load them to my PLEX server and use Plex on the TV via the TIVO app.  Options...options.. options!!

Good luck!

[Mario]

I tested wireless vs wired connection and suprisingly, it doesn't seem to make much of a difference.

IMWS does a lot of caching to minimize database access. Still, the latencies of a network connection are much, much higher than accessing a local disk.

Videos are not great for viewing this way.

You are viewing videos on which device and which browser? In IMatch WebViewer? In the video layout?
The video rendition is performed by the embedded video routines in the browser you use. Not all browsers support all formats, not all browsers support all formats equally well.
IMatch WebViewer just lets the <video> HTML tag retrieve the video file. IMWS is not a specialized media streaming service or something. It just delivers the file as the browser requests it.

What problems do you encounter? No image? No audio? Stutter?...

[ben]

Great!!
Thanks for both of your answers. Seems that this works in general, so this will be a project next year.

I didn't know about the relocation options. :-)


So, if once IMA is able to change metadata, would there be a way to sync these database changes back to my laptop?
Even if only in the future.
Or will it always be a one direction syncing (from my laptop to the NAS/stick)

[Mario]

If you make changes to your database with multiple IMatch instances (or IMWS) you'll end up with different versions of your database. Only you can decide which is the correct one.
Of course you can also use RoboCopy or whatever to copy the modified database back to your other computer. I recommend to avoid to make changes to the copies of the 'same' database.

There will be no "merge" feature which can combine two databases into one. This is technically impossible. Or at least prone to errors, potential for data loss, unresolvable changes etc. Just not worth it.

[ben]

ok, thanks!
Will think about your first solution

[Jingo]

Videos are not great for viewing this way.

You are viewing videos on which device and which browser? In IMatch WebViewer? In the video layout?
The video rendition is performed by the embedded video routines in the browser you use. Not all browsers support all formats, not all browsers support all formats equally well.
IMatch WebViewer just lets the <video> HTML tag retrieve the video file. IMWS is not a specialized media streaming service or something. It just delivers the file as the browser requests it.

What problems do you encounter? No image? No audio? Stutter?...

Yes.. stutter was the main issue.  Need to try it again though... using Firefox but really didn't play all that much since video is not my main focus with IMatch.

[Mario]

Stutter is caused by too low transmission rate. Hardware. Network. Caching. Browser. ....

[Jingo]

Figured as much because they play relatively well using just the TV software directly from the folder (or USB stick). 

[Mario]

When you use watch Prime or Netflix etc. over a network, the client (software) and the server dynamically manage the bitrate (quality) to get a stutter-free video experience. This may mean that they go very low, reducing the quality etc.

IMWS of course cannot do this. When a browser requests a file (video / MP3) it sends the file over the network as fast as possible. The browser requests the data in packages, doing some caching internally etc. If the browser wants the data faster than the network can deliver, it may stop playing until more data has been received.

This is where dedicated clients / servers (Netflix, Prime, YouTube) come into play, which do dynamic bandwidth management between the server and the client software using proprietary protocols. This is not one of the the primary purposes of IMatch WebViewer. It requires re-encoding of video files and lots of stuff I don't want to learn about. Especially since this also involves license and patent fees and paying royalties to the MPEG.

[Rene Toepfer]

There will be no "merge" feature which can combine two databases into one. This is technically impossible. Or at least prone to errors, potential for data loss, unresolvable changes etc. Just not worth it.

Is this statement still valid?

[Mario]

Yes.
And since IMatch Anywhere allows users to modifiy metadata, categories and keywords for some time, you cannot use two separate databases anyway if you enable the FileLens.

[Rene Toepfer]

Thanks, then is everything fine with the configuration. I will use my main DB with IMWS for the other users and need to switch off IMWS if I do changes to the DB.

[Tveloso]

This is a very timely resurrection of this topic for me!

I was preparing to ask for some advice on an approach I have been considering, for using IMA with two Database instances (on two different machines), and wanted to try to find a topic I thought I had seen in the past, that discussed something similar.  I'm not sure if this is the specific topic I was remembering, but it's very much along the same lines.

I have not actually started using IMA yet, and I too use IMatch exclusively on my laptop, and would like to keep that arrangement for the most part.  So I have the following:

• I work with IMatch exclusively on my laptop
• Files reside on a NAS (that's their final destination)
• New Files are ingested to the local hard drive on my laptop
• After processing, a Renamer Preset renames, and moves the files onto the NAS

So for my current ongoing workflow, I wind up with files that will likely not need to be updated further, but many of the other files already on the NAS, do still need to be updated (Face Recognition, GeoCoding, Keywording, etc.)

Once I incorporate IMatch Anywhere, I was thinking that I would like to still keep my laptop as the primary means of ingesting new files, and even also of working with those older files. 

But since IMA can now also update Metadata, I was thinking that rather than using a "publishing model", or directly synching IMatch Databases between the two machines...that I would instead let the files themselves be the means by which the databases would be synched.  So I was thinking something like this:

• Set up a dedicated desktop PC to act as the Server that runs IMA (this becomes the "main" IMatch Database)
• The Laptop continues as before
• New Files moved to the NAS by the Laptop are picked up by the "main" IMatch Database on the "Server" PC
• Writebacks made via the Laptop for files already on the NAS are also picked up by the "Server" PC
• Updates made via IMA (after an IMatch write-back at the Server) are picked up by the Laptop

This of course means that I'll need to be careful to ensure that following an IMA session with family at the TV, that IMWS is shut down, in order for IMatch to perform the pending write-backs that were collected there, and likewise, any updates that were made via the Laptop are committed to the files, and that the "Server" has picked them up before a new IMA update session takes place.

I imagine that I must also keep the Thesaurus in synch between the two IMatch instances (I'm still building out my Thesaurus as I work on Keywording) - which I imagine must be done manually.

Does this sound like a reasonable approach?...or am I asking for trouble by doing this?