MP4-files

Started by rienvanham, September 11, 2021, 10:44:09 AM

Previous topic - Next topic

rienvanham

Hi Mario,

Wer have a large amount of MP4-files (created by GoPro), all recorded during our bicycle-holidays. I want to index them by iMatch but that takes a huge amount of time. At this moment iMatch is already busy for more then 3 hours to index around 1000 files. The files are stored on a NAS. With the Process Monitor I can see that iMatch is reading the whole file, not only the tags. Why is that? Can I do something to avoid this? Why should it read all frames?

Thanks in advance for your answer.

Rien.

Mario

#1
When IMatch indexes a file it does many operations, it never "reads only the tags".
IMatch creates a checksum of the file, extracts visual query data, lets ExifTool read and merge the metadata, extracts several thumbnails etc. That's a lot of work.

Doing all that over a network and from a NAS is the worst-possible case, performance-wise. Worse even, when you do that over Wi-Fi and not a cable.
Maybe even your virus checker kicking in, scanning the whole file during transfer?

How long does IMatch need per file (log file)?
How large are your video files? 100MB? 2GB? each?

rienvanham

#2
Hi Mario,

I indeed saw that the virusscan (Bitdefender) was doing a lot of work with the files and I disabled it temporary. Now it's going a lot faster. Most of the files are H.265 4K-files (25 or 50 fps) and the size is around 4 GB each.
I know that indexing from a NAS is (much) slower then from a SSD but the files are mostly there for backup. Off course I do the editingwork (Davinci Resolve Studio) from my SSD's.

I think I have to be patient and leave the computer on as long as indexing is not ready.

Rien.

Mario

If your AV software intercepts, this will cause a drop in performance indeed.

If your network is saturated (100% utilization) you may improve performance for this step by reducing the number of parallel threads IMatch uses to ingest files.
See Process Control for Slow Media (CD, DVD, NAS ...)
Maybe set both values to 4 or 2 for a start and see how this improves things.
You can set it back to 0 for processing files from your local disk afterwards.

rienvanham

Thanks Mario! I will follow your suggestions.