slowcats-lines in the log-file

[sinus]

Hi Mario
First: the change from the category-view or folder-view to the collections - view is much quicker, this is my feeling, not measured; thanks for that!

The new debug-lines-possibility with "slowcats" is interesting.

Although I am unsure, what the lines are saying. I thought, somewhere should be the name of such a slow category.
But I cannot find a name. But I can see, that it has in my case something to do with RegExFile.

I think, I do also know, what category it is, I guess, it means my category "a Master+Version+Kopie"
with this formula:

"@All" AND "@FileRegExp[_m]" OR  "@FileRegExp[_v]" OR  "@FileRegExp[_k]"

But should this category not be written somewhere in the slowcats-lines?
It is not that important to me, I want only ask, if I have understood something wrong?

Code Select Expand


03.02 17:56:34+    0 [10E8] 10  M>                    < 19 CIMCollection::InnerCalculate
03.02 17:56:34+    0 [10E8] 02  I>                   ##SLOWCAT '5 Analyse Details'
03.02 17:56:34+   31 [10E8] 10  M>                    > 19 CIMCatalog::RegExFile  'catalog\imcatalog.cpp(4704)'
03.02 17:56:34+  125 [10E8] 10  M>                    < 19 [125ms] CIMCatalog::RegExFile
03.02 17:56:34+    0 [10E8] 10  M>                    > 19 CIMCatalog::RegExFile  'catalog\imcatalog.cpp(4704)'
03.02 17:56:34+  109 [10E8] 10  M>                    < 19 [109ms] CIMCatalog::RegExFile
03.02 17:56:34+    0 [10E8] 10  M>                    > 19 CIMCatalog::RegExFile  'catalog\imcatalog.cpp(4704)'
03.02 17:56:34+  109 [10E8] 10  M>                    < 19 [109ms] CIMCatalog::RegExFile
03.02 17:56:35+  203 [10E8] 10  M>                    > 19 CIMCatalog::RegExFile  'catalog\imcatalog.cpp(4704)'
03.02 17:56:35+  125 [10E8] 10  M>                    < 19 [125ms] CIMCatalog::RegExFile
03.02 17:56:35+    0 [10E8] 10  M>                    > 19 CIMCatalog::RegExFile  'catalog\imcatalog.cpp(4704)'
03.02 17:56:35+  109 [10E8] 10  M>                    < 19 [109ms] CIMCatalog::RegExFile
03.02 17:56:35+    0 [10E8] 10  M>                    > 19 CIMCatalog::RegExFile  'catalog\imcatalog.cpp(4704)'
03.02 17:56:35+  109 [10E8] 10  M>                    < 19 [109ms] CIMCatalog::RegExFile
03.02 17:56:35+   31 [10E8] 10  M>                    > 19 CIMCatalog::RegExFile  'catalog\imcatalog.cpp(4704)'
03.02 17:56:35+  125 [10E8] 10  M>                    < 19 [125ms] CIMCatalog::RegExFile
03.02 17:56:35+    0 [10E8] 10  M>                    > 19 CIMCatalog::RegExFile  'catalog\imcatalog.cpp(4704)'
03.02 17:56:35+  109 [10E8] 10  M>                    < 19 [109ms] CIMCatalog::RegExFile
03.02 17:56:35+    0 [10E8] 10  M>                    > 19 CIMCatalog::RegExFile  'catalog\imcatalog.cpp(4704)'
03.02 17:56:35+  109 [10E8] 10  M>                    < 19 [109ms] CIMCatalog::RegExFile
03.02 17:56:35+   63 [10E8] 10  M>                   < 18 [14149ms #sl] CViewPaneGroupTreeCtrl::LoadLayout
03.02 17:56:35+    0 [10E8] 10  M>                   > 18 CViewPaneGroupTree::Load  'ViewPaneGroupTree.cpp(641)'

[Mario]

##SLOWCAT '5 Analyse Details'

The name of the category which takes more than 2000 ms (2s) to calculate is displayed directly after the ##SLOWCAT

"@All" AND "@FileRegExp[_m]" OR  "@FileRegExp[_v]" OR  "@FileRegExp[_k]"

This formula has to run a 3 regular expressions on every file in your database whenever it is calculated. That is slow for sure. Your database has, what, 180,000 files? So this formula needs to execute ~ 540,000 regular expressions on every calculation...

[sinus]

Thanks, Mario

You would not believe it, I have wondered about "5 Analyse Details" ... and not checked, that this IS actually a category of me! 

This category is very interesting to me, but I will check it and reduce the amount of formulas.

So your new lines in the logfile are very helpful! 

[Mario]

Before you start working on the formula, check the log file after a normal session, e.g. after a work day.
How often is the category calculated? IMatch keeps the result in memory once calculated, and only re-calculates when a) something in the database changes that affects the category and b) you access the category (e.g. display it somewhere).

That a category is slow may, or may not, interfere with your work in IMatch. If it's only calculated once, no problem. If you do frequent changes in IMatch and the category needs to be calculated over and over, it may slow down IMatch. In this case, may categories or collections are better to hold information about which file is a master or a copy. Masters are also in the Relations collections, and versions too...

[Carlo Didier]

For starters, you could combine the three RegExes into a single one:
@FileRegExp[_[vmk]]
and you don't need the @All at the beginning

[sinus]

Your explanations are very helpful, Mario, thank you!

And, Carlo, so your RegEx is much better than mine, I guess!
I will change it, thanks realy for your hint!

Like scripting, using Regular expressions are fascinating, but also quite difficult for me to understand!
Thanks! 

[sinus]

For starters, you could combine the three RegExes into a single one:
@FileRegExp[_[vmk]]
and you don't need the @All at the beginning

I tried it, but IMatch gives me an error, see the attachement.

[attachment deleted by admin]

[Carlo Didier]

You need quotes around it, like this:
"@FileRegExp[_[vmk]]"

[sinus]

You need quotes around it, like this:
"@FileRegExp[_[vmk]]"

Hi Carlo

Of course ... you are right! Thanks a lot - vielen Dank!
It works great.

[Mario]

With this change, your category formula should be nearly 3 times as fast. Great.

[sinus]

With this change, your category formula should be nearly 3 times as fast. Great.

I knew, that Carlo's proposal will be faster (and cleverer) than mine - but I did not expect that quick!
Great, thanks again, both of you! 

[Mario]

From 3 regular expressions per file down to one. I'd wish improving performance would always be that 'easy'