VistaDB 3.4 performance tuning and optimization
3.4 Optimizations and Changes
I think many of you know we have been working under the hood lately for optimization. Way, way under the hood. Some of these changes are at the very lowest levels of the engine.
Strongly Typed
At the heart of VistaDB everything is a row. Database rows, clustered rows, index rows, etc. We have not completely changed out this system yet for the new strongly typed classes, but I would estimate around 80% of the heavy lifting work is done now.
See the first post about strongly typed performance gains for a background…
Some of the internal objects were holding Hashtables for quick lookup of information. This is great, until you realize the cost you are paying for conversion to type Object for all your items in the Hashtable. In one instance the Hashtable was replaced with a Dictionary<> generic and the class sped up by over 200%.
Numbers – prove it?
OK, this is a fairly unscientific method to test this release, but it is relevant to our product. I took the complete NUnit suite and ran it on my box with Build 57, and Build 58. I make each of them run 4 times and then I present the averages below. For reference this machine is running 64 Bit Windows Vista with 4GB of RAM on a Quad Core 2 Duo (Q6700). VistaDB does not take advantage of the 4 cores (yet), but I also have similar results on my Core Duo laptop and Athlon 3800 X2 at home.
Build 57 Complete Nunit
Total Time: 271.5 seconds
Peak Memory: 185 MB
Peak Memory: 185 MB
Build 58 Complete NUnit
Total Time: 142.4 seconds
Peak Memory: 132 MB
Peak Memory: 132 MB
Performance Gain = 52% faster
What about your app?
Well, I can’t promise that performance gain on your app. But the low level core of the engine is getting faster and faster.
Some of the big changes are NO more GC.Collect() statements. By eliminating the number of objects being created the GC is not having to work nearly as hard and we are not needing the GC.Collect() at the end of large operations.
Where do we go from here?
We still have a lot of room for low level engine performance improvement. Maybe not another 50%, but certainly a reduction in number of temp object allocations by 20%+. There are still some Object based lists in the code, and a few places where intelligent caching can seriously help performance.
We plan on a sprint / drift cycle for the 3.4 release. We are going to work on Optimizations, then pause and work on bug tickets, then go back to Optimizations. It should keep us fresh for the cycle and keep updates flowing.
Similar Posts
- Strongly typed performance gains
- The GC does not solve all memory leaks
- SQL Server 2008 (Katmai) Information
Comments
Jason Short on on 5.13.2008 at 1:27 AM
Here are the results of my home machine: Athlon X2 3800 with 2GB of RAM on 32 Bit Vista.
Build 57 Complete Nunit
Total Time: 353 seconds
Peak Memory: 179 MB
Build 58 Complete NUnit
Total Time: 192 seconds
Peak Memory: 131 MB
So it is about the same % increase.
OmariO on on 5.13.2008 at 4:00 AM
Do you use dictionaries where keys (or parts of keys) are System.Type?
OmariO on on 5.13.2008 at 4:39 AM
Very interesting information regarding "Inlining of value types" in CLR V3.5 SP1!! blogs.msdn.com/.../what-s-coming-i
Jason Short on on 5.13.2008 at 11:46 AM
Yes, many of the dictionaries are using system types. It would be interesting to build a 3.5 version from your post above it sounds like the 3.5 optimizations would help us as well.
Jason Short on on 5.13.2008 at 7:33 PM
Got an interesting note from a user that for his specific application he saw a 70% performance increase on normal program operation. That is good news. We are not hearing any problem reports yet on the new low level changes (phew).
OmariO on on 5.14.2008 at 11:26 AM
Then try to use RuntimeTypeHandle instead. It is runtime identity of .NET type, but weighs only an IntPtr and can be retrieved very quickly.
msdn.microsoft.com/.../system.type.get http://msdn.microsoft.com/en-us/library/system.type.gettypefromhandle.aspx msdn.microsoft.com/.../system.type.get http://blogs.msdn.com/vancem/archive/2006/10/01/779503.aspx
May be in future versions of CLR Type.GetTypeHandle will cost only a machine instruction.
ps. and sorry for my english :)
OmariO on on 5.14.2008 at 11:37 AM
There is a catch. RuntimeTypeHandle doesnt implement IEquatable<T> so each operation with key in Dictionary<RuntimeTypeHandle, V> or HashSet<RuntimeTypeHandle > boxes its value what lead to allocating 12 bytes (on X86). To avoid allocations you have to use with your dictionaries IEqualityComparer<RuntimeTypeHandle> implementation.