.NET Ramblings - Brian Noyes' Blog - Address perf WHERE IT IS NEEDED

Saturday, May 08, 2004

I sat in a talk recently on performance with .NET applications. The speaker was suggesting things that I here others say often but I strongly disagree with. Two specific issues were the recommendation to use ngen and to never include debug info in release builds to achieve the best performance. The conversation went something like this:

I pressed him on the ngen:

Me: I thought one of the reasons you shouldn’t ngen is because the JIT compiler can do optimizations that result in superior runtime performance over ngen?

Him: Well, yes that is true

Me: So if you really care about perf, you shouldn’t ngen?

Him: Yes, but if you ngen you dramatically improve the start up time for the app

Me For the first time the app is run only, right?

Him: Well, yeah….

Me: And, there are load time effects that ngen can have (rebasing) that can actually result in worse load time perf?

Him: Well, yeah…

Then the debug info in release mode:

Him: You should definitely avoid putting debug info in your release builds

Me: So what is the impact if you do?

Him: Well, a couple extra instructions per method call

Me: So do you think an extra couple instructions actually has an impact on the overall perf of most apps? Aren’t they usually IO bound or have other perf bottlenecks that really form the performance limit for most apps?

Him: Well, yeah…

Why don’t people get this? I'm not saying you shouldn't always consider perf or that you should do stupid things that will affect perf. But if you are doing things that affect the maintainability of your system, I would always favor doing the things that result in more maintainable systems. Then address perf where needed. If your performance is already adequate, then you don't have a problem. If it is not, it is probably due to one or a couple hot spots in your application that you can do some optimization on to get things back in check.

Ngen can either help or hurt perf, and can result in unpredictable results if anything in your system changes with respect to what version gets used. Me, I like deterministic systems. Debug info in your release builds can give you additional insight when errors happen in the production environment including line numbers in exceptions and additional ways to debug.

A lot of it has to do with good architecture and design too. Most often people apply performance tweaks liberally because they have no idea where the hot spots might be in their app or how to find them.

I just get frustrated when people latch onto a concept and apply it blindly without understanding when/where it is actually needed. Let's face it, if a couple extra instructions per method call is really driving the performance of your entire application, you either write some incredibly tight code, or your app is a simple little test app that you created to drive home your point that doesn't go out and make any DB queries, network calls, access the disk, or any of the other things that really drive the perf of real world apps.

.NET | Architecture

Saturday, May 08, 2004 2:32:20 PM (GMT Standard Time, UTC+00:00)

Comments [11] |

Saturday, May 08, 2004 5:05:27 PM (GMT Standard Time, UTC+00:00)

Agreed. When people design applications for performance there are layers to peel.

a) The most important item is architecture…hands down. If you don’t understand where the potential bottlenecks in the application are, how do you know where performance will matter most? Also, only sound software architecture combined with appropriate hardware selections and network architecture can support application scalability on grand scale.

b) Next, the basics...avoiding file IO or architecting for its optimization, applying concepts such as message queuing, loose coupling, appropriate multithreading usage where appropriate.

c) The nitty gritty details that can add up like string concatenation, debug settings, avoiding reflection, etc…to promote granular “optimize everywhere” best practices. These are the items that may be observed on a varied scale based on trade-offs between design, productivity, and maintainability. These are the items that usually don't make or break scalability if a) and b) are done correctly.

This is covered in the IDesign coding standard as well: http://www.idesign.net

Michele Leroux Bustamante

Thursday, May 13, 2004 2:54:27 PM (GMT Standard Time, UTC+00:00)

"Most often people apply performance tweaks liberally because they have no idea where the hot spots might be in their app or how to find them."

I couldn't agree more with this. I've seen people start going after complicated solutions to address performance issues and end up with hard to maintain solutions that do little (or do not) improve performance.

In my opinion, problem-solving skill is still one of the most important skills to look for when hiring developers.

Hector Correa

Friday, May 14, 2004 7:31:18 AM (GMT Standard Time, UTC+00:00)

With regard to leaving debug code in a release: the thing that makes sense to me about taking care with debug code is that you don't want to give out too much information. I find it useful to see what line or method threw the error, but I don't want others to get too much insight about the code from it. Users don't want to see a lot of technical stuff spewed at them and hackers might find the information far too useful.

buggy

Friday, May 14, 2004 11:32:25 AM (GMT Standard Time, UTC+00:00)

Excellent point buggy. The hard thing about performance, security, and maintainability is that they are all at odds with one another, pulling your design in three different directions for their own means. The trick is finding the right balance of those design aspects for your app, customer, and organization.

Brian

Friday, May 14, 2004 3:47:20 PM (GMT Standard Time, UTC+00:00)

Exactly, balance is the thing. I've predominently developed in-house applications and tools during my career, so security hasn't been a huge focus. Usability, performance and maintainability have been higher priorities. Now that I'm focusing more on website development, I'm educating myself on those security issues.

buggy

Wednesday, May 19, 2004 7:12:13 PM (GMT Standard Time, UTC+00:00)

Great advice. Bruce Eckel said it best, something like "the way most programmers optimize is to guess". Don't guess, profile. On top of that, platform-level profiling is incredibly difficult. If you're not very careful, you'll end up altering your environment between test runs and end up with worthless benchmarks. Doing that kind of profiling correctly is HARD.

Gordon Weakliem

Saturday, May 22, 2004 7:47:22 PM (GMT Standard Time, UTC+00:00)

One thing I discovered recently is compiling Release builds with debug information; its on the project's properties page. This gives you the benefits of a Release build with debugging information such as line numbers.

Monsur

Saturday, May 22, 2004 7:52:08 PM (GMT Standard Time, UTC+00:00)

Monsur,
That's exactly what I was referring to. I agree, nice, easy, worth the cost in the general sense.

Brian

Friday, May 28, 2004 5:48:37 PM (GMT Standard Time, UTC+00:00)

I TOTALLY AGREE WITH ALL THESE STATEMENTS!

I recently had a discussion with a well known author of Microsoft development books. He had talked about the performance difference between a For Loop vs. For Each. He demonstrated that the performance was much different between the two. After thinking about this for 0.0001 seconds, I thought to myself "Who cares?" If someone wants to use For Each vs. a For statement. Application performance bottlenecks are typically not the use of For vs. For Each, but what you do within that loop. That is just one example.

I see many developers every day and the number one thing I can say to them about performance and scalability is that it starts with and continues into a good testing process that includes some form of performance testing. The push-back that I often get is "I don't have time for performance testing." I have many comments on that statement. However, it doesn't need to be hard. Understand the major functions in your application and begin there.

Rich Crane

Wednesday, June 02, 2004 6:25:10 PM (GMT Standard Time, UTC+00:00)

An alternative to NGEN for getting high-performance startup is to integrate a load-on-demand system into your app. Only load what you need on startup, and then load the rest on demand, or during idle time. I haven't played with NGEN, but I have implemented load on demand and reduced the startup time of our app from 24 seconds to 4 seconds.

Mark Miller

Tuesday, August 30, 2005 12:02:05 PM (GMT Standard Time, UTC+00:00)

There are some disputable moments in your article. I do not absolutely
agree with the author.

There are some disputable moments in your article. I do not absolutely

Comments are closed.