39 replies to this topic

[starstutter]

So, you probably know there's another thread currently going on and got derailed (kind of) to a similar topic. I was kind of wondering how to clearly distinguish between good design and planning for performance, and pre-mature optimization which many programmers suffer from (and that totally doesn't sound like an innuendo).

So to throw out an example here, I designed a deferred lighting system in which I got the idea to use shadows to my advantage speed wise. By using the stencil buffer when rendering shadows (shadow maps that is) and culling black pixels, I was able to cut out a large chunk light processing by causing the stencil test to fail where shadows were being projected. Now, I knew that I had to do this optimization while I was writing the lighting system itself, because implementing later would be an

malignant said:

actively reproducing female k9...

So, is there some kind of guideline to follow? How do you know what you should implement immediatley and what's just a waste of time? Is there some tricky experience behind it, or is it just more common sense (probably a bit of both)?

[alphadog]

I'm afraid the only guides are experience and insight.

And, would your example really be hard to revisit, assuming you had good code that showed great fundamentals like low coupling, high cohesion, etc?

[Sol_HSA]

There's one optimization that is never premature: Optimizing for readability.

If you're charting new ground, it's usually best to play it safe. First make it work, then make it fast. If two routes are more or less equally safe (i.e. should I use gl extension A or gl extension B to do the same thing?) then picking the faster is probably a good decision, and not 'premature optimization'. (Heck, in the case of FBOs compared to older techniques, it's probably safer AND faster).

So, my guidelines would be:
1. Optimize for readability. Even if it seems to make the code slower. It probably doesn't. Another hint: comments never made anyone's code slower.
2. First make it work, then make it fast.

[starstutter]

alphadog said:

And, would your example really be hard to revisit, assuming you had good code that showed great fundamentals like low coupling, high cohesion, etc?

Well I will say that this example was from a while ago when I was more inexperienced, and unfortunatley yes, there was a fairly high level of coupling that I now know makes things a nightmare. Now my design philopsophy goes something like:
Everything I make except for the highest level (game specific) functions should be able to be copy and pasted into another program and still work.

Seems to be working for me too. The only thing all the "tools" (as I'm calling them) link to is a simple debugging tool that writes to a file when an error occurs (either from directX or my engine). I re-coded my old engine in this new style and so far it retains the same capabilities with literally 1/20th of the code and I have spent less than 1.5 hours cumulative debugging the thing, as compared to many days with the old style :) . Yes I sound like a total noob right now but I'm happy, shut up! :D

Finally nice to hear that from another coder though, so thanks. But btw, to answer your question, it would have been pretty difficult considering how badly things were set up, I certainly do know much better now.

And thanks Sol_HSA for the advice as well.

[Goz]

Deliberate pessimisation is just as evil, if not more so, as premature optimisation ...

[alphadog]

Sol_HSA said:

There's one optimization that is never premature: Optimizing for readability.

Interesting way to see it. Personally, I wouldn't call "working towards readability" an optimization technique, but I totally agree with the spirit of the suggestion. The reason I wouldn't call it optimization is because I consider it a "baseline" aspect of good coding, and optimization is something that is highly recommended, but can be deferred or not done at all if you still fit within requirements.

Goz said:

Deliberate pessimisation is just as evil, if not more so, as premature optimisation ...

Just to make sure we're on the same page, what do you mean by "pessimisation"? I take it to mean putting off optimization/refactoring until the very end.

It's interesting to note that premature optimization syndrome (or POS, in order to optimize further typing :dry: ) will often, but not always, lead to less maintainable/readable code.

The only evil is the over-generalizations being bandied about!

A balanced view is that if there is a need in the reqs, and there always is with games, you must design for performance amongst other factors. That is basically pre-optimization. However, optimization cannot occur in some cases without actual code to measure; the whole is not always the direct sum of its parts. Sub-optimization can trap you into a slower solution.

The problem with both generalizations is that they attempt to position all of the design/optimization work at one end, either the start (POS) or the end (DB).

That's because most coders like to spit out code and never revisit it. The real way to operate is in cycles. TDD helps here...

[Goz]

Pessimisation is deliberately writing un-optimised code.

As an example: Deliberate pessimisation would be saying i need a sort and implementing a bubble sort. It doesn't really change the readability of your code to use a radix, for example, but it makes your code far slower pointlessly.

[alphadog]

Will the bubble sort's slower performance actually impact the overall hypothetical application? Can you determine that when you are coding that particular block?

What will the impact of using a more complex sorting technique have on future maintainability versus the gain in performance?

Can the bubble sort be refactored if it is found to affect performance?

Do you want to stop your deliverable at every block of code to think this through?

[Goz]

alphadog said:

Will the bubble sort's slower performance actually impact the overall hypothetical application? Can you determine that when you are coding that particular block?

What will the impact of using a more complex sorting technique have on future maintainability versus the gain in performance?

Can the bubble sort be refactored if it is found to affect performance?

Do you want to stop your deliverable at every block of code to think this through?

Does algorithmic optimisation cause your code to become any less readable and hard to maintain?

Because thats why pre-mature optimisation is so "evil". Algorithmic optimisation appears to be sensible to me. Use the best tool for the problem.

[alphadog]

Goz said:

Does algorithmic optimisation cause your code to become any less readable and hard to maintain?

Answering a question with a question, are you?

My answer to yours: often yes. More performant algorithms are often more complex and less common than less performant ones. Thus, less readable and harder to maintain, respectively.

PS: Of course, this is a matter of degree. On a case-by-case basis, it is usually not order of magnitude more complex and less readable, but it is. And, when you aggregate that over an entire codebase, then it has a non-negligible cost wrt to those two factors.

[JarkkoL]

starstutter said:

I was kind of wondering how to clearly distinguish between good design and planning for performance, and pre-mature optimization

If you can delay optimization to a later stage, then doing it earlier is premature. However, there is some disagreements with people what kind of optimization you can delay and how far.

Personally I think it's important have good idea about performance characteristics early in a project for many reasons. One of the major reasons is that in game project there are artists and designers creating content for the project and if you drastically change performance characteristics over the project, you waste a lot of artist/designer resources. And in an average game dev team there are ALOT more artists/designers than there are programmers.

Another purely programmer reason is that if you implement features without having good idea about the performance, you potentially implement features that you don't have cpu/gpu resources for, so you end up cutting them from the final product in order to hit your performance target.

It's also important to learn to write efficient code straight off the bat and know what kind of choices have major impact to the code performance on your target platform(s). If you don't follow good practices while writing code it's very difficult and time consuming to fix later (it's the "goo" I refer in the another thread).

So, any optimizations that would get me to the range.. let say ~30% mark from the final performance early in the project I wouldn't consider premature. That last 30% you can then squeeze from code + assets when approaching the end of the project.

[Grumpy]

I have to agree with Sol HSA about first getting it running then optimizing. Pre- planning is probably %75 of the work, knocking out re-writing code multiple times and removing redundant code (and yes, changing that "pre-mature optimized code").
For example, Malignat had defined Pi; that would most likly be used in his engine... He also had to other floats that are 2Pi and HalfPi (but not by those names) but would he actually use those floats in his proggy ??
When going through the pre-planing phase, you would know (hopefully) wich library funtions you will be using and subsequently have an idea what arguments you need for those functions... Do you still need 2Pi or HalfPi now?? Do you need both If_Odd and If_Even when you could get by with 1 of them and use an else statement cuzz it returns a bool. If not then dont include those definition, saving a few bites and micro seconds in compiler time.

JMHO...

Grumpy

[Nick]

JarkkoL said:

If you can delay optimization to a later stage, then doing it earlier is premature. However, there is some disagreements with people what kind of optimization you can delay and how far.

Personally I think it's important have good idea about performance characteristics early in a project for many reasons. One of the major reasons is that in game project there are artists and designers creating content for the project and if you drastically change performance characteristics over the project, you waste a lot of artist/designer resources. And in an average game dev team there are ALOT more artists/designers than there are programmers.

Another purely programmer reason is that if you implement features without having good idea about the performance, you potentially implement features that you don't have cpu/gpu resources for, so you end up cutting them from the final product in order to hit your performance target.

Which is why I referred to prototyping in the other thread. But note that if you or anyone else on the team already has the prior experience to know for a fact that certain things will need certain optimizations to meet the goals, then clearly there is no reason to delay them and could even be included in the design phase. In all other situations, there is an extremely high likelihood that optimizing something in advance will turn out to be a waste of time, bloat the code, and attract all the other evils of premature optimization. I also believe that much too often people assume they have the experience to know what will need optimizing in advance. After 10 years of 3D graphics programming I still catch myself doing that from time to time (though I also believe that some of it is unavoidable due to design goal changes and uncertainties)...

Quote

It's also important to learn to write efficient code straight off the bat and know what kind of choices have major impact to the code performance on your target platform(s). If you don't follow good practices while writing code it's very difficult and time consuming to fix later (it's the "goo" I refer in the another thread).

I agree that ideally your team members should have all the relevant prior experience possible. Unfortunately there is no such thing as "learning to write efficient code straight off the bat" for the less experienced. And that's when they should be most aware of premature optimization. Lets take for example the popular topic of what programming language someone should use for a first project. All the pros would tell them they used C++ on their last AAA title, but likely advise a simpler language. If he doesn't take the advise, there is no doubt the project will take longer (if ever completed) and the performance goals have been overshot. Of course an important flip side is that he'll have more experience for the next project...

Quote

So, any optimizations that would get me to the range.. let say ~30% mark from the final performance early in the project I wouldn't consider premature. That last 30% you can then squeeze from code + assets when approaching the end of the project.

Indeed, with experience from alike projects your optimization choices are much more likely not to be premature. But the point is you can't generalize that as an advise to everyone. Also, what was a good optimization in a previous project might bite you in the ass in the next (e.g. the compiler beats your bit twiddling hack).

Like I said in the other thread: When in doubt, it's premature.

[JarkkoL]

Nick said:

Which is why I referred to prototyping in the other thread.

Well, it's not the prototyping how you determine the performance in game projects, but by early optimization of the final implementation. If you were doing more research type of work, then you would do prototyping, but that's not what you do in game projects in general.

Nick said:

Unfortunately there is no such thing as "learning to write efficient code straight off the bat" for the less experienced

I have a bit more faith on those less experienced programmers, but I wasn't talking only about them. Even if you are experienced programmer you have to learn what are the performance characteristics of your target platform to be able to write efficient code straight.

Nick said:

But the point is you can't generalize that as an advise to everyone...

I'm not writing a newbie programmer tutorial here but just sharing my experiences I have had regarding the topic hoping to spawn some fruitful conversations.

[starstutter]

You know I'd just kind of like to say that game programming is kind of a depressing field... I've been programming for 6 years, written two 2D engines and two 3D rendering engines and I still qualify as a newbie =/
It's a looooooooooong road...

[Reedbeta]

starstutter said:

I've been programming for 6 years, ... and I still qualify as a newbie =/

You think other fields are any different?

[Sol_HSA]

Reedbeta said:

You think other fields are any different? ;)

I've been given the impression that six years in bicycle repairs just might make you an expert in that field.

[alphadog]

OMGWTFBBQ! :ohmy: woot biking R0XX0RZ! :yes: who can say he dusnt, is a retarded hat4r n00bsauce, ill :ninja: those fuggahs wit my +4 spork of pwnage! lollololololo. i ried alot. i have ideas 4 better bike + mad screwdriver skillz. lol... hahahaha... i said screw. i liek 2 b design teh trek madone but better!!111!!!!!!!1~~1!!

give me bluprint n recipe 4 carbun fiberz n wats gears? how do brakez wurk?

plz send me info
thx luv u guys u all rok! :worthy: :surrender :worthy: :yes:

[Nick]

JarkkoL said:

Well, it's not the prototyping how you determine the performance in game projects, but by early optimization of the final implementation. If you were doing more research type of work, then you would do prototyping, but that's not what you do in game projects in general.

A prototype can also be made using engine code that is still unfinished, but then it's done in a different branch of source control. If it works out as expected (i.e. meeting design goals) then it's further developed into production quality code and then merged back into the trunk, else discarded. What you describe sounds like doing optimizations straight in the trunk, which is a very bad idea not unlike premature optimization.

Quote

Even if you are experienced programmer you have to learn what are the performance characteristics of your target platform to be able to write efficient code straight.

By prototyping. ;)

Quote

I have a bit more faith on those less experienced programmers, but I wasn't talking only about them. [...] I'm not writing a newbie programmer tutorial here but just sharing my experiences I have had regarding the topic...

The problem I have is that your experiences could put both newcomers and experienced programmers on the wrong track (but most likely the former). The way you describe it, the advice about premature optimization is incorrect for certain aspects of game development, advocating to optimize things without prior information about whether it will be gainful or not, right?

Now consider the possibility that you've been fairly lucky with your assumptions and it never lead to serious project delays. Or take it as a complement; intelligent people are more likely to make better decisions for a complex issue having only little prior info. But with all due respect that doesn't prove that your advise is any good.

In my experience development is faster to some degree if you stick to Knuth's advice. My number one experience is being the lead developer of SwiftShader, a real-time software renderer. And I believe that's relevant because performance is never high enough and there are always more features to implement. So I can't afford to waste any time optimizing something I'm not sure will give me any significant gain. In almost a decade the only recipe that has proven to work is: profile, prototype, optimize.

You could argue that I've proven nothing either but then I'd have to ask you to find a faster software renderer with the same feature set that has been written by a small team. I don't want to make this any more personal than necessary though. But keep in mind that Knuth has also been awarded the Tokio Prize for lifetime achievement so you still have a lot of proving and disproving to do.

Quote

...hoping to spawn some fruitful conversations.

I think you definitely achieved that already. :yes: Also please note that even though I don't fully agree with you I certainly appreciate that you share your experience and defend it. :happy: Maybe in the end we all learn something. It already made me have a closer look at the exact definition of premature optimization, only to discover that it applies exactly as much to game development though...

Thanks.

[JarkkoL]

If you had any experience in game development, I might put some value to your comments, but you simply got no clue what works and doesn't work in game projects. You need to understand that different type of projects have different constraints and you have to have different strategy also regarding optimization. It's simply ridiculous and arrogant for you tell me what kind of strategy should be taken in game development without any experience in game development or understanding of game project constraints, don't you think?

Btw, what you try to achieve with provocative comments like "only to discover that it applies exactly as much to game development though"? How did you discover that? Do you imply that this conversation somehow proved your point or could it rather be that I managed to piss you off in previous discussion and now you try to return the favor?