19 replies to this topic

[.oisyn]

Tr8 currently takes about an hour for the Xbox360 build on my (ok, pretty old) P4 3.2 GHz (with hyperthreading, yay! :dry:). The other platforms are about 25% faster, though.

[Reedbeta]

I worked on "Infamous" (from Sucker Punch Productions) over the summer, and IIRC it would take about 20-30 minutes to compile from scratch, that was using a compile farm of 4 machines though (each one probably a recent dual-core P4 or Athlon - not really sure.)

[Mattias Gustavsson]

My (small) game I'm doing on my own takes 32 seconds to do a full build. I'm doing some contract work on a big AAA title, and it takes just over 5 minutes for a full rebuild. Both on a P4 3 Ghz.

I've worked on lots of projects with unnecessary long compile times. It's actually quite easy to reduce compile times; just avoiding including header files from header files makes a massive difference.

[Dia]

The project I'm working on takes around 10 min to compile using the VC++ compiler, but takes around 3 min to compile when utilizing Incredibuild...it's one of those tools that once you use, you can't live without

[Nick]

I reduced my build time by 30% with a Western Digital 'Raptor' hard disk. Now I'm waiting for SSD's to become affordable. :whistle:

Mattias Gustavsson said:

It's actually quite easy to reduce compile times; just avoiding including header files from header files makes a massive difference.

Very true. Every year or so I go through all my source files and I systematically remove all unnecessary header files. I takes half a day but it pays off for the rest of the year.

I wonder whether there's any tool that can do this automatically... I've also been looking for something that continuously compiles in the background so when I hit F5 (Visual C++) it only has to to compile the most recently changed files. :huh:

[.oisyn]

Mattias Gustavsson said:

I've worked on lots of projects with unnecessary long compile times. It's actually quite easy to reduce compile times; just avoiding including header files from header files makes a massive difference.

I know. I've already reduced the dependencies on the subsystems I work on throughout the game - it's a pain to see a quarter of the game to rebuild when you change a single header file. Then again, there's a limit as to how far you can go. If your project inherently needs all those dependencies you're simply stuck . Nested classes for example are typical dependency whores as you can't forward declare them

Dia said:

The project I'm working on takes around 10 min to compile using the VC++ compiler, but takes around 3 min to compile when utilizing Incredibuild...it's one of those tools that once you use, you can't live without

Yeah I think that is being investigated at the moment. As for the content builds, we are currently in the process of making our build system distributed

[monjardin]

.oisyn said:

Nested classes for example are typical dependency whores as you can't forward declare them

I'd say you could place them in a namespace to keep them out of global scope, but I imagine you're doing some template trickery that requires them to be nested...

[.oisyn]

monjardin said:

I'd say you could place them in a namespace to keep them out of global scope, but I imagine you're doing some template trickery that requires them to be nested...

I was talking in general, not specifically to our project. Of course you could make the classes not nested, but if it's a good choice from a design perspective, then why not make use of it?

Now, a real solution would be C++ modules

[J22]

Our game takes around 4mins with IncrediBuild for Xbox360, including linking (around 100GHz farm speed). Haven't tried without IncrediBuild for long time.

[Reedbeta]

100GHz? That's a lot of farm!

[J22]

Not that much actually. Just around 20 IncrediBuild agents running on 3GHz dual core machines.

[gardon]

Visual studio has a method to only re-compile those files that have changed.

[.oisyn]

No shit . I think every build system out there has that option.

[Nick]

gardon said:

Visual studio has a method to only re-compile those files that have changed.

As do most compilers. And it has several other tricks to limit compilation time.

But unfortunately it's not perfect. If you externally change project related files you might want to do a full rebuild. Also, when you're doing regression tracking you have to build multiple older versions. If the full compilation time is only a couple minutes you can 'manually' search for the change that caused the regression. But if it takes longer you either need an automated method for regression tracking, or find ways to shorten your compilation time...

So no matter how clever the compiler is, it never hurts to keep your compilation times limited by checking header dependencies and maximizing your system's efficiency. If you know of any compiler tricks that are less known we'd love to hear about them though.

Some other advise I can give to keep compilation times low is to code only what you need, deprecate early, and K.I.S.S. It's very tempting to 'think ahead' and add features that you might or might not need later. It's not only a waste of time if it turns out you don't need it later (9 out of 10), it also makes compilation time longer than necessary. Don't be afraid to deprecate code and remove it entirely. It's distracting and again eats away compilation time. Rely on your versioning, backup and/or archive systems to store the old code. And finally keep your designs simple. If you can do something adequately with a global table of function pointers, don't use a hierarchy of abstract classes with a singleton facade. Always refactor to simpler constructs if they are more suited for the situation.

[gardon]

.oisyn said:

No shit :p. I think every build system out there has that option.

No shit, no shit (lol). Nick said he wanted to "compile the most recently changed files". Just making sure he knew :)

[m4x0r]

Compile times have never been that big of an issue for me since you rarely need to do a full build (and you can keep the dependencies low if you're careful). Linking times on the other hand have been a big problem. On the last big project I worked on, a link could take up to 10 minutes. Does anyone have advice on how to reduce those?

[TheNut]

It usually takes 5-10 seconds with my own projects. My engine takes about 20 seconds to compile on either Winjoes or Leenux. It's whistle clean goodness ;)

The stuff I do at work takes anywhere from 10-20 minutes. I have a script that I execute on the build machine that automagically builds everything. From downloading the latest sources, compiling, organizing, configuring, and installing. Of course I don't let the boss know cauz then he'd be like "WTF?! am I paying you for instead of the machine" :lol:

Now if only I can invent AI that makes games. I could sleep-in every morning :lol:

[.oisyn]

m4x0r said:

Compile times have never been that big of an issue for me since you rarely need to do a full build (and you can keep the dependencies low if you're careful). Linking times on the other hand have been a big problem. On the last big project I worked on, a link could take up to 10 minutes. Does anyone have advice on how to reduce those?

The visual studio compilers support incremental linking. I don't know whether gcc does, however.

[J22]

m4x0r said:

Compile times have never been that big of an issue for me since you rarely need to do a full build (and you can keep the dependencies low if you're careful). Linking times on the other hand have been a big problem. On the last big project I worked on, a link could take up to 10 minutes. Does anyone have advice on how to reduce those?

An ugly way to reduce link times is to create few cpp files that #include many cpp files and compile those cpp files instead. Of course if you edit a cpp file that's included in a big one, it causes the big cpp file to recompile, but you might want to try it out anyway if it still compensates the long link times.

[Reedbeta]

A related, but perhaps not quite so hackish solution is to split a large program into a collection of DLLs. You'll only have to relink the libraries that were modified at each step. This might work for static libraries too, but I'm not sure. Of course, DLLs can introduce a whole host of other issues into your code, like having different memory allocators for each one.