5 replies to this topic

[starstutter]

I feel sort of strange asking this, but no one's ever actually given me a straight answer (although admittedly I never asked a lot). My ongoing C++ project (for about a year now) is around 30,000 lines of code and it (re)compiles in about 22 seconds or so. Why is it that I constantly read about projects only about twice or 3 times as large taking several hours or days to compile? I know that visual studio and most other compilers only rebuild code in files that have been changed (unless there's dependancy or coupling). It just seems alien to me that a project would take that incredibly long to rebuild.

Now, on the very first time I open my project on another computer or I delete some build info files, it takes significantly longer (somewhere around 4 minutes or so). That makes sense to me, but still wouldn't explain day-long or overnight compile times unless the program was just an absolute monster. Would engine programmers just code for an entire day without recompiling? Possibly accumulating 20 or so game-breaking errors without being able to test? Any input on this please?

[Reedbeta]

Hmm, I don't know about hours or days to compile. Possibly those sources are counting art compilation as well as code? At my studio it takes maybe 10-15 minutes to build the engine from scratch, and much less if you only changed a couple files (although linking sometimes takes 2 minutes even if you only trivially changed a single file, for reasons that escape me). Also, it's worth noting that we have a distributed build system that uses a "compile farm" of 9 machines, and that gives a massive speedup. I've never tried building the engine completely locally but it might take quite awhile.

If you include compiling all the art assets from scratch, though, on Infamous it would take a couple hours to do. Most of the time was spent DXT-compressing textures and precomputing AO on all the models. At that point we didn't have any multithreaded build system for art, though. We've since added the ability to spawn as many build processes as your system has processors, which massively improves things. Also, we build art incrementally of course so full rebuilds of art are relatively rare.

[Wernaeh]

If you are on C++, a good way to unnecessarily increase build times is to have lots of complicated template instantiations - in particular on older compilers. For instance, you can get notoriously bad build times by using the boost library, and including large parts of boost into any of your source files.

Mix that with not-so-clever forwarding and including schemes (i.e. don't forward, but instead include everything). Then you quickly arrive at a point where a single header change requires large rebuilds, and each rebuilt source file requires all templates to be parsed and instantiated again.

So, in total, my current experience is that ugly build times mostly stem from a flawed file organization.

Cheers,
- Wernaeh

[SamuraiCrow]

My past experience with programming large programs was working on plugins for ParaView. ParaView took 2 hours to do a clean build on a PowerBook G4 not counting the time it took to generate the makefiles from CMake.

Later on in that job I was provided a Mac Pro with 2 quad-core Xeon processors at about 3 GHz each. Building on 7 of the 8 cores provided the quickest build time which was about 10 minutes. (Trying to build on all 8 cores at once caused the make utility to starve out and abort, not to mention that the operating system overhead of maintaining 7 compiler threads worth of disk caching was enough to keep the eighth core busy.)

I tried building using XCode a few times but it was actually somewhat slower since it maintained a constant 25 threads regardless of how many cores the system actually had installed (in this case, 8).

Among the causes of slowdown were some custom build scripts for the Mac version written in BASH which, of course, has no JIT compiler.

[JarkkoL]

I use pretty advanced template code in my engine project, which is around 100klines, and debug build takes around 1.5minutes to compile + link from scratch on my PC (Core i7) with VC2008. I believe my bottleneck is the HD and getting SSD would help, but with those compile times I haven't bothered since I'm not so crazy about reinstalling my system (: I haven't put much effort optimizing the build times anyway other trying not to #include useless stuff and rather forward declare. At a previous job I optimized full rebuild times of UE3 to be only ~3mins (~1 million lines) mostly by changing the project to use unity builds (helped a lot with link times too) and distributing the builds with IncrediBuild to 10 dual core PC's. Going beyond 10 PC's started to slow down due to the extra network traffic IIRC. So I would say that 20sec for such a small project is pretty ok.

[starstutter]

thanks for the replies guys, that does clear some of it up