106 replies to this topic

[xeonx]

epsilon said:

xeonx said:

What exactly are you doing that is 5x slower. Is it an entire rendering engine which has entire demos running a 1/5 the number of FPS, or is it just perf-analysis of a couple method calls. What C# rendering engine are you looking at that a speed hit is noticable (ie consistently under 30 FPS).

It is a tool to calculate some trajectories on a 3d triangle mesh.
It needs to store vertices and triangles on a octree to speed-up the intersection tests. That's why I said it's a kind of raytracer.

xeonx said:

A speed hit on one thing like ray tracing does not in any way make for a slow game, you will always find algorithems that are faster and slower depending on what you are comparing.

I don't find any algorithm better than I am using, if it is not the best algorithm, it is one of the best. My goal was not to try to improve the algorithm (already enough fast and good) but to compare two languages.

kind regards,
Epsilon

You wrote a ray-tracing alogrithem, by the very nature of it, it would be better suited to C++. But a 3D engine (and even more so with a game) is not entirely 3D ray tracing. This *is* one particular case. I am looking at 3D rendering engines as a whole and considering all of the benifits. There are cases (such as the DirectX demos) in which C# runs faster (though you can decide to go write your own GC...), but when comparing Ogre and Axiom, Axiom runs faster and provides many other perks becuase of its use of C# and the .NET framework. You will be hard pressed to find a managed language that runs faster then C or C++ in many algorithems, but when you consider the way in which all of the features are implemented and the systems work together - the big picture - it doesnt always match the speed of one particular type of algorithem. Its pretty impressive that you wrote your own ray tracing comparison though. Does it render anything or just make the calculations?

[epsilon]

xeonx said:

Its pretty impressive that you wrote your own ray tracing comparison though.  Does it render anything or just make the calculations?

It is a kind of renderer because I only calculate some trajectories on triangle mesh. It mades lots of intersections and find tangent point between plane/sphere, edge/sphere. So It doesn't render for the moment. I was looking for TAO to render the mesh and the results but I will stay in the current state because of the lack of performance (5.5 times slower, it is really too much). I don't exclude to reuse this C# code in case I (or others) would want to use it from .net plateform or using from scripting environnment.
I will continue now with C++ code and will use Opengl and Qt to make the UI part and render the results. Some years ago, I started to make my own renderer (z buffering) but I stopped the developpment and used Opengl instead.
I am searching the best way to store the mesh for making algorithms on it (fastest ray intersection, collision, face count reduction ...) and to have the fastest opengl rendering with vertx buffer for example. It will take some time I think :-)
I will develop on linux (gentoo) but Qt will allow to run this app on windows / mac / unix. Qt is the best window toolkit I ever seen.
I will think about to make this tool in dll and maybe in the futur make bindings for C# :-))))

xeonx said:

There are cases (such as the DirectX demos) in which C# runs faster (though you can decide to go write your own GC...), but when comparing Ogre and Axiom, Axiom runs faster and provides many other perks becuase of its use of C# and the .NET framework

Please could you explain to me the kind of benchmark you made to compare these 2 engines ? Is Axiom relied on Ogre ? does it use Ogre by native dll ? The 3d model is stored on dll side or completely in C# ? Have you some collision algorithms ? in which side dll or C# ?
I am very curious :-O

kind regards,
Epsilon

[xeonx]

epsilon said:

Please could you explain to me the kind of benchmark you made to compare these 2 engines ? Is Axiom relied on Ogre ? does it use Ogre by native dll ? The 3d model is stored on dll side or completely in C# ? Have you some collision algorithms ? in which side dll or C# ?
I am very curious :-O

Axiom is written entirely in C#, it is a full features rendering engine in which the same build can be thrown on any computer of any platform run just the same (no more if-defs and maintaining builds for different systems). The average FPS in the demos is faster then OGRE in some and slower then others (just like most comparisons) so it is not that it is consistently slower then a C++ engine.

It is full-featured, covering everything from materials (very high-level shader definitions) and the like. It is a C# port of OGRE so it does not use it. It nearly covers all of the OGRE functionality but with a much smaller code-base which has many new and great features thanks to the use of the .NET framework. The efficiency is progressing nicely and soon it will be much faster. The RealmForge GDK is a full features game engine, framework, and editor which uses Axiom as the rendering engine. It adds support for 3D Audio, scripting, data driven scene and world creation, many different types of entities and controllers, and provides a very high-level API for game development.

It actually provides an API which is independent of Axiom (so you could plugin in another renderer) and allows you to define every aspect of the game in the *in-game* editor. Much of these features are made possible becuase of the features of the .NET framework. C# really allows for many things to be done which are not possible in OGRE with C++ and with on average the same speed (and this will only get much better with .NET 2.0 and the planned optimizations). Both Axiom and RealmForge GDk are incredible products, especially when considering how "young" the projects are. I have seen far too many threads in which developers state that the .NET framework is to slow to be of any use in game development when this just isnt the case. When you get to the high-level framework which is required for the development of any game of significant complexity it gains back the efficiency that may have been lost with particular algorithems. The variance in FPS between the Axiom and OGRE demos just goes to show the efficiency doesnt depend on the language so much as the implementation, while the .NET framework provides many useful features and allows a very simple and quickly development implementation which will also allow more time for optimizations.

[Michael]

In which commercial games has Axiom been used?

[xeonx]

Michael said:

In which commercial games has Axiom been used?

Considering that it hasnt even made its alpha release yet, none. The RealmForge GDK which I am building on top of it does have several credible dev-teams waiting on it though becuase it provides the needed features that complete Axiom in terms on full-features game engine. You cant expect a comercial game after a couple months of development, especially since it requires at least a year or two for the game itself to be developed. But the level of progress after such a small time atests to it. Considering how it compares to other engines which were used for comercial games it is inevitable that it will be once completed.

[Michael]

xeonx said:

Michael said:

In which commercial games has Axiom been used?

Considering that it hasnt even made its alpha release yet, none. The RealmForge GDK which I am building on top of it does have several credible dev-teams waiting on it though becuase it provides the needed features that complete Axiom in terms on full-features game engine. You cant expect a comercial game after a couple months of development, especially since it requires at least a year or two for the game itself to be developed. But the level of progress after such a small time atests to it. Considering how it compares to other engines which were used for comercial games it is inevitable that it will be once completed.

I see, thanks.

[xeonx]

Actually, scatch that. A company based in Indiana just decided to use RealmForge to develop a comercial MMORPG and are providing hosting for the code-base and server-based demo games.

[Ravyne]

Firstly, I would like to point out that Microsoft does not control the future of .net any more than it does C++. While MS was involved in the development of the CLI/CLR and the sole developer(I believe) of C#, all of these technologies have been submitted to a standards body (I forget which.) This standards body is what will decide the future of .net (in the platform sense, not MS's tagging '.net' onto the end of all their products :rolleyes: )

Anyhow, I think C# is a viable platform for many types of games, certainly any non-realtime game can be done in c#, and I would say that any independant-quality real-time game (fps or what-have-you) can be done using c# as well. Someone setting out to be on the bleeding edge should stick with C/C++ but thats not most people, so I think one should ask themselves if C# is adequate for their goals, and if it provides other benefits (such as shortened development time) that another language would not.

I think that alot of independant developers need to realize that they aren't competing directly with the pros (not that they shouldn't push themselves.) Even if your engine has the technical abilities of Doom3, indies don't often have the art-staff to support it. Indies don't have to push the bleeding edge, so we are more free to take advantage of technologies which provide benefits outside of shear performance.

All that said, I like C++ and thats what I use, but I'm keeping an eye on .net.

[xeonx]

Very well put. Also if you check out the OGRE vs Axiom comparision thread at the Axiom forums, you will see that implementation matters more then the language selected as which engine outperforms the other depends on the specific demo and the results remain very similiar.

[Michael]

Ravyne said:

Firstly, I would like to point out that Microsoft does not control the future of .net any more than it does C++. While MS was involved in the development of the CLI/CLR and the sole developer(I believe) of C#, all of these technologies have been submitted to a standards body (I forget which.) This standards body is what will decide the future of .net (in the platform sense, not MS's tagging '.net' onto the end of all their products :rolleyes: )

I think this is a slightly idealistic view, the standards cover only C# and CLI (which only defines the core standard libaries).

Cheers,
Michael

[epsilon]

xeonx said:

Very well put.  Also if you check out the OGRE vs Axiom comparision thread at the Axiom forums, you will see that implementation matters more then the language selected as which engine outperforms the other depends on the specific demo and the results remain very similiar.

Have you some data structure algorithms in Axiom ?
I think it relies mostly on the graphic card. I checked again and again my C# code but I didn't make any fault. It means you will be speed equivalent as C++ if you relie on your video card, but I think you will lose a lot of performance when you will have to make some algorithm on your triangle mesh. I agree with you, C# as Java allows more flexibility over C++, but we should always take care about the performance. I am curious to see the first developpements using Axiom in C#.... very curious.
Please could you just explain what are exactly the functionalities C# allows to do and C++ cannot ?

kind regards,
Epsilon

[UnknownStranger]

epsilon said:

I agree with you, C# as Java allows more flexibility over C++, but we should always take care about the performance.

Sorry, you lost me there.

How do C# and Java "allow more flexibility" than C++?

[xeonx]

UnknownStranger said:

epsilon said:

I agree with you, C# as Java allows more flexibility over C++, but we should always take care about the performance.

Sorry, you lost me there.

How do C# and Java "allow more flexibility" than C++?

C# certainly doesnt impose any needless restrictions as one might think other then important one such as single-class inheritance, and declaring the use of pointers or unmanaged code. It does provide flexiblity in not just the excelent framework, but also in features of a manged language. Metaclass programming with attributes and reflection, dynamic serialization using this, dynamic inspection of types in assemblies for loading plugins, dynamic C# source file compilation using a class in the framework (not an external compiler) in a fraction of the second for an entire assembly. By far the best GUI framework, controls like the propertygrid which allow complex applications to remain very simple, .NET security which allows you to place restrictions on running code to accessing certian types of methods, assemblies, etc - even taking indirect access into consideration. An excelent RPC system (.NET remoting) and pretty much every important system like this all implemented in the core framework (eg. a whole slew of different highly-customizable encryption techniques), the design (like the ability to use a StreamReader to decompress or decrypt on the fly while reading the stream as needed), by far the best documentation I have ever seen (easily a bottle neck for many good frameworks and really helps for rapid development and troubleshooting). Also with C# nearly everything is very intuitive and simple and mature/tested or standardized. This is why C# (the same language as that of development) would even be the scripting language for the game. The ability to interface seemlessly with now hundreds of languages ranging from COBOL, to Python, to Perl, to VB, to Ada. The ability to depoly the same application or framework on any computer or OS without a single if-def statement and without maintianing a seperate build... The list really goes on. C# brings a lot to the table and these features would even be worhth a somewhat sizable speeed hit (considering that generally complex feautres do bring a speed hit to a game anways) but that is not even the case!!!

[xeonx]

Michael said:

Ravyne said:

Firstly, I would like to point out that Microsoft does not control the future of .net any more than it does C++. While MS was involved in the development of the CLI/CLR and the sole developer(I believe) of C#, all of these technologies have been submitted to a standards body (I forget which.) This standards body is what will decide the future of .net (in the platform sense, not MS's tagging '.net' onto the end of all their products :rolleyes: )

I think this is a slightly idealistic view, the standards cover only C# and CLI (which only defines the core standard libaries).

Cheers,
Michael

It doesnt matter what is registred as a standard, open-source projects like Mono and Portable.NET are implementing their own frameworks and adding significant functionality to them over the standard .NET framework. Dev-teams may eventually restrict themselves exclusivly to these runtimes due to this (though not until they play catch up with WinForms). My only gripe is that cross-platform GUI development is more difficult then cross-platform 3D graphics. Portable.net got the right idea to develop everything from the ground up entirely in C#, while Mono is using a huge 3rd party dependency and an awkward API for C#: GTK#. QT# may be something to look into though and I am working with wx.NET which is very good, but also not so mature (though the wxWidgets core is).

[Michael]

Quote

C# certainly doesnt ... The list really goes on. C# brings a lot to the table and these features would even be worhth a somewhat sizable speeed hit (considering that generally complex feautres do bring a speed hit to a game anways) but that is not even the case!!!

I think you and UnknownStranger have a different understanding of the term "flexible".

xeonx said:

Firstly, I would like to point out that Microsoft does not control the future of .net any more than it does C++. While MS was involved in the development of the CLI/CLR and the sole developer(I believe) of C#, all of these technologies have been submitted to a standards body (I forget which.)

xeonx said:

It doesnt matter what is registred as a standard

Hrm.

[epsilon]

I tried my application with Mono yesterday to see the difference of speed with microsoft.
So I summarize:
C++ : 8 seconds
C# Microsoft : 55 seconds
C# Mono : 160 seconds

Now definitively I am shocked about these results.

I think personally it may be good to spend a bit more time to manage pointer / new / delete and to have the speed gain than to have this difference. With C++ we will run at maximum speed and I think we are more compatible on all plateform. Qt will allow to manage more easily some kind of difference we could have.

Of course, I think this is easier to program with Java / C# because it is simpler to not care about the memory management and don't see heavy pointer * in the code and to have only one file for a class and to have a faster compilation and ... I can find many many arguments but the fact is C++ is a lot faster. All depends what we want to do with.

I agree with Stroustrup: I don't see why we would need a proprietary language. I fear that in the futur, industry will not want to heard something else that C# .NET. Also I fear that these tools will not be free in the futur. For the moment it is free and .NET is trying to convert the C++ community by bringing lots of functionnalities : who wants to be alone with pointers ???
For how long it will stay free ?

I am thinking that when C++ will be dead, Linux / Unix will have no chance to compete with Windows.
I think it will be soon impossible to run a native application on windows. Why ?
because interaction with windows should call a .net function which is managed and so your application will be managed, and a managed C++ application lose all of its power. We will gain no speed improvement and we will be annoying with the language difficulties. What will be the reason to use C++ on windows now ??? and now you can eliminate every potential application on Linux / unix because Windows means 90% of the market.

I am sure you will say : we have mono and portable.net and if they don't success other people will bring a solution. But are you sure about it ? I am more and more less confident with these tools. Never they will really compete with Microsoft of course. Never they will have the last functionalities you will be impatient to have ... Never an industry will want to have a .NET diiferent of Microsoft, and even the service provider will not certificate their program with other tools than microsoft.

For the games, using C++ will give you the possibility to develop for PC, linux, all gaming consoles
included Playstation, gamecube at full speed. And C# ?

I am sure someone will find good argument again and again but for me there is no way.
Now I am not confident about .NET, for speed and for the futur other than windows.
I am asking simply why people don't move and didn't move on Java ???
I think if someone is searching for a simpler language and very portable, why waiting for .NET ????
Java is even faster than .NET !!!
Also I am just wondering why Axiom is not made in Java ?

Just for information, each time I was thinking to make a program in .NET at work, each time I though about unix and each time my program had to run on a UNIX (sun / IBM) computer.
You will ask me why I didn't use C++ ? because customer don't like to heard C++ because it means too expensive to maintain, too difficult to find programmers .... but they don't want to use Java on desktop, only with J2EE / jsp. For desktop they are using VB. And I am sure they will want soon to use only .NET.... or maybe Java never know if they will think about portability.

For all of these reasons, I have (re)installed a linux partition with gentoo, and I will developp with KDevelopp and Qt. Of course I would prefer visual studio but I am flexible I will make effort use a "simpler" IDEA. Only important is the result.

kind regards,
Epsilon

[xeonx]

You test is still very limited in scope, the speed of one particular algorithem has nothing to do with the overall performance of a game engine, let alone the game running on it. It has everything to do with implementation though. Also when considering how advanced Axiom is and how it is even faster then OGRE – the C++ of which it is a port – in some cases, attests to this. These speed of raw algorithms like ray tracing is far less critical then systems like AI, scripting, and scene management (which are all implementation-centric).

Below is a list of the results from comparing Axiom in C# to OGRE and C++. And in the places where there is a difference, it is just a couple percent less or even greater (not a factor of 5). This trial is far more relevant considering that it is comparing what you will actually see in FPS in a finished product, not a particular task found in the engine like ray tracing. And this is even further skewed in how OGRE has put time into optimizations while Axiom really hasn’t yet. Once Axiom does and the .NET framework 2.0 comes out, it would not be unlikely to see Axiom beating OGRE in more of the trials.

My Computer

• Direct 9.0c October
  
• .Net Framework: 1.1.4322.573
  
• 1600MGz Pentium M w/ 512 RAM
  
• ATI Mobility Radion 9000 32MB
  
• 1400x1050 @ 32bpp Fullscreen
  
• Operating System: Microsoft Windows NT 5.1.2600.0

Axiom Results (Release Build from CVS 10-16-04) vs Ogre Results (0.14.0 Prebuilt Release)

• CameraTrack
  Average: 99.93502, Best: 110.5578, Worst: 70.78764, Frames: 95, Time(ms): 10008
  vs
  Average: 89.547371 Best: 101.391647 Worst: 72.564606
  
  
• CelShading
  Average: 172.4322, Best: 172.4825, Worst: 172.1393, Frames: 167, Time(ms): 10001
  vs
  Average: 172.977478 Best: 173.306778 Worst: 172.310760
  
  
• CubeMapping
  Average: 130.0475, Best: 130.3483, Worst: 126.3682, Frames: 123, Time(ms): 10001
  vs
  Average: 124.790047 Best: 125.748497 Worst: 119.641075
  
  
• Dot3Bump (NOTE: Uses a different model)
  Average: 81.20572, Best: 81.83633, Worst: 80.11869, Frames: 78, Time(ms): 10011
  vs
  Average: 118.834244 Best: 121.272362 Worst: 114.087296
  
  
• Fresnel
  Average: 28.34465, Best: 28.94212, Worst: 27.31707, Frames: 23, Time(ms): 10015
  vs
  Average: 24.264023 Best: 26.239067 Worst: 23.692005
  
  
• ParticleFX
  Average: 33.28254, Best: 67.6617, Worst: 32.00776, Frames: 34, Time(ms): 10026
  vs
  Average: 36.369019 Best: 93.688362 Worst: 33.300686
  
  
• SkeletalAnimation
  Average: 135.8369, Best: 136.0477, Worst: 135.0546, Frames: 132, Time(ms): 10003
  vs
  Average: 153.118439 Best: 154.228851 Worst: 152.390442
  
  
• Smoke
  Average: 57.66978, Best: 101.8793, Worst: 57.08661, Frames: 53, Time(ms): 10006
  vs
  Average: 82.717522 Best: 121.878128 Worst: 78.063240
  
  
• Water
  Average: 101.3532, Best: 116.0714, Worst: 98.21429, Frames: 95, Time(ms): 10002
  vs
  Average: 108.639755 Best: 112.983154 Worst: 89.463219

As you can see Axiom is often very close in FPS, and in the more complicated demos like Fresnel actually beats the OGRE FPS. The large variance in performances just goes to show that implementation matters more then the language which you choose to develop in, so why not choose the best tool for the job: one which will produce a robust and dynamic product in minimal time.

By the way, Java is hardly faster then C#. Especially when considering like versions or that Java does not JIT by default. What test results have you seen for this?

[xeonx]

I want to know where you are getting the idea that Java runs faster then C#. Check out these reports, the code is available for download, the results are detailed, and the soure credible. There are a series of tests covering different aspects and C# performs consistently better (about 20-30% so). I have worked with Java developers and they often cede on the fact, so why would you argue it?

http://www.cs.cornel...2002/11/25.html
http://www.cs.cornel...2002/11/24.html

[Michael]

So how exactly are OGRE and Axiom related? I supposed they use the same algorithms, but then you mentioned that OGRE was optimized but Axiom wasn't optimized yet, so it seems they aren't. But on which base are you doing your benchmarks, then?

(Sorry if I missed something essential from your post, I don't have time to go through it completely atm)

- Michael

[epsilon]

I will port my application in Java to make a benchmark.
I use Java at work and I know very well the speed of Java.
Also I have already done a renderer in Java : z-buffering, triangle filling
I think the language itself is faster than .NET
but all graphical things or some specifical areas seems to be a bit faster in C#
but you must know that all the Java libraries are written in Java, instead of C# which relies a lot on native library.
We will see in few weeks with the benchmark of my application.