20 replies to this topic

[gentleman]

Hi

I’m planning to use a component-based game object system. I’ve checkout some implementations (Yake, OGE) and threads. Everyone is so far also using a Message Queue System for events. I’m trying to figure out why. Why use a message queue instead of directly calling objects?

Cons with Message Queue:
• Slower than direct
• Must define message types and parameters
• More work/coding

Supposed Pros:
• Can spread work across frames by delaying event handling – however this can create interesting effects where objects lag behind. Besides queuing is slower.
• Can send a physics collision event to a non-physical object – however why even consider sending to an irrelevant object? The non-physical object shouldn’t even exist to the physics engine.

I’m looking for the advantages with a message queue. Any ideas?

[bladder]

well the goal is simply to do be able to know when something happens and deal with that situation appropriately without stalling the entire system. You can use a message queue (ala check message queue and if message found then deal with it, signal object, whatever), which is old and tried. or you can use a trigger system (ala attach handlers/functors to events and then call event->trigger() as appropriate), or if want to use something else you can do that as well. As long as the main goal is solved.

Your problems with message queues: Slower than direct? Sure, it adds another level of indirection. But, with the number of objects a tytpical system has and sitations and threads that it needs to deal with using a direct approach can become very messy, and probably even slower. Plus, I don't believe I've ever come across a situation where the speed of my message queue was significant in any way. Defining types, sure, but you're going to have to define something at the end of the day. More work/coding? not unless your other method is significantly less work/coding and provides the same amount of flexibility.

Quote

I’m looking for the advantages with a message queue. Any ideas?

Advantages over what else?

[Jare]

bladder said:

I don't believe I've ever come across a situation where the speed of my message queue was significant in any way.

Picture a couple thousand entities with 8 or 10 behaviours, each sending & receiving 4 messages per frame. Those numbers are not unheard of. Queued communications in that situation can create a lot of overhead in both memory and performance (including worse cache behaviour). For loose collaboration among entities it may work (basically events, but consider alternatives like blackboards), but for intra-entity functionality it easily becomes a nightmare. Frame-to-frame delays can create single-frame glitches and evil race conditions. Let's not even talk about debugging queued messages. Who sent it and why? Oops no callstack to give me a detailed context.

If your game is reasonably simple, then those problems may be small, but then why do you need queued messages for such a game?

[t0rakka]

Queue has a nice property that messages can be handled at appropriate spots in the client. If you are in middle of doing something, and you get event that changes the state of that something you are working with, you have to have a shadow state which is refreshed at some well controlled point.

You have to create complicated and error prone mechanisms to deal with all kinds of situations that arise from "interrupt scheme" of handling events.

If I am drawing image, the LAST thing I want is event_resize, and actually going and changing backbuffer dimensions WHILE in middle of rendering! That is one example of where you want the shadow state, which you then read when you BEGIN a frame and apply changes, THEN render the image.

Well, sir, message queue does just that; you can handle the pending messages before processing a new frame. That kind of arrangement is simple and just works, maybe that's why it is very popular. When and if you need something different, one can always go out and do it. You said yourself a lot of engines and crap use message queues, maybe that is because the need to do something else doesn't arise too often?

Just a thought.

[bladder]

Quote

For loose collaboration among entities it may work (basically events, but consider alternatives like blackboards),

I was under the impression that "events" is what's on the table right now. I wasn't suggesting every single object send a message to a message queue whenever it wants to do something, especially for "intra-object functionality."

[Wernaeh]

One large advantage I see in messages is the ability to give each a "net" relevancy, and then replicate messages over the networking system in a multiplayer, hard-synched game.

The advantage here is that you hardly need any net code if you properly set up your messages (i.e. also use "move" messages that an entity sends to itself and so on)

[J22]

In a game engine, message queue is particularly useful if you are developing a multi-threaded game engine. Different sub-systems can't communicate directly, because that would cause objects to be in an inconsistent state (e.g. think of physics system moving an objects in one thread while rendering system renders them in another thread). Instead, the physics system sends a message for the object to move the object at a later state.

Another advantage of a message queue is that you don't expect any specific interface from your objects. Let say that one day you want to add "set light brightness" message. If you would implement this with direct call, you would have to add virtual SetLightBrightness(float)=0; to your objects base class (or ensure the type and typecast to a sub-class with that interface). With message system you just add new message type called "SetLightBrightness" and process that message in the message handler of the object class (other types just ignore it). Of course it depends on the situation which solution is better, i.e. if you have some functionality that's required practically by all objects, then you should implement it as direct call.

[Wernaeh]

Quote

Another advantage of a message queue is that you don't expect any specific interface from your objects.

I'd say that generally is where the standard implementation would use multiple inheritance from interface classes, which also ensures that you don't pass messages to objects that do not support them.

Cheers,
- Wernaeh

[J22]

I wouldn't really go for multiple inheritance for message passing though. Querying and checking for the message handler interface is already pretty much as expensive as just sending the message to the object, unless if you think of caching the interface for further use, but that's not very practical and you could do similar caching without MI anyway. MI just tends to make things really messy.

[Reedbeta]

Multiple inheritance of *interfaces* isn't really problematic. What Wernaeh is proposing is much more like the Java interfaces, where a class extends only one superclass but can implement as many interfaces as you want.

[J22]

You have the same virtual inheritance issues with interface classes as well and C++ doesn't have concept of interface class nor can you count on empty-base class optimization as it's not guaranteed by the C++ standard. Anyway, MI doesn't really help here anyway, because all you got is base scene graph object class interface and you would have to query for the interface with custom/C++ RTTI.

[Reedbeta]

J22 said:

You have the same virtual inheritance issues with interface classes as well

No you don't. The virtual inheritance problem occurs when you create a diamond structure and the base class of the diamond has fields, which are then inherited twice unless you use virtual inheritance. But interfaces contain no fields, so the problem doesn't arise.

Quote

and C++ doesn't have concept of interface class

You can make pure abstract classes.

Quote

nor can you count on empty-base class optimization as it's not guaranteed by the C++ standard.

What are you referring to? I'm not familiar with the "empty base class optimization".

Quote

Anyway, MI doesn't really help here anyway, because all you got is base scene graph object class interface and you would have to query for the interface with custom/C++ RTTI.

Sure, you do have to use some kind of RTTI to make things work, but that certainly doesn't mean that interfaces aren't useful at all. It allows you to design so that all entities that support a particular functionality expose it through the same interface.

[Wernaeh]

Quote

Sure, you do have to use some kind of RTTI to make things work, but that certainly doesn't mean that interfaces aren't useful at all. It allows you to design so that all entities that support a particular functionality expose it through the same interface.

Actually, the point with interface classes is to avoid RTTI as much as possible.

Say you have an interface class CAiTarget for every object that can be used as a navigation target for AI calculations.

This class might look as this:

class CAiTarget

{

       virtual CVector GetNavigateToPosition() = 0;

       virtual AITargetType GetTargetType() = 0;

       /* ... */

};

Now, several other classes can derive from this interface, and provide appropriate methods - i.e. health pickups, camping positions, other AI controlled objects, each providing a different type, and target position.

However, the target type (i.e. semantics in AI calculations) is seperated from the actual target class. Moreover, if you then, inside your bots, only keep a pointer to CAiTarget, rather than some ominous entity base class, you make sure that you only have a valid AiTarget to begin with (... this gives you compile-time type security, rather than RTTI...)

Cheers,
- Wernaeh

[J22]

Okay, maybe I should reprhase myself as this thread seems to be badly off the track: MI is bad option for message handling because you have to first use RTTI to verify the interface. However, this prevents things like delegation of messages to e.g. composite objects (i.e. class A contains classes B and C as members, which also handle messages). MI also defines the supported messages in compile time, which makes it much less flexible system. And of course, using MI tends to lead to virtual inheritance, which makes things messy & slow. Better option is to send the message to the object (e.g. virtual void SendMessage(int id, void*)=0; ) and let the object process the message if it has handler for the message type.

[J22]

Okay Wernaeh, I think you talk about quite different thing than what message system is ment for. If you have a scene data structure where you have entity objects, and you shoot some object in the scene, you send message "Take Damage" to that object. What you are talking about is more of the component system, where specific component interface is exposed. Those systems are not mutually exclusive though.

[Wernaeh]

Yup they are not exclusive, but one may be implemented via the other and vice versa, each with different advantages.
However, because of type safety, I'd usually prefer the "exposed interface" over the "message sending" approach.

For example, sending a "Take Damage" message may be implemented by a "LivingObject" interface with a TakeDamage() call, and then, you wouldn't randomly send out this damage message to entities (in the hope that they might react to it), but be enforced to only call it on entities that support that interface.

Yet, the interface system is not easily run-time extensible, it is difficult (though not impossible) to add scripted additional interfaces to objects - though the same holds true for messages, just at a different place: If a message was not properly handled for at compile time within the message handler, you need some sort of scripted message handler, while if a interface didn't exist at compile time, you'd need a general scripted interface component that allows runtime binding of additional interfaces.

An advantage (maybe) of message systems is their relative simplicity: The sender needn't worry about the interface of the targeted object, just send on the message and hope that it does what it should.

Hope I didn't miss your point again ;)

Cheers,
- Wernaeh

[J22]

Wernaeh said:

For example, sending a "Take Damage" message may be implemented by a "LivingObject" interface with a TakeDamage() call, and then, you wouldn't randomly send out this damage message to entities (in the hope that they might react to it), but be enforced to only call it on entities that support that interface.

But with the exposed interface via MI, you would randomly query the interface from entities anyway so it's no different, right? For example if you shoot something, you pick the entity you shot, and would query if it's LivingObject via RTTI. But let's think for a second. What if this TakeDamage needs to be handled also by Breakable objects? Would you now inherit Breakable from LivingObject or would you implement Damageable interface and inherit Breakable and LivingObject interfaces from that?

Wernaeh said:

Yet, the interface system is not easily run-time extensible, it is difficult (though not impossible) to add scripted additional interfaces to objects - though the same holds true for messages, just at a different place

Infact, as I mentioned earlier about object composition, with message system you can easily add components to your entity object at run-time and have the messages delegated to them, which you can't do with interfaces. Or you would have to circumvent the interface with the generic script interface like you said, but that sounds awfully lot like implementing message passing.

[Wernaeh]

Quote

For example if you shoot something, you pick the entity you shot, and would query if it's LivingObject via RTTI.

No, I wouldn't. This is a workaround for the entire type safety you get with the interface system :) Basically, the point with interfaces is to see that you never ever need to downcast in the derivation tree. Don't keep a general list of "entities" unless you only are interested in things an entity class provides. Keep - per interface - a list (or more, you may encapsule this in a class) of objects implementing this interface. Obviously these lists then may properly be typed, and you don't need RTTI at all.

Quote

Infact, as I mentioned earlier about object composition, with message system you can easily add components to your entity object at run-time and have the messages delegated to them

Okay, adding entity attributes on a per entity basis (possibly at runtime) is a valid point, this is more difficult to do with interfaces. I think I finally got what you meant ;)

Don't get me wrong - I'm using a message system for certain components as well, such as for my gui system (though I didn't go for attributes there) :) My point rather was to detail what other alternatives are there.

Cheers,
- Wernaeh

[Jare]

Wernaeh said:

the point with interfaces is to see that you never ever need to downcast in the derivation tree. Don't keep a general list of "entities" unless you only are interested in things an entity class provides. Keep - per interface - a list (or more, you may encapsule this in a class) of objects implementing this interface. Obviously these lists then may properly be typed, and you don't need RTTI at all.

You can have dozens of different interfaces for groups of operations (health-related, collision-related, rendering-related, etc). It becomes impractical to maintain dozens of lists for a given operation. Or you may need to chain operations: find all objects that collide with this explosion, and then take a chunk of health from them, and then apply a physics impulse to send them flying... that kind of stuff. Some frequently-used lists like you describe can still be worth it for optimization, regardless if you use MI of interfaces or aggregation.

[Wernaeh]

Quote

Or you may need to chain operations

Yup, that becomes a problem with my approach.

Cheers,
- Wernaeh