Cockpit display of drone camera

88dc730f0f71e55be39de0ad103bd9ff
0
Alienizer 109 Dec 12, 2013 at 20:09

Here is what I’m trying to do in my airplane game…

I have some drones (really only cameras for now) floating in mid air. I want one of the drone to lock on a specific airplane target and display that target on my cockpit display as the drone sees it, but zoomed to fit the display (not the screen).

Any suggestions as to how I should go about displaying an object from an arbitrary camera view and zoom in/out to fit the cockpit display not the screen?

I would assume I need to project the object onto a plane (which is the drone camera’s normal?) and then perform the zoom to fit thing? then copy the result on my cockpit display (re-scaled to fit)?

18 Replies

Please log in or register to post a reply.

A8433b04cb41dd57113740b779f61acb
0
Reedbeta 167 Dec 12, 2013 at 20:25

It sounds like maybe you want to render the drone’s camera to an offscreen buffer and then map it as a texture when rendering the cockpit display? That would probably give you the most flexibility.

88dc730f0f71e55be39de0ad103bd9ff
0
Alienizer 109 Dec 12, 2013 at 20:57

Yes, exactly, but I still have the problem to “zoom extents” so that the object fit’s the entire cockpit display! I know how to do that now for the screen as you explained in your other post, but I’m not sure how to do that on a plane instead of the screen!

A8433b04cb41dd57113740b779f61acb
1
Reedbeta 167 Dec 12, 2013 at 23:56

So you want to have the drone’s camera zoom in so the target object just fits in the display? Well, I suppose you can look at this as a problem of calculating the FOV. The drone would rotate its camera so it’s pointing at (the center of?) the object, then zooming is done by adjusting the FOV.

So for each point on the drone, you could calculate the FOV needed to enclose that point, by looking at the angle between the camera-to-point vector and the camera’s view vector. Then find the maximum FOV out of those. (You can simplify it to avoid too many trig functions by finding the minimum dot product, instead of maximum angle, etc.)

Another approach would be to project each point to the near plane (basically a ray-plane intersection) and find their bounding rect in the 2D coordinate system implied by the camera (camera is above the origin, xy axes point to the camera’s right and up). Then you can plug this straight into your projection matrix function as the “left, right, bottom, top” inputs they usually have (in glFrustum for instance).

88dc730f0f71e55be39de0ad103bd9ff
0
Alienizer 109 Dec 13, 2013 at 00:19

I see. I didn’t exactly tried that, but I was simply zooming in bit by bit and re-calculating the object fitting in the plane and re-centered it, because each time I zoom in, the layout changes a bit. What I mean is, when the airplane is far from the drone, it looks more ortho than when the drone is very close, say, close to the tip of a wing, and it’s FOV=65 that wing looks very close and when you zoom just a little, that wing goes out of the view.

The other problem is the “center of?”, if I set my camera direction to the AABB center, it’s not always centered, especially when the airplane is rotated.

So what I’ve done is project the object on the plane, zoom in/out little by little until the object fits within the plane (not perfectly), then I move my camera so that the object is centered on the plane, and at that time, the object no longer fit in completely because of the FOV being so wide, so I redo all the above steps again until it’s good enough. May take 20+ passes. So that’s not the right way to do it, obviously.

So what you propose should take care of fitting the object within that view plane? But what about it being centered as well?

B5262118b588a5a420230bfbef4a2cdf
1
Stainless 151 Dec 13, 2013 at 01:20

You know the centre of the object, it’s 0,0,0

You know the location of the object

Therefore you know the centre of the object you need for displaying it (0,0,0) + position

To find the correct FOV you use the bounding sphere of the object. Take the radius of the bounding sphere, and the position of the camera to calculate the FOV. Think of it as a triangle with one point being the camera location, one point the position of the object, and one point the intersection with the bounding sphere.

The FOV you need is then obvious, it’s tan(angle) = sphere_radius / length(camera_position-object_position); FOV = 2 * angle;

Personally I like to increase the FOV slightly to allow a bit more of the background, but it works perfectly.

I use it all the time for racing games.

88dc730f0f71e55be39de0ad103bd9ff
0
Alienizer 109 Dec 13, 2013 at 02:01

I see what you mean, but the problem remains. Say for example my airplane is half way visible in the top left corner of my view plane. If I rotate the plane’s normal (which is the camera) to the direction of the AABB’s center (the airplane), the shape of the airplane will change (not the actual geometry but the perspective of it). At this point, I have to re-calculate how to center it in the view plane, and each time I rotate/move the view plane, the airplane perspective geometry changes, offsetting how centered it is.

I guess it’s something as simple as a “zoom extents” in a CAD software. So in other words, how do you “zoom extents” an object to the screen or a view plane?

A8433b04cb41dd57113740b779f61acb
1
Reedbeta 167 Dec 13, 2013 at 02:15

So the drone camera isn’t already pointing directly at the airplane? (I.e. the airplane is in the top left corner of the drone’s view)

In that case, you can still zoom to it, using the second method I posted in my last comment. It requires setting non-balanced values for the “left, right, bottom, top” parameters in the projection matrix. Basically it creates a skewed view frustum that corresponds to cropping/zooming a rectangle out of the original image.

But if the object is far from the center of the view it’ll still appear perspective-distorted when zoomed this way. And if you then animate the camera to turn to face the target over some amount of time, you’ll see the distortion changing, which seems like it could be weird.

88dc730f0f71e55be39de0ad103bd9ff
0
Alienizer 109 Dec 13, 2013 at 02:45

That was only an example. I do rotate the drone’s camera to the center of the airplane’s AABB so it is somewhat centered in the view. The problem I have at that point is to “zoom extents”.

Say for example you have the airplane perfectly centered on the screen. It is also rotated so that one of the wing is in a diagonal coming at you (bottom right), and have a FOV of 65+ If you move the camera away from the airplane so that the airplane appear half the size in the view, and draw a rectangle around the airplane, you’ll see that the rectangle is no longer centered, because one, that wing looks shorter as you move away, and, as you move away, the whole airplane’s perspective changes and affect how centered it was at first.

So if you do it the reverse way, the airplane is in the view but far away so it looks say half the size, it’s easy to make it centered and leave it, but as you zoom in, it’s no longer centered. So it seems we have to zoom/center, zoom/center, zoom/center as many times needed until it’s right, but that’s crazy isn’t it?

A8433b04cb41dd57113740b779f61acb
0
Reedbeta 167 Dec 13, 2013 at 03:07

OK, well, if pointing at the center of the BB isn’t centered well enough then the sheared-frustum method should take care of it.

88dc730f0f71e55be39de0ad103bd9ff
0
Alienizer 109 Dec 13, 2013 at 03:13

ah ok. But what do I do next after I’ve set the frustum??

A8433b04cb41dd57113740b779f61acb
0
Reedbeta 167 Dec 13, 2013 at 03:15

Uh…you render it?

88dc730f0f71e55be39de0ad103bd9ff
0
Alienizer 109 Dec 13, 2013 at 03:42

LOL. ok, seems too simple. You’re saying to project the object onto the view plane and get its 2D bounding rectangle. That part is easy, providing the object fit in the view entirely. But how does the frustum helps me with centering it? and how does this deal with “zoom extents”?

A8433b04cb41dd57113740b779f61acb
0
Reedbeta 167 Dec 13, 2013 at 03:47

Putting the bounding rectangle (assuming it’s been calculated in the correct coordinate system) into glFrustum (or a similar D3D function) gives you a projection matrix. You use that to render the scene. The image rendered by that will contain the contents of that rectangle. So if the rectangle is already centered on the object then the image will be centered.

88dc730f0f71e55be39de0ad103bd9ff
0
Alienizer 109 Dec 13, 2013 at 04:27

I see what you mean. But will this also account for the FOV? I mean, if the airplane is 1/3 the size of the view plane size, rendering as you suggest will only magnify it, or will it zoom it as in moving the camera closer? If it zoom it, then the airplane will no longer look the same as it get more skewed the closer it is to the camera, hence no longer centered!

A8433b04cb41dd57113740b779f61acb
0
Reedbeta 167 Dec 13, 2013 at 05:32

The bounding rectangle will get smaller and larger as the plane moves farther and closer away, so the plane will stay the same size on the image. It’s basically a 2D operation. BTW, “zoom” means to change the FOV, not to move the camera closer or farther.

88dc730f0f71e55be39de0ad103bd9ff
0
Alienizer 109 Dec 13, 2013 at 05:43

oh ok. I get it now. It’s basically the same as a real camera zoom, the camera doesn’t get closer to the object but the zoom lens changes the focal length.

But because my drones are not fixed in the sky, as the drone gets closer to the airplane, all that will change? I guess I just have to re-calculate the frustum as the drone moves?

A8433b04cb41dd57113740b779f61acb
0
Reedbeta 167 Dec 13, 2013 at 07:02

Yes, recalculate it as the drone moves. You’d have to do that anyway, as presumably the plane is also moving and not just staying fixed in one spot relative to the drone…

88dc730f0f71e55be39de0ad103bd9ff
0
Alienizer 109 Dec 13, 2013 at 17:02

Thanks Reedbeta (and everyone) for the help (and your patience). Have a great Christmas and a happy new year :)