You shouldn’t get all the values be zero in that case. Calculate [A,B,C] as the normal of the plane. That is, if the three points of the triangle are P1, P2, P3, then let [A,B,C] be the vector cross(P2 - P1, P3 - P1). Then take D = -[A,B,C] dot P1 (which is simply the solution of [A,B,C] dot P1 + D = 0.

I want to determine whether a point lies inside a triangle or not. and If no, then determine to which side of triangle it lies, to back side or front side. I have followed many papers on internet.

I only had to create plane’s equation:-

Ax + By + Cz + D = 0.

Determine A,B,C,D and put the point into that equation. If its zero it lies

inside the plane.If > 0 then on 1 side(u can call it front side) and If < 0 then on the other side(say back side).

I did but I am facing problem.

Assume A(0, 0, 0), B(10, 0, 0), C(20, 10, 0). I get all A,B,C,D zeros.

I cant proceed onwards.

What I be doing wrong. Plus on another fast and good solution. Please do solve with this example.

Thanx alot for any help.