# evaluation of uv coordinates and normals in domain shader

6 replies to this topic

### #1rouncer

Senior Member

• Members
• 2758 posts

Posted 16 March 2011 - 09:37 AM

Im writing a highpoly lowpoly raster program, but ive come across a problem when I finally try to tesselate it and bring it on screen.
Its working quite well, I can get it done in less than a second with a simple hipoly sphere.

Im using tri patches, and im getting correct reading on the position, but incorrect reading on the normal and texture coordinates.

heres my domain shader, I cant think of what im doing wrong...

[domain("tri")]
DS_OUTPUT DS_T( HS_CONSTANT_DATA_OUTPUT input, float3 BarycentricCoordinates : SV_DomainLocation, const OutputPatch<HS_CONTROL_POINT_OUTPUT, 3> TrianglePatch )
{
DS_OUTPUT output = (DS_OUTPUT)0;

float3 vWorldPos = BarycentricCoordinates.x * TrianglePatch[0].vWorldPos + BarycentricCoordinates.y * TrianglePatch[1].vWorldPos + BarycentricCoordinates.z * TrianglePatch[2].vWorldPos;
float3 vNormal = BarycentricCoordinates.x * TrianglePatch[0].vNormal + BarycentricCoordinates.y * TrianglePatch[1].vNormal + BarycentricCoordinates.z * TrianglePatch[2].vNormal;
vNormal = normalize(vNormal);
float2 texCoord = BarycentricCoordinates.x * TrianglePatch[0].texCoord + BarycentricCoordinates.y * TrianglePatch[1].texCoord + BarycentricCoordinates.z * TrianglePatch[2].texCoord;

// Displace vertex along normal
//vWorldPos += vNormal * (layer0.SampleLevel(samLinear, texCoord,0).r-0.5f)*10;

output.vPosition = mul( float4(vWorldPos.xyz, 1.0), wvp);
output.texCoord=texCoord;
output.vNormal=vNormal;

return output;
}


you used to be able to fit a game on a disk, then you used to be able to fit a game on a cd, then you used to be able to fit a game on a dvd, now you can barely fit one on your harddrive.

### #2Reedbeta

DevMaster Staff

• 5344 posts
• LocationSanta Clara, CA

Posted 16 March 2011 - 05:59 PM

Incorrect how? Can you post a screenshot that shows what's going wrong?
reedbeta.com - developer blog, OpenGL demos, and other projects

### #3rouncer

Senior Member

• Members
• 2758 posts

Posted 16 March 2011 - 06:35 PM

I know, its totally right isnt it, something screwy is going on.

ok, ill get a screenshot.

ok, its a little hard to see what it is, but its actually a "reducted" sphere, the picture in the background is the displacement map ive raytraced (in a few milliseconds ;)) to make the lowpoly model push out into the original sphere. (it may have bugs yet tho, but i havent had the chance to see it working to fix it...)

as you can see the uv's are completely all over the place, but as you can also see my displacement map has the 6 faces of the reducted sphere (its box wrapped) fine.

If I did a normal output, they all tend to point in the same direction?!!?

One thing tho, its a pretty degenerate reduction, as in there is lots of skinny triangles... could that be the problem? (im yet to do a proper gridded reduction to keep all the lowpoly triangles the same size)

Just so you can see I havent made an obvious mistake, heres the whole set of shaders->

struct VS_T_INPUT
{
float4 pos : POSITION;
float2 uv : TEXCOORD0;
float3 nor: NORMAL;
};

struct HS_T_INPUT
{
float4 pos : POSITION;
float2 uv : TEXCOORD0;
float3 nor : NORMAL;
};

struct HS_CONSTANT_DATA_OUTPUT
{
float	Edges[3]		 : SV_TessFactor;
float	Inside			 : SV_InsideTessFactor;
};

struct HS_CONTROL_POINT_OUTPUT
{
float3 vWorldPos : POSITION;
float2 texCoord  : TEXCOORD0;
float3 vNormal   : NORMAL;
};

struct DS_OUTPUT
{
float4 vPosition		 : SV_POSITION;
float2 texCoord         : TEXCOORD0;
float3 vNormal          : NORMAL;
};

HS_T_INPUT VS_T(VS_T_INPUT Input)
{
HS_T_INPUT Output;

Output.pos=Input.pos;
Output.nor=Input.nor;
Output.uv=Input.uv;

return Output;
}

HS_CONSTANT_DATA_OUTPUT ConstantsHS( InputPatch<HS_T_INPUT, 3> p, uint PatchID : SV_PrimitiveID )
{
HS_CONSTANT_DATA_OUTPUT output = (HS_CONSTANT_DATA_OUTPUT)0;

output.Edges[0] = 60.0f;
output.Edges[1] = 60.0f;
output.Edges[2] = 60.0f;
output.Inside   = 60.0f;

return output;
}

[domain("tri")]
[partitioning("fractional_odd")]
[outputtopology("triangle_cw")]
[outputcontrolpoints(3)]
[patchconstantfunc("ConstantsHS")]
[maxtessfactor(60.0)]
HS_CONTROL_POINT_OUTPUT HS_T(InputPatch<HS_T_INPUT, 3> inputPatch, uint id : SV_OutputControlPointID )
{
HS_CONTROL_POINT_OUTPUT	output = (HS_CONTROL_POINT_OUTPUT)0;

// Copy inputs to outputs
output.vWorldPos =	inputPatch[id].pos.xyz;
output.vNormal =	inputPatch[id].nor.xyz;
output.texCoord =	inputPatch[id].uv.xy;

return output;
}

//--------------------------------------------------------------------------------------
//--------------------------------------------------------------------------------------
[domain("tri")]
DS_OUTPUT DS_T( HS_CONSTANT_DATA_OUTPUT input, float3 BarycentricCoordinates : SV_DomainLocation, const OutputPatch<HS_CONTROL_POINT_OUTPUT, 3> TrianglePatch )
{
DS_OUTPUT output = (DS_OUTPUT)0;

float3 vWorldPos = BarycentricCoordinates.x * TrianglePatch[0].vWorldPos
+ BarycentricCoordinates.y * TrianglePatch[1].vWorldPos
+ BarycentricCoordinates.z * TrianglePatch[2].vWorldPos;

float3 vNormal = BarycentricCoordinates.x * TrianglePatch[0].vNormal
+ BarycentricCoordinates.y * TrianglePatch[1].vNormal
+ BarycentricCoordinates.z * TrianglePatch[2].vNormal;
vNormal = normalize(vNormal);

output.texCoord = BarycentricCoordinates.x * TrianglePatch[0].texCoord
+ BarycentricCoordinates.y * TrianglePatch[1].texCoord
+ BarycentricCoordinates.z * TrianglePatch[2].texCoord;

//vWorldPos += vNormal * (layer0.SampleLevel(samLinear, output.texCoord,0).r-0.5f)*10;

output.vPosition = mul( float4(vWorldPos.xyz, 1.0), wvp);
output.vNormal=vNormal;

return output;
}

float4 PS_T(DS_OUTPUT Input):SV_TARGET
{
float col=layer0.Sample(samLinear, Input.texCoord).r;
return float4(col,col,col,1);
}



Although note If I dont tesselate, the uv and normals are fine.... i can get you a screenshot of that if you dont believe me.
the fact I can raster a completely correct displacement map means the uv information is intact and operational.
(i use the exact same lowpoly buffers to raster the displacementmap as to actually render it tesselated)

Its times like these that really bring me down and make me think im gonna get nowhere with my programming career. :{

* it doesnt matter how much i tesselate, even at 1, (no tesselation) it still doesnt show me correct uv's or normals...

* if i use one corner of the tri for the uv and normal, its actually slightly more correct?? possibly, well the normal at least starts pointing
around the sphere, the uv pretty much looks clamped tho like that.

If you have no idea, I guess ill try doing something drastic like rewording the shader all over, or maybe change to quad patches, its a pity
it didnt work tho.
you used to be able to fit a game on a disk, then you used to be able to fit a game on a cd, then you used to be able to fit a game on a dvd, now you can barely fit one on your harddrive.

### #4rouncer

Senior Member

• Members
• 2758 posts

Posted 17 March 2011 - 01:24 PM

no idea hey, i guess this is the tesselation piece of shit it is.
you used to be able to fit a game on a disk, then you used to be able to fit a game on a cd, then you used to be able to fit a game on a dvd, now you can barely fit one on your harddrive.

### #5rouncer

Senior Member

• Members
• 2758 posts

Posted 18 March 2011 - 11:04 AM

here it is when i dont tesselate, see the uv is fine, only when i tessellate does it become jumbled.
you used to be able to fit a game on a disk, then you used to be able to fit a game on a cd, then you used to be able to fit a game on a dvd, now you can barely fit one on your harddrive.

### #6rouncer

Senior Member

• Members
• 2758 posts

Posted 18 March 2011 - 11:09 AM

oh silly me, i didnt set the vertex layout... holy shit, sorry guys :)

well here it is working, except theres seam error, complete reproduction of a sphere from a lowpoly object!!!

the lowpoly object is about a 12th of the size of the hipoly object, so im making about a 20% size reduction including the
8 bit displacement map (which compresses on disk significantly further), which is what displacement mapping is about, compression.

Ive got a feeling displacement map compression is better than voxel compression, those who know about the flame war on there. (i love john carmack :))
you used to be able to fit a game on a disk, then you used to be able to fit a game on a cd, then you used to be able to fit a game on a dvd, now you can barely fit one on your harddrive.

### #7rouncer

Senior Member

• Members
• 2758 posts

Posted 18 March 2011 - 04:57 PM

heres a slightly better normal output of the sphere (theres no normal map cheating, this is the actual output from a 16 bit displacement map, i think 16 bit is better even if it compresses it less, it makes a better silhuette), so its mostly right just a few cracks and bumps around the place, nothing major. :)

ive decided to leave the artifacts as they are and try a complex made model, see how good it goes, if it works ive got a really nifty idea for
a new kind of 3d modeller...
the real plus to this is it happens in less than a second from totally hipoly to lowpoly.
you used to be able to fit a game on a disk, then you used to be able to fit a game on a cd, then you used to be able to fit a game on a dvd, now you can barely fit one on your harddrive.

#### 1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users