Custom Texture Samplers in Quad Shaders

[Anonymous]

Hey everyone,

is there a way to change the default WRAP texture address mode in quad fx shaders?

It doesn't matter how I declare the texture sampler in my fx shader, it sticks to WRAP mode:

SamplerState gMainSampler {
	Filter = MIN_MAG_MIP_POINT;
	AddressU = CLAMP;
	AddressV = CLAMP;
};

It can write CLAMP, MIRROR, BORDER, but I always get the WRAP behaviour seen in the image bellow (UV address is shifted by float2(0.5, 0) in order to show the problem better).

 

Is there maybe something I'm missing or writing wrongly?

I'm sampling the texture like this (simplified code)

output = gRenderTex.Sample(gMainSampler, (i.uv + float2(0.5,0)));

[Anonymous · Accepted solution]

I have found the way to set custom texture samplers in shader instances (also quad shaders) within Maya. I'm sharing my findings here for anyone that might be interested.

First of all, any sampler state that you define in your *.fx or *.ogsfx file will be blatantly ignored. This means that if you write something like the following HLSL code:  

SamplerState gMainSampler {
	Filter = MIN_MAG_MIP_POINT;
	AddressU = CLAMP;
	AddressV = CLAMP;
};

That won't be happening.

Nevertheless, you still need to define your sampler state within your shader:  

SamplerState gTargetSampler;  

Sampler states within Maya are assigned using the Maya API, via the MSamplerState class. There are close to no examples in the devkit that illustrate its usage in VP2.0 overrides, so let me try to explain this on a high-level. As we will be using the Maya API, the examples apply to both, dx11 and openGL viewports.

Create a sampler state description

First of all, when initializing your render operations, you can create a MSamplerStateDesc, which will allow you to set your custom sampler state

//set sampler state
MHWRender::MSamplerStateDesc sDesc;
sDesc.addressU = sDesc.addressV = sDesc.addressW = MHWRender::MSamplerState::kTexClamp;

The example above will set the texture address mode in all directions to clamp, changing the default wrap behavior. You can change any attribute within the sampler state through the MSamplerStateDesc class.

Acquiring and releasing the sampler state

After the sampler state description has been set, you should assign it to the sampler state of your quad operation class during initialization:

//create a sampler state and acquire description
const MHWRender::MSamplerState *mSamplerState = MHWRender::MStateManager::acquireSamplerState(sDesc);

Make sure you also release the sampler state from the quad operation when the destructor of the MQuadRender is called:

//release sampler states
if (mSamplerState) {
	MHWRender::MStateManager::releaseSamplerState(mSamplerState);
	mSamplerState = NULL;
}

Assigning the sampler state to your shader

Once the sampler state is acquired, you only need to assign it to your shader instance. When initializing your shader instance (after you call the getEffectsFileShader() function) assign the sampler state:

mShaderInstance->setParameter("gTargetSampler", *mSamplerState);

There you go, now you should be able to assign any custom sampler states to your HLSL or GLSL shaders. I hope this helps anyone, it surely would have helped me a lot back then!