Thank you for your question.
You can find the sample "XPSTest" from DWF Toolkit package that shows how to use the definePageLayout:
WT_Matrix2D oTransform;
{color:#999999}//
// This transform must match the one on the graphics resource exactly
//{color}
double fScale = 0.0016792611251049536;
oTransform(0,0) = fScale;
oTransform(1,1) = fScale;
oTransform(2,0) = 4.3706968933669188;
oTransform(2,1) = 6.1601595298068847;
{color:#999999}//{color}
{color:#999999}// The heigh and width values must match those of the section's paper
// the scale is a factor to scale the units given in the page size to inches...
// so if the paper size is in mm, it should be 0.03937
{color}{color:#999999}//{color}
ERR_CHECK( pxFile->definePageLayout(oTransform, 11, 8.5, 1) );
Please let me know if you have any follow-on questions.
-Herbert