I'm sorry, I don't understand what do you try to do. transformation matrices works well, when you need to recalculate coordinates from one coordinate system to another. For example:
1) You know the 3D point in global coordinate system (GCS) and you want to get its coordinates in sketch coordinate system (SCS). You can use ModelToSketchTransform matrix to calculate its coordinates (ignore Z value, it is always 0).
ModelToSketchTransform_Sample
Similar to project point, but without reference to original geometry
Dim part As PartDocument =ThisDoc.Document
Dim targetSketch As PlanarSketch = part.ComponentDefinition.Sketches("TargetSketch1")
Dim workPoint1 As WorkPoint = part.ComponentDefinition.WorkPoints("Work Point1")
Dim pt1 = workPoint1.Point
pt1.TransformBy(targetSketch.ModelToSketchTransform)
Dim pt1_2D As Point2d = ThisApplication.TransientGeometry.CreatePoint2d(pt1.X, pt1.Y)
targetSketch.SketchPoints.Add(pt1_2D, True)
2) You can manipulate with objects within coordinate system.
DrawHexagon_Sample
Move point from one vertex of hexagon to the next one and draw the lines between them
Dim part As PartDocument =ThisDoc.Document
Dim targetSketch As PlanarSketch = part.ComponentDefinition.Sketches("HexagonSketch1")
Dim pt1_2D As Point2d = ThisApplication.TransientGeometry.CreatePoint2d(0,0)
Dim startSketchPoint As SketchPoint
Dim endSketchPoint As SketchPoint
Dim mtxTranslation As Matrix2d = ThisApplication.TransientGeometry.CreateMatrix2d()
Dim vectorTranslation As Vector2d = ThisApplication.TransientGeometry.CreateVector2d(1,0)
mtxTranslation.SetTranslation(vectorTranslation)
Dim mtxRotation As Matrix2d = ThisApplication.TransientGeometry.CreateMatrix2d()
Dim center As Point2d = ThisApplication.TransientGeometry.CreatePoint2d(0,0)
mtxRotation.SetToRotation(PI / 3, center)
Dim mtxNextVertex As Matrix2d = ThisApplication.TransientGeometry.CreateMatrix2d()
mtxNextVertex.PostMultiplyBy(mtxRotation)
mtxNextVertex.PostMultiplyBy(mtxTranslation)
startSketchPoint = targetSketch.SketchPoints.Add(pt1_2D)
For i = 1 To 6
pt1_2D.TransformBy(mtxNextVertex)
endSketchPoint = targetSketch.SketchPoints.Add(pt1_2D)
targetSketch.SketchLines.AddByTwoPoints(startSketchPoint, endSketchPoint)
startSketchPoint = endSketchPoint
Next