@jeremytammik
I tested your formula and the Original to see where the switch point from true to false lies.
As you suspected the distance from the switchpoint to the line is dependant on the length of the line, however your formula makes things worse.
My suggestion for the formula : return ((da + db) - f) * f < tolerance;
Here are my results: (tolerance = 0.0001)
Line length 1
Original switch point (0.007100000, 0.500000000, 0.000000000)
Jeremy switch point (0.007100000, 0.500000000, 0.000000000)
FAIR switch point (0.007100000, 0.500000000, 0.000000000)
Line length 5
Original switch point (0.015850000, 2.500000000, 0.000000000)
Jeremy switch point (0.035400000, 2.500000000, 0.000000000)
FAIR switch point (0.007100000, 2.500000000, 0.000000000)
Line length 10
Original switch point (0.022400000, 5.000000000, 0.000000000)
Jeremy switch point (0.070750000, 5.000000000, 0.000000000)
FAIR switch point (0.007100000, 5.000000000, 0.000000000)
Line length 50
Original switch point (0.050050000, 25.000000000, 0.000000000)
Jeremy switch point (0.353600000, 25.000000000, 0.000000000)
FAIR switch point (0.007100000, 25.000000000, 0.000000000)
Line length 100
Original switch point (0.070750000, 50.000000000, 0.000000000)
Jeremy switch point (0.707150000, 50.000000000, 0.000000000)
FAIR switch point (0.007100000, 50.000000000, 0.000000000)