It's not Max's fault or Turbosmooths's fault. You need to model differently. Your two objects are just too low resolution to start with. They would smooth better if they had more polys. Secondly, when you add edges to hold things in place, you need to do that with more care to make sure that you have only 4-sided polys evenly spaced out in the areas you want to smooth nicely. The problem and the solution are better modeling and topology. This isn't a Max failure or issue. It's a model topology issue and not understanding how Turbosmooth calculates how to do it's job.
If you don't have the modeling experience yet to solve this by cutting edges and crafting topology I have a suggestion. Don't 'Boolean' right away. Instead, take you 'box' and your 'cylinder' and 'Turbosmooth' them first. Get them the way you want them with the poly count you need. Then, 'Boolean' them together. I've attached a file that does that. My file is a bit fast and not as clean as it could be but I'm short on time and just wanted to get you going. It might be a better way to approach this for now.
Rob Holmes

------------------------------------------------------------------------------------------------------------------------------------------
3ds Max (2023-2025), V-Ray 6.2, Ryzen 9 3950-X Processor, DDR 4 128MB, Gigabyte Aorus X570 Master motherboard, Sabrent Rocket NVMe 4.0 M.2 drives, NVidia RTX 4090, Space Pilot Pro, Windows 11 Pro x64, Tri-Monitor, Cintiq 13HD, Windows 11 x64
------------------------------------------------------------------------------------------------------------------------------------------