This error indicates possibly two things:
1. Extension is not installed properly on your machine. I am having 2021.1.502429796 version on my machine and it does have debugpy module so probably you can check how is the status on your side.
2. We assumed that extension is installed at the following location and add the path of debugPy module in python system path from this location. So another possible reason is extension installed at different path other than below path for you.
Windows: '%USERPROFILE%\.vscode\extensions'
Mac:~/.vscode/extensions
As a workaround you can try adding the path of debugPy module in to Fusion python interpreter system path and see if it resolve problem for you or you can keep the extensions at the above path.
I hope it helps. Thanks.
Shyam Goyal
Sr. Software Dev. Manager