Hi, I just wanted to add an update to this for Max 2024. I rarely use Slate editor, prefer to stay in Compact. Since upgrading to 2024, all new materials have been showing up as teapots in material/map browser (saved libraries And scene materials). Old materials still have their previews. I was initially confused by the answer here, since like I said I'm in Compact mode... but I switched to Slate, found the teapot in low left, clicked it, went back to Compact and everything has a preview now!
Sure seems like there should be a setting for this in the browser options, but I don't see anything.