The following answers are just my conventions I've adopted so you can always do something else.
Q: So I set this up in Page Set up Manager BEFORE I go to plot.
A: Page Setup Manager is used to save all the various settings when you go to plot like pages sizes, plot devices, pen weights & etc. This way you don't have to remember exactly what settings you used to generate the plot file.
Q: Do you name the set up?
A: Yes, as you can see from my previous screen shots, I had named the Page Setups starting with the Plot Device like "DwgToPdf...." followed by the Paper Size. But you can always name this starting with the Paper Size like "11x17...", "24x36...", "36x48..." and etc. This really depends on what makes it easier for you to see and select to Set Current in each of your Layouts
Q: What would you call the different sheet sizes?
A: As I mentioned in my previous answer, you can name the Page Setup based on the Sheet/Paper Size you plan to use to generate the plot. Some users like to even differentiate between Landscape (like 36x24) vs Portrait (like 24x36). Then the Page Setup name may look like DwgToPdf 36x24 vs DwgToPdf 24x36. Or you can name it by adding L or P at the end like: DwgToPdf 24x36L vs DwgToPdf 24x36P. Again the naming convention s is all dependent on what helps you identify what you’ve set within each Page Setup.
Q: Is it best to refer to the size in the title?
A: Again I typically just include the Plot Device & the Sheet/Paper Size in the Page Setup name because usually these are the options I would alternate when it comes to plotting. But if things change like if I need plot often using differing scales like: 1=1 vs 1=2 or 2=1 then I would most likely add this parameter in the Page Setup name as well like: DwgToPdf 24x36 1-1 vs DwgToPdf 24x36 1-2 vs DwgToPdf 24x36 2-1
Q: Once I set up the different sized paper in Page Setup Manager it seems like you can just bring all other drawings into the set ups. Is that correct?
A: Now that all the Page Setups are done, you can go into the Layout like 36x48-1-16 Scale, launch Page Setup Manager (command: PAGESETUP), then select from the Page setups list. Highlight the one you want that has all the plot settings desired and select Set Current. Do this on all the other Layouts and now whenever you need to Plot, there’s no need to make any changes within the Plot window.