I think having a section for recently used modifiers as the first section of Modifier List will be handy. It should be context sensitive and the maximum number of modifiers in this section can be set through preferences (please see the picture below).