For anyone reading this, I did exactly what I guessed above and it's working perfectly. The application-specific functionality is encapsulated in a wrapper rather than putting it up in the addin-level code. A couple things confused me but I fixed them:
When creating CommandButtons, assign a unique ID to all buttons, otherwise a user that has two inspectors up will see an event firing for every button with the same name each time they click a button. I used the local _inspector.Id+"B1" for button1, etc. The Id is a Guid that's generated when the Inspector is launched.
CommandBar objects don't seem to be destroyed with the wrapped objects that create them. So when adding items to a menu/toolbar from an inspector or explorer, you need to make sure a CommandBar doesn't exist before you create it. I haven't looked into the detail of why buttons are global but commandbars are not. I'll do this soon.
Helmut, you're on my short list of people to contact for paid consulting if I get stuck in this area. Thanks again.