The core "unload" button in the UI currently will completely remove a core from solr.xml. With the implentation of transient cores, there should be a way to ask Solr to unload a core without removing it entirely.
This leads into a discussion about terminology. UNLOAD isn't a good single-word description for what it does. A case could be made for having REMOVE and DELETE actions for CoreAdmin, with confirmation prompts if you click on those buttons in the UI. DELETE could simply be an option on REMOVE - which I think you can actually currently do with UNLOAD.
Another idea, not sure if it needs its own issue or is part of this one: If a core is mentioned in solr.xml but not actually loaded, it would be very cool if it were listed, but with a different background color to indicate the non-loaded state.