Harmony is a library for patching and overriding C# code at runtime. I think this would be extremely powerful if addons could patch and modify the gamemode, rather than just slotting into predefined places. If you can find a way to sandbox it sufficiently, it might be worth looking into.
That way, gamemodes don’t have to be specifically designed for addons, and addons can do more than just what the game/gamemode provide APIs for. UnityModManager is an example of how to implement mods/addons this way - perhaps you would want to go with a more managed approach to guarantee that an addon’s patches are unloaded when the user requests for the addon to be unloaded in the UI.
The downside is that the power it gives requires a deep understanding of what it is you’re trying to do, and careless addons could break compatibility or even break the entire game in subtle ways. If you decide to go this route, it might be worth providing some easier options on top, so that addons can either use Harmony or just go with gamemode predefined slots (like adding new entities or toolguns in sandbox).
“Predefined slots” like adding new entities/toolguns/whatever would probably have to be added by the gamemode. I don’t think it would fit s&box to add this functionality itself, especially since those concepts might not exist in other gamemodes.
Gamemodes could declare which entrypoints they support, mods could declare which entrypoints they provide, and at runtime the gamemode could go over the entrypoints that are enabled and call into them however they like. A similar concept is used in Fabric, which is a powerful and lightweight modloader for Minecraft. In Fabric, mods can also call into the entrypoints of other mods, allowing for some pretty powerful integrations. S&box could probably support this too.
Looking forward to hearing feedback on these ideas.