An outpost or base is a collection of modules.
All new players start out with the essential buildings - The Bridge, Arms Lab, Ship Lab, Tech Lab, Ship Factory, Fleet Bay and Workshop - all at level I. They also have a single Helium-3 Miner I and Mineral Ore Miner I, as well as a Combat Module I.
Modules in player outposts (except for Terminus Mines) must be connected to a power source, i.e. The Bridge. Read more about that here.
You can scrap any non-essential module except for the Commerce Module via the Upgrade menu, which removes it and refunds a portion of the original resource cost.