Thermal inertia can be a benefit: while it largely prevents you from benefiting from turning off the heat when you’re not home, it smooths out diurnal fluctuations and lets your control just the average temperature. So your windows can provide lots of solar gain, for example, without overheating you during the daytime.

The real problems are that stone is a terrible insulator, that single-pane windows are even worse, and that air leakage in old structures is incredibly wasteful.