Our house had no heating system installed when we bought it. I've installed a cheap Chinese heat pump to compliment an electric oil column heater and a small fan heater.
The heat pump was baught off TradeMe from NightDragon for $1100 in 2004. The system I baught uses up to 1950 Watt electricity which is the maximum size you can plug into a standard wall socket. Any bigger and you have to get it wired up by an electrician. It outputs 5400 Watts of heat (according to the sticker).
I installed it 90% and got an air conditioning firm to "commission" it - which means they finished off the copper tubing, then evacuated the tubes so the system worked more efficiently than if I'd just connected it up myself.
Works very well. Not as quiet as more expensive systems. But the warmth is fine.