How the infamous property evolved from a family home into a bunny haven