Do as you see fit, but if your house burns down, all you have left is a vacant lot (with ruins that have to be cleared off before you can rebuild, all out of one's own pocket).
Here's an example of what happens when you don't have insurance:
https://wchstv.com/news/local/powerball-winner-jack-whittakers-home-in-virginia-destroyed-by-fire
Surely he could have afforded the premiums, but his life was one big train wreck in slow motion. Worst of all, his beloved granddaughter died under mysterious circuмstances and they didn't find her body for several days. He said he wished he had just torn up that lottery ticket.
We are aware of the risks.
12 years ago my parents bought the property that our house is on. They were going to tear it down. They bought the land for farming, not for the house. But my mom likes a good project. We moved in 5 years ago to an uninsured house and decided to keep it that way being aware of the risks. We are also aware of what crooks insurance companies can be too. There is no guarantee your insurance company will pay out when you need them to after a disaster. My husbands family found that out the hard way after losing a house that was insured.
Leaving your house uninsured is not something everyone should do. In many, even most cases it would be imprudent, I agree.
Thankfully my husband has built many houses, roofed many roofs, etc. For us to replace our humble little home in the country it would require a small loan for building materials and time to build it back up doing the work ourselves. We prefer that over some man at a desk telling us what we can and can’t do with our house over something that statistically is unlikely to happen.