Only thing I'd add to this, is that while small towns can be idyllic, they can also be pressure-cookers and fever swamps of deeply ingrained ideas and prejudices, people who have "been there forever", and appalling critical masses of ill will.
Before I would ever move to a small town anywhere, I would want to do my "detective work" to the nth degree. Go through the phone book (assuming they even have one anymore), and if you see half a dozen surnames in abundance, sometimes strange names you'd never see in those large numbers anywhere else, be assured that these people's families have been there ever since Moses was a little boy, and they run everything. In a lot of those places, the "townies" aren't going to be much amused by someone saying "I was looking for a nice, quiet, wholesome place to raise my family". They might be hospitable, or they might be resentful as hell, and say to themselves "you want 'nice, quiet, and wholesome'? --- we'll fix that real fast!". It all depends on where you go.
I consider myself blessed in that, in all of our moving around, we have finally found a mid-sized metropolitan area that is just small enough to be friendly (and slower-paced in that quintessentially Southern way), but large enough that people are pretty much spread-out geographically in fairly large numbers, have a certain amount of self-absorption (it's harder to mind other people's business when you're so busy minding your own, keeping body and soul together), and really don't give a rosy red rat's rump what you do, and what you don't do. Twenty miles in any direction, and you might as well be in another country --- my son and I went exploring in our nearby hill country this weekend, and discovered an area vaguely reminiscent of the Smoky Mountains in miniature, airy, dulcet microclimate, I was prompted to look for real estate with the thought in the back of my mind that I might finally have found that ideal spot for camping, hunting, and target practice --- bearing in mind, though, that we had wandered across an unmarked county line (it was that isolated) into a neighboring county that is unaccountably lacking in prosperity, and where for all practical purposes segregation still exists, the blacks go to public school, the whites almost all go to a private academy that exists for one purpose and one purpose only. Not that we'd be doing either public or private school in the first place, but no wonder nobody wants to move there or set up shop there. Sounds like it would be a little toxic to me.
Hell hath no fury like a small-town busybody. Look before you leap!