Fatigue is only detrimental to your spiritual life if you cause it yourself by excessive fasting, work, etc. If you cause it yourself, you are at fault for your neglect of family duties, spiritual duties, etc. That's why we're supposed to have spiritual directors -- so they can slap us when we suggest fasting on bread & water for a week.
Seriously though, they will tell us that it's probably pride, the devil, etc. tempting us to overdo it.
Time (and energy) must be set aside for prayer -- just like collection money should be set aside first, before you start paying bills. Otherwise the money has a tendency to be gone before you can put something in the collection. Same for personal energy to say prayers.
If you find you're "zonked" by 9 or 10 PM, say the Rosary earlier.
I've experienced this personally, so I've learned how to deal with it. You get to know how much energy you have in a day, and you can plan accordingly.