William - Is your hobby a hobby for the good of souls/the church(perfect), or is it just a worldy activity which helps one avoid idleness(imperfect)? I suspect that is where the problem lies. Your "labora" is either not traditional or sanctifying. In the way of perfection(religious life), your labora must be fulfilling yet at the same time pleasing to God. If you want some ideas, look no further than the hobbies of monks. Monks paint holy images/icons, carve statues, make rosaries, grow their own food, and so on. Growing food for example is very uplifting and fulfilling. Pray to God about your hobby/labora. The examples I gave are few. There is much more labora to choose from. Pray about it, pray for an answer.
Also, monastics keep a routine. So, keep a routine. And, keep it simple. Because, being alone, you will likely not always keep it. But, simple enables you to easily get back up and on the horse again. Our lord, carrying the cross, fell three times. But, he got up and continued. Do likewise.