So many errors are due to people being unable to think in terms of distinctions (which is why I often get on my soapbox about them) and thereby creating false dilemmas.
Of course we are called to do all things for Christ. But not all things are DIRECTLY for Christ. When I go to the bathroom for a #2, that is directly at the service of nature, and indirectly at the service of Christ in so far as our nature is ultimately ordered toward Christ. That does not mean I need to put on some Gregorian chant while I do my business.
You fail to distinguish between -- 1) grace and nature, 2) merit, sin, imperfection, and even neutral things, 3) harmful, beneficial, or neutral.
And there are many permutations among these.
Then, on top of that you throw out a very broad and meaningless term "worldy" that requires proper distinctions.
Distinguo. If by "worldly" you mean songs with impure or blasphemous lyrics or lascivious melodies, concedo. If by "worldly," you mean Mozart, nego.
So the permutations between some of the above distinctions:
BENEFICIAL for grace and BENEFICIAL for nature
BENEFICIAL for grace but NEUTRAL for nature
BENEFICIAL for grace but HARMFUL for nature
NEUTRAL for grace but BENEFICIAL for nature
NEUTRAL for grace and NEUTRAL for nature
NEUTRAL for grace but HARMFUL for nature
HARMFUL for grace but BENEFICIAL for nature
HARMFUL for grace but NEUTRAL for nature
HARMFUL for grace and HARMFUL for nature
Of course, grace builds upon nature, so that, generally speaking, in broader terms, when you benefit nature, you benefit the potential for the life of grace to increase. But that's indirect.
Let's take listening to Mozart. I would hold that it can benefit nature (it stimulates the mind and uplifts the emotions). But directly it does not result in any increase of grace. Indirectly, however, by helping our nature, it may give more fertile ground for grace to increase. For a Jansenist, however, if it doesn't directly increase grace, it's bad or sinful (that's one of the chief condemnations of Jansenism by the Church). We are not required to do everything at all times to directly benefit grace and the supernatural life.
And, finally, as the Jansenists do, you make no distinction between God's commandments and the evangelical counsels. While it is sinful to break the commandments, it's not sinful to not always meet the standards of perfection. Perfection would have everyone living as celibate religious, but it is not sin to be married. And one could go on ...