Did the Church ever criticize or condemn the Olympics before V2?
Well, keep in mind that the Olympics didn't even exist from a.D. 390 until 1896.
Although I am not aware of any official Church pronouncements, I think it is significant that it was Emperor Theodosius the Great -- the same Roman Emperor who first made Catholicism the official state religion of the Roman Empire -- who put an end to the Olympics by condemned and banning them as pagan and unworthy of Christian culture, with the approval of the Catholic bishops (Saint Ambrose in particular).
I think it is also significant that for 1,400 years, when Christendom was relatively strong, no one saw the need to bring them back. The French Revolutionaries, on the other hand, thought it a good and important thing to do although their attempt fizzled out. It was then in 1896 when they were re-established -- hardly the pinnacle of Christendom.
That the Church did not make any pronouncements between 1896 and 1958 is not all that surprising, given that they had "bigger fish to fry" in the form of Modernism and two world wars, plus the popes after St. Pius X were all pretty weak (although not heretics).