Guess which one Holy Ghost came from? The German.
Strictly speaking, not really. The phrase has been native English since the Anglo-Saxons drove the Brythonic Celts into Wales, Cornwall, Scotland, and the sea. The phrase used by Bede, if I recall aright, is "Hālga Gāst" or "Hālig Gāst."
Anglo-Saxon—Old English, as it is also called—is a member of the Germanic language family, but at the time it was being spoken, the ancestral equivalent of modern German was, earlier, Old Frisian or Old Saxon and, later, Old High German. In short, the earliest form of a language called English slightly antedates the earliest form of a language called German.
Thus, "Holy Ghost" is not an import. It has been a term native to English from the beginnings of the language. In Chaucer's dialect, now called Middle English, the form was usually "hooly goost," but probably half a dozen other forms also appear in print.