Tradman, did you actually read the entire On the Literal Meaning of Genesis or do you base your understanding on isolated quotes? I find it hard to believe that a person who had read the whole thing would misunderstand it as badly as you do.
Here is a passage in which St. Augustine is being explicitly critical of literal interpretation:
19. "And God said, 'Let there be light,' and the light was made." We ought to understand that God did not say, "Let there be light," by a sound brought forth from the lungs, or by the tongue and teeth. Such thoughts are those of carnal persons, and to be wise in accord with the flesh is death. "Let there be light," was spoken ineffably. One can ask whether what was spoken is the only-begotten Son. For what was spoken is called the Word of God by whom all things were made. Only let us banish the impiety of believing that the Word of God, the only-begotten Son, comes about like a sound uttered by us.