It's difficult to impossible to keep a veil on a baby or toddler. So when it's done, it must be temporary, "for fun", and non-mandatory.

Personally, I don't look askance at a 3 or 4 year old without a veil on, the same way I would an adult woman missing a veil.

I should think that a young girl should wear a head covering at Mass when she is able to do so.  I've seen young girls wearing a veil as young as two or three while I've seen young girls around four of five who still didn't keep them on very well.  There's one little girl at Mass who looked for a veil very at a very young age because she saw that her mother had one.  

Much of this simply involves instilling discipline.  When a toddler is given a veil for Mass as just an expected thing for her to wear, most will do so--for the most part, anyway.  It is very similar to not allowing horseplay in the pew, requiring very young children to stand, sit, and kneel when they are able to do so.  Etc., etc., etc.

There is no age for proper dress and behaviour at Mass, or indeed, for any endeavor.  Problems arise when parents train their children to be children rather than allow their children to be children while training them for life.

I think baby bonnets are a good choice for infants and toddlers. There are certain designs more suited for little girls that are easier to handle than the traditional veil. Mothers could opt for those in the earlier years. I do think that as with everything Catholic, girls should be taught to properly cover their heads during Mass at the earliest opportunity, as part of the Church protocol.

I asked my 90-something parents about this.  Here is their response.

Decades ago, little girls wore hats to match their outfits.  Most women wore hats as well.  Those wearing veils were mainly older girls and women of Hispanic origin. Baby girls wore bonnets, but often, very young infants and toddlers weren't brought to mass until old enough to not cause a distraction.  

In those days, (1920s-1940s), mass was close by and there were 3 or 4 every Sunday and holy day.  Parents took turns while one stayed home with the babies.  School aged children went to 9:00 mass and sat up front together where they were supervised by the nuns.  Communities were safe so that children walked to Church on their own.  In both my parents' families, Dad went to the early mass while Mom got the older children ready for 9:00 mass and got the roast in the oven.  Mom and grandma/grandpa heard the late mass, around 11:00. By the time everyone returned, the family gathered for Sunday dinner, served mid-afternoon.  

Most girls and women wearing veils and the problem of keeping them on little girls' heads is a recent is a recent development, much as churches having "cry rooms." IMO, girls not able to wear a veil without distraction should wear hats that are tied on.  Baby girls should wear bonnets. Parents of children apt to create distractions should sit where a quick retreat can be made.  This is one area where adjustments to suit the times are necessary to preserve the reverence due Our Lord during Holy Mass.

Here is our youngest (taken many months ago):

The girls all have hats, too, and they seem to prefer them to veils.

I agree that they should wear veils from "as long ago as they can remember" -- that makes it easier. Otherwise, there's one fine day that you expect something of them that you didn't expect the week before.

Same goes with acting modest, same goes for having your girls wear skirts/dresses, same goes for attending Mass every Sunday, and the same goes for abstaining from meat on Friday.

It's easier to have all our kids abstain from meat, even those under 7. If we have some meaty leftovers that need to be consumed on Friday, we give them to the youngest (2 years old) -- someone who doesn't even know what Friday or fasting is yet.

Even if in some cases I'm "early", I figure better early than late.

Some of those are applied much easier on a practical level than others.

No meat on Friday? Sure, no problem.

No shorts / pants? Easy enough.

Sit through Mass under age 2? Um... now it's getting difficult.

Keep a hat or veil on a baby, toddler, or even older? That's not a battle worth fighting. They'll do it fine on their own when they want to be like the big girls. Have a hat/bonnet/veil available every week, put in on them when you enter the church, and then don't sweat it if it doesn't stay there more than a few minutes. Once they've masted sitting still and quiet during Mass, then you can work on keeping a hat on their head.

I have just asked the same question on the women only forum. 

My daughter has covered ‘to be like mummy’ since she was 4 and we would have expected it from her first Communion but I see that many here believe it should be as early as possible which I think we will apply if we are blessed with another girl.