We would never consider having our childrens' "picture taken with the Easter Bunny".
In fact, we don't even do "the Easter Bunny" around here.
That having been said, we do color hard-boiled eggs on Holy Saturday, do an Easter egg hunt on Sunday, and each child gets a (re-used) Easter basket filled with various candy, plastic eggs, and small toys/books (we don't buy the pre-made ones; ours are put together by hand)
And of course we have a big party with the family on Easter Sunday (after Mass).
In general, we try to extract the GOOD things that "everyone does at Easter" -- the things that are compatible with the True Meaning of Easter.
No one can say we're not celebrating Easter; yet we completely ignore the Easter Bunny.
Our celebration of Christmas is similar: lots of songs (mostly religious ones), Christmas Tree, presents for each child, parties and food with relatives -- but no Santa.