NEW YORK (CNN) -- Charles and Anna Burge were so certain of retiring to Florida this year that they had begun packing up their house. The dining room of their Long Island, New York home is barren, spare a virtually empty china cabinet and a pile of cardboard boxes that hold its former contents. The couple sold their kitchen table, got rid of chairs, took pictures down from the walls and began sending belongings to their Florida condominium.

'We sold our dining room chairs and now we're sort of living half and half not," said Charles, a superintendent with New York City's Sanitation Department.

The Burges weren't counting on a housing market collapse, a stock market tumble, and an energy squeeze that's sent gasoline prices above $4-a-gallon in their town of Coram, New York.

"Does it make sense to retire now? No. Do I want to retire now? Yes," lamented 53-year old Charles.

Anna was devastated when the couple realized they had to defer their dream.

"I was mad with my husband for two weeks when he came home he said, 'We can't go'. I'm like, 'What do you mean we can't go?'"

"I was crying I just want to go. My son lives in Florida and I like to be close with my son," said Anna.

As the economy stumbles, a growing number of baby boomers are pushing back retirement plans, 27% of workers aged 45 and over, according to a survey released Tuesday by AARP.

Labor Department data back up the poll, showing more middle-aged and older Americans are joining the labor market. During the 12-months ended in April, 915,000 Americans aged 55-64 entered the workforce, and another 389,000 aged 65 and older joined the job market, according to the Bureau of Labor Statistics.

"People are scared," said David Frisch of Frisch Financial Group, the Burge's financial planner, who says he has other clients who are putting off retirement.

"The ultimate question is how long is the market going to be suffering, how much will expenses rise, how long will it take the housing market to come back into play?" said Frisch.

Charles Burge was able to consider retirement in his mid-50's because of the generous pension he's earned from New York City, nearly 2/3'rds of his salary. But selling his home is part of the plan as well. When real estate agents told Charles his home had lost 15% of its value, and could take the better part of a year to sell - even at its reduced price - he began to realize Florida would have to wait.

"I have college to pay for. I have gas to pay for, for three cars," said Charles. "How do you retire when all these things are going out of sight?" The couple's revised plan now is to retire in three years when they hope the real estate market will have recovered. Anna is counting on it, intending to leave her fine china packed away until she can use it in Florida.

"Everything stay in the box," said Anna. "It's too much...aggravation to take everything out of the box again and then start to pack again."