I think the ideal for daily wear is at least a polo shirt and some nice pants with a belt. Sweatpants and shorts, I would not say they are a sin against modesty (being inherently masculine articles of clothing due to being pants or pants-like, so one does not sin against their sex by wearing them). But they are very imprudent because they do not reflect or express authentic masculine gentlemanly behavior. Shorts are for boys, and have generally always been viewed that way throughout history.
T-shirts and sneakers are more tolerable, though still not the ideal (there are nicer shoes that are neither dress shoes nor sneakers/tennis shoes). I personally have little nicer clothing in comparison to t-shirts because I was not raised Traditional and clothing is expensive now. I am also more rough on clothes and so I don’t wear my polo shirts or nice pants every day. I have enough jeans to get by without wearing sweatpants. Also, men should comb their hair, not leave it disheveled (having naturally curly hair is not immodest, but keep it under control), and nice hats (not baseball caps, those I don’t have a problem with wearing those occasionally) are always welcome, but not required.
Obviously tight fitting is bad. Don’t wear tight jeans or other such clothing. I do not have a problem with short sleeved shirts for daily weekday wear, provided that they are not too short. One should wear a dress shirt for Mass if they can, not short sleeves.
These are just my personal positions and opinions about male modesty. Obviously, every man and boy has a different wardrobe and is only able to accommodate what they can. I know that thrift stores and places like Good Will have cheaper polos, and I plan to get some more shirts when I have some money to spend.