Shop around to find the best price. That said, the best prices I have seen recently are at Liberty CPM, Tulving, Gainesville, Provident. All are reputable. If you do a Google search on those names you will find web pages discussing them and their pros and cons and can also find other dealers that way. Check the BBB website for complaints about any dealer in which you are interested. Tulving has the lowest prices but delivery can take a while. When checking price, pay attention to details like shipping and volume discounts. If you are buying silver, demand is currently high so it can take a few weeks to get delivery. Gold is generally not an issue in that way.
Silver generally speaking can be purchased in bar form, bullion coin form, and junk coin form. Bar has the lowest dealer premium but bullion coins issued by a government (e.g. American silver eagles) are probably more acceptable to more people when you want to sell. Junk coins are interesting -- American coinage pre-1965 was 90% silver, and a silver dollar that is 90% silver is worth (round number) about $20 today because of the debasement caused by the Federal Reserve. I believe it is actually so bad now that pennies and nickels cost more to produce than face value (which means you can make a profit by selling them for their metal content (which is undoubtedly illegal by the way)).
Local coin shops where I live keep records (of at least some transactions) by law. The records do not go to the govt but they certainly could should the govt demand them. Local coin shops also are subject to state sales tax laws and only some states have exemptions for precious metals purchases and then sometimes only for larger amounts.
The sell value can be impacted by condition so be careful not to scratch up whatever you buy. Some gold bullion coins in particular are .999 purity and scratch very easily. Sell value is also a consideration in deciding what to buy -- a bar from a minor mint may not be as liquid as an American Silver Eagle for example. For that matter, don't just look at the sell price, also pay attention to the buy price -- if you want to sell someday you don't want to have bought something at a really nice price that does not sell at a good price!
Precious metals is a vast topic -- there is lots of evidence of massive US government / bank manipulation of the precious metals markets to suppress prices to keep the dollar looking good.
Be careful about the websites where these topics are discussed, some of the best ones are frequented by people with very bad morals. They know money and economics but nothing of God. Reading the comments on such sites can be really informative but is akin to wading through a sewer.