Puerto Rico is under US sovereignty and Puerto Ricans are, by law, US citizens. The same is true of all the other territories you mentioned, except that American Samoans are US nationals, not US citizens. Residents of any of these territories are free to come to any state of the US and live as long as they wish to. There is no reason why their driver's licenses should be treated any differently for reciprocity purposes. In the case of Puerto Rico, driving skills are not language-specific (road signs would be an exception).
An American driver's license is ridiculously easy to get, compared to many other countries, and compared to what it should be. As it stands now, any dimwit who can pass a very simple written and road test can get a driver's license for life, and usually it can be reciprocated by any other state. In any locality of any size, you see accidents right and left throughout the day, many if not most caused by inattentive or just plain unintelligent drivers.