Politically correct term is dwarf. Midget is derogatory. The genes encode for the structure, the species is why they are all the same. Since we are humans we share the same codes, its just the genes that make a dwarf a dwarf.
This particular trait is a genetic anomaly which causes them to look this way, as such when the anomaly occurs the traits tend to be remarkable similar. The thing to keep in mind is that genetic code is based on patterns which then repeat, if they didn't our experience of life would be dramatically different. My point being is that when an anomaly occurs the basic code remains the same short of the anomaly, so it follows a pattern and attempts to incorporate the anomaly with some degree of balance. However within this subject it is important to understand that there is a difference between what is typically characterized as dwarfism and those we call midgets, in the sense that "dwarfs" will have normal sized heads and trunks with shorter legs and arms overall while "midgets" are just smaller all the way around while still remaining proportionate. Additionally there are other genetic issues which an cause similar effects without necessarily being the two traits listed above. Now I have put those traits in quotes so as not to offend anybody who might have those traits, as my goal is to explain this to the best of my knowledge. At any rate that is why you see similar bone structures within the group of people who share that trait. You will also note that there are similar repetitive traits for, say people who have Down Syndrome, in the sense that there are certain physical and mental characteristics that are both common and noted. I hope this helps you understand this a little better.