There's many ways to define physical sex, since there are many aspects to it.
Chromosomal
Genetic
Phenological (Gonadal and Penis/Clitoris Size)
Hormonal Response
The Intersex Society of North America (
has more information. The generally accepted estimate of the incidence of intersexuality is 1 in 1000, though figures vary because of disagreements over the definition of intersex.
Chromosomes:
The majority of human beings have 46 chromosomes, which come in 23 pairs of 2. One pair is sex related. We inherit these from our parents; their sex cells each contribute 1 of each of the 23, which results in our having 23 pairs of 2.
Sometimes the gametes (sex cells) are missing a chromosome or have a full pair of a chromosome. The resulting child will then have 1 or 3 in a set of chromosomes.
Normally, this results in a fetus that won't anchor to the womb. If the child survives, abnormal development usually results. For example, Down Syndrome is caused by the presence of 3 of a particular chromosome (I think #21).
For sex, the mother usually contributes a big chromosome (known as X). The father contributes a big chromosome or a little chromosome (known as Y). XX is chromosomally female. XY is chromosomally male.
If the child just has an X, she is considered chromosomally female with Turner syndrome. If the child has XXX, she is considered chromosomally female with TriplioX syndrome. If the child has XXY, he is considered chromosomally male with Klinefelter syndrome. Y will abort (at least an X is necessary for survival). XYY is not possible.
Genetic
Sometimes, the chromosomes are missing the genes that determine sex. For instance, Y, which signifies male, may be missing the genes necessary for male development. Therefore, a person with XY (chromosomal male) would develop as someone with X (chromosomal female with Turner syndrome). This is one example of a person with conflicting definitions of physical sex.
Phenological
This is usually the most telling sign in the doctor's office, since it's what our eyes can see. Usually, the doctor checks for the presence of a penis or penis-like structure. If it is over a certain length, the child is declared male (phenologically). If it is under another length, the child is declared female. Sometimes, the length is in between. Then the child is considered intersex.
Beyond that measure, though, is the state of the gonads. A child may have a penis with ovaries. Or undescended testicles and a penis, but a vulva instead of a scrotal sac. Or a child may have ovaries and a vulva, but no vagina. If you can name it, it probably exists.
Hormonal Response
Sometimes a chromosomal male will lack the genes necessary to respond to androgens (male sex hormones). This is Androgen Insensitivity (AIS). This individual is XY, with small testes, no uterus, no penis, and perhaps a short vagina with no cervix.
The confusion arises because of conflicting signs. A person may be chromosomally male but have genitals that appear completely female.
The large number of physical sexes account for these variations. They are simply names given to each observed combination. I don't know them and don't think they really matter. I don't believe that have conflicting signals about sex should imply the existence of a completely new sex.
Just to reiterate, intersexuality is in 1 in 1000 children. Klinefelter, or XXY, is the most common chromosomal abnormality.
BTW, the intersexed don't consider the term "hermaphrodite" to be PC.