Ferenc Molnár, Thomas Caraco, Gyorgy Korniss

We model sex-structured population dynamics to analyze pairwise competition between groups differing both genetically and culturally. A sex-ratio allele is expressed in the heterogametic sex only, so that assumptions of Fisher's analysis do not apply. Sex-ratio evolution drives cultural evolution of a group-associated trait governing mortality in the homogametic sex. The two-sex dynamics under resource limitation induces a strong Allee effect that depends on both sex ratio and cultural trait values. We describe the resulting threshold, separating extinction from positive growth, as a function of female and male densities. When initial conditions avoid extinction due to the Allee effect, different sex ratios cannot coexist; in our model, greater female allocation always invades and excludes a lesser allocation. But the culturally transmitted trait interacts with the sex ratio to determine the ecological consequences of successful invasion. The invading female allocation may permit population persistence at self-regulated equilibrium. For this case, the resident culture may be excluded, or may coexist with the invader culture. That is, a single sex-ratio allele in females and a cultural dimorphism in male mortality can persist; a low-mortality resident trait is maintained by father-to-son cultural transmission. Otherwise, the successfully invading female allocation excludes the resident allele and culture and then drives the population to extinction via a shortage of males. Finally, we show that the results obtained under homogeneous mixing hold, with caveats, in a spatially explicit model with local mating and diffusive dispersal in both sexes.