Synchronous evolution of an odor biosynthesis pathway and behavioral response

Background: Rodents use olfactory cues for species-specific behaviors. For example, mice emit odors to attract mates of the same species, but not competitors of closely related species. This implies rapid evolution of olfactory signaling, although odors and chemosensory receptors involved are unknown. Results: Here, we identify a mouse chemosignal, trimethylamine, and its olfactory receptor, trace amine-associated receptor 5 (TAAR5), to be involved in species-specific social communication. Abundant (>1,000-fold increased) and sex-dependent trimethylamine production arose de novo along the Mus lineage after divergence from Mus caroli. The two-step trimethylamine biosynthesis pathway involves synergy between commensal microflora and a sex-dependent liver enzyme, flavin-containing monooxygenase 3 (FMO3), which oxidizes trimethylamine. One key evolutionary alteration in this pathway is the recent acquisition in Mus of male-specific Fmo3 gene repression. Coincident with its evolving biosynthesis, trimethylamine evokes species-specific behaviors, attracting mice, but repelling rats. Attraction to trimethylamine is abolished in TAAR5 knockout mice, and furthermore, attraction to mouse scent is impaired by enzymatic depletion of trimethylamine or TAAR5 knockout. Conclusions: TAAR5 is an evolutionarily conserved olfactory receptor required for a species-specific behavior. Synchronized changes in odor biosynthesis pathways and odor-evoked behaviors could ensure species-appropriate social interactions.