There has been speculation - mostly informally through individuals posting on the internet - that too much sex for males leads to eyefloaters and fatigue. The link might be DHA, which is concentrated in the sperm, brain, and eyes. The body must replace lost sperm/DHA by manufacturing it or scavenging it from other parts of the body - like the brain or the eyes.
"DHA is a major fatty acid in sperm and brain phospholipids, particularly in the retina."
Another possibility is that the cllmax's intense rush of pleasure chemicals overtaxes the nervous system, like a drug user. Fatigue from sex might be the equivalent of withdrawl from drug use, with a period of adjustment as the body tries to repair itself.