Joseph McElroy was born on August 21, 1930, in Brooklyn, New York, the only child of Joseph Prince and Louise (née Lawrence) McElroy. McElroy's father was a scholarship student to Harvard University who majored in chemistry, but later worked as a stockbroker. He died when McElroy was 15 years old.
In 1961, McElroy married Joan Leftwich, of London, in London. She is the daughter of Yiddish-speaking Orthodox Jews. Her father, Joseph Leftwich, was a translator and anthologizer of Yiddish poetry. The McElroy's only child, a daughter Hanna, was born in 1967. McElroy assisted with the birth.
McElroy's writing is often grouped with that of William Gaddis and Thomas Pynchon, due to the encyclopedic quality of his novels, particularly the 1,192 pages of Women and Men (1987). His short fiction was first published in literary journals. Echoes of McElroy's work can be found in that of Don DeLillo and David Foster Wallace. McElroy's work often reflects a preoccupation with how science functions in American society;Exponential, a collection of essays published in Italy in 2003, collects science and technology journalism written primarily in the 1970s and 1980s for the New York Review of Books.