Since 1960, only two incumbent presidents seeking reelection have declined to debate their challengers: Lyndon B. Johnson in 1964 and Richard Nixon in 1972.
LBJ was said to have feared the impact of a poor showing in a debate. Nixon ran against John F. Kennedy in 1960 as the incumbent vice president, and his opponent's well-reviewed televised performance (the first head-to-head debate on TV) was followed by his narrow defeat. Nixon won the White House without a debate in 1968, and refused to debate his challenger in 1972.
In 1976 Gerald Ford debated challenger Jimmy Carter, and lost the election. Carter opted out of the first debate in 1980, due to the inclusion of a third-party candidate, but then lost to Reagan. Every incumbent since has debated his opponent; all but George H.W. Bush in 1992 prevailed in their campaigns.