Being overweight or underweight could knock four years off life expectancy, a study in a Lancet journal suggests.
The report, one of the largest of its kind, involved nearly 2 million people who were registered with doctors in the UK.
Researchers found that, from the age of 40, people at the higher end of the healthy Body Mass Index (BMI) range had the lowest risk of dying from disease.
But people at the top and bottom ends of the BMI risked having shorter lives.
BMI is calculated by dividing an adult's weight by the square of their height.
A "healthy" BMI score ranges from 18.5 to 25.
Most doctors say it is the best method they have of working out whether someone is obese because it is accurate and simple to measure.
The study, published in the Lancet Diabetes and Endocrinology, showed that life expectancy for obese men and women was 4.2 and 3.5 years shorter respectively than people in the entire healthy BMI weight range.
The difference for underweight men and women was 4.3 (men) and 4.5 (women) years.
BMI was associated with all causes of death categories, except transport-related accidents, including cancer, cardiovascular diseases and respiratory diseases.
However, not everybody in the healthy category is at the lowest risk of disease, according to report author Dr Krishnan Bhaskaran.
He told the BBC: "For most causes of death we found that there was an 'optimal' BMI level, with risk of death increasing both below and above that level.
"At BMIs below 21, we observed more deaths from most causes, compared with the optimum BMI levels. However, this might partly reflect the fact that low body weight can be a marker of underlying ill-health.
"For most causes of death, the bigger the weight difference, the bigger the association we observed with mortality risk.
"So a weight difference of half a stone would make a relatively small (but real) difference; we could detect these small effects because this was a very large study."
Some experts have questioned whether BMI is an accurate way of analysing a person's health.
However, Dr Katarina Kos, senior lecturer in Diabetes and Obesity at the University of Exeter, believes it is.
"For the majority of people, BMI is a good measure," she told the BBC.
Dr Kos added that the report did not contain any surprises but added that overweight people who could lower their BMI may reap the health benefits.
"We know from the diabetes remission data how low-calorie diets and weight loss can improve diabetes, for example," she said.
"And we know weight loss can also help in improving risk so that would also then improve mortality rates."
The report suggested that a higher BMI in older people may not be as dangerous, because a bit of extra weight was "protective" for them.
But Dr Kos, who worked on a report on this topic in 60 to 69-year-olds last year, disagreed with the findings.
Her report, on what is known as the obesity risk paradox, did not "support acceptance" of the theory.