Nigel Havers doesn't normally do lunch, but for us he is willing to make an exception. "I haven't had lunch for years," he says, as he slides suavely onto a red leather banquette in La Brasserie, South Kensington. It is his local restaurant and he has been coming here ever since it opened – only for dinner, though, obviously.

How long has he not eaten lunch for exactly? "Oh, years," Havers replies nonchalantly. "My waist is 30 inches and I want to keep it that way. In the job I do, I need to keep in shape."

Does he have breakfast?

"No," he says blithely, but after further questioning he reveals he might occasionally have a grapefruit or porridge if he's really pushing the boat out. He must be starving by dinner time, I say. Does he have a massive, compensatory meal at the end of the day? He shakes his head and his fine mop of hair shivers becomingly.

"My wife will cook something healthy like pasta with crab and chilli."

When he was flown to the Australian jungle for a short-lived stint on I'm a Celebrity in 2010, he lost over a stone in two weeks. "I didn't eat anything," he says proudly. "I mean anything."

And when he's in the throes of doing a play, Havers doesn't eat at all before the final curtain.

"It's why I only do short plays," he jokes. He is about to appear as Algernon Moncrieff in a new production of The Importance of Being Earnest at the Harold Pinter theatre in the West End. It is a role he originally played as a 26-year-old. Now 62, he seems unfazed by the challenge.

"Your body starts to deteriorate but your mind doesn't," he says. "I still regard myself as the same man."

He orders a Diet Coke and, noticing that the table is jiggling slightly, asks the waiter to bring us something to put under the wobbling leg. This involves a lot of "Would you mind terribly-s?" and "Thank you awfully-s" and the deployment of a deeply sincere smile that makes his eyes crinkle.

It is almost impossible to meet Havers and not be charmed. His charm weaves its way through the conversation with all the insistence of a purring cat pressing against your legs. You know you are being played and yet it is such a pleasing sensation you somehow don't mind. Throughout his 30-year career he has made a name for himself playing smoothies, gentlemen and cads in films such as A Passage to India and Empire of the Sun and in sitcoms, dramas and soaps (he was the charismatic con-artist Lewis Archer in Coronation Street).

In real life, Havers is deeply solicitous, full of compliments and one of those rare interviewees who asks me as many questions as I ask him.

Does he think of himself as charming?

"I'm not charming to people I don't like," he continues, citing Labour MP Diane Abbott. Apparently Havers once appeared on BBC1's This Week alongside Abbott and Michael Portillo and she refused to shake his hand after taking exception to some comments he had made about women in the acting profession being treated equally to men. She is, he now says, "the rudest woman in England".

He orders an avocado with prawns. I had been lusting after a giant lobster with lashings of chips and mayonnaise but, shamed by his discipline, I opt for a salad involving halloumi, asparagus and other green things.

Havers nods approvingly. It wasn't ever thus. His earliest food memories, he says, are of an epicurean holiday to Italy as a child where he gorged himself on gorgonzola and sun-ripened tomatoes. Later, he and his elder brother, Philip, were sent to boarding school. Havers was six.

"It was hard," he admits. "I was homesick." He hated the meals which seemed mainly to consist of pilchards.

At home, his parents enjoyed the finer things in life – one of the first things Havers's father, Michael, taught his younger son to do was open a bottle of wine. Michael Havers was one of the most notable barristers of his generation – famous for being the lead prosecutor of the Yorkshire Ripper and for defending Mick Jagger and Keith Richards on drug charges in 1967. He later went on to become a Conservative Lord Chancellor. His father was, says Havers, "very impressed with Mick and Keith on an intellectual level".

Havers, an impressionable 15-year-old at the time of the trial, remembers 60s London as being a fragrant haze "of Brut aftershave mixed with cannabis. I didn't really smoke [cannabis] until I was 20 and I didn't really like it. I used to pretend to like it because other people who were very cool did. I wasn't really inhaling. I just wanted to have a glass of wine. Mick Jagger, I think, secretly didn't like drugs. He liked being in control."

His brother went to Eton but somehow Havers persuaded his parents to send him to an independent arts school in London and let him live in the family pied-à-terre in London.

"It was just so exciting!" he says as the avocado arrives, surrounded by a few lonely-looking prawns. "Sex was the most amazing thing! And the pill freed women up a bit."

Gosh, I say. Your brother must have been rather annoyed, trapped in his boarding school while you were cutting a swath through London.

"He was furious! But he'd come up at the weekends and I'd organise …"

Orgies?

"Not quite orgies but … girlfriends."

After school, Havers went into the theatre while his brother followed his father into the law. Havers has a long legal pedigree – his grandfather was the high court judge Cecil Havers and his aunt is Elizabeth Butler-Sloss, who recently had to step down as chair of the inquiry into historical child abuse. As a result, Havers has trouble thinking of what he does as "a proper job".

"What are proper jobs?" he muses. "Doctors, lawyers … what I do, I suppose, is quite frivolous."

As the acting career took off, the romping continued. He got married at 22 to Carolyn Cox (and had a daughter, Kate, now 36), but had flings here and there – including once with the director David Lean's wife Sandy which was interrupted by Lean himself who didn't seem to mind that much. While married, Havers had a three-year affair with model and actress Polly Williams and struggled to choose between his wife and his lover, eventually ending up in hospital with stress before plumping for Williams who became his second wife in 1989. Tragically, Williams died of ovarian cancer in 2004. Three years later, Havers married her best friend, Georgiana Bronfman.

All he will say now is that: "Looking back, I don't know how I did it. I couldn't do it now. I think if you are married, you should stay married."

There is a brief hint of sadness in his eyes before he recovers his default good humour. His daughter, Kate, is expecting a baby boy shortly and Havers is "very excited" at the prospect of becoming a grandfather.

Will she name him Nigel? He grimaces.

"Oh God, I hope not. It's unbelievable that my parents called me that, isn't it? Nigel Alan. And my brother is Philip Nigel. So they used it twice."

The avocado has been finished. The Diet Coke has been knocked back. He has to get going, he says, as his wife is flying back from a trip to Namibia and has already given him a shopping list with strict instructions as to what to buy for the dinner she is planning to make him tonight. It will be healthy, he tells me. After all, he has that 30-inch waist to consider.

The Importance of Being Earnest is at the Harold Pinter Theatre until 20 September, then on tour.