Sir Keir Starmer should stop trying to “justify” accepting free gifts as it is “making things worse”, former Labour MP Harriet Harman has said.

The baroness, now a peer on the red benches and the co-host of Sky News’ Electoral Dysfunction podcast, said the row was “out of line” for the prime minister and he should change his approach rather than “doubling down”.

Sir Keir has come under scrutiny over the past week for the more than £100,000-worth of gifts he has accepted, including tickets to football matches, concerts and luxury clothes.

While this has all been declared as per the rules, backbench MPs have expressed concern that the government’s rhetoric of tough decisions on the economy is jarring with the image of a prime minister accepting freebies.

Sir Keir and his allies have issued various defences of the situation in recent days, but Baroness Harman said “doubling down and trying to justify it is making things worse”.

The Labour veteran told the podcast: “You can either double down on it and try and justify it or you can just say it was probably a misstep, if I had my time again I wouldn’t do it and therefore I’m going to auction for charity or something.

“It’s not a hanging offence, but I think doubling down and trying to justify it is making things worse.

“He’s not a sort of money focused, greedy type person. He is a person of public service. So this is slightly out of line. And he’ll be wanting to deal with it.”

The row started over the weekend with controversy over Sir Keir’s wife’s clothes and has escalated since Sky News’ Westminster Accounts project revealed he has been gifted more freebies and hospitality than any other MP in the past five years – a total worth £107,145.

The Premier League is one of the biggest donors of hospitality, and Sir Keir – a renowned Arsenal fan – has received almost £40,000 in tickets overall since December 2019.

The prime minister has defended his right to accept gifts, saying he is following the rules and there is a “good framework” in place.

On his acceptance of Arsenal tickets, he said he can’t go into the stands because of security reasons “therefore, if I don’t accept a gift of hospitality, I can’t go to a game”.

Allies of the prime minister have also sought to downplay the issue, with Business Secretary Jonathan Reynolds telling Sky News on Thursday that Sir Keir has a “pressure” job and should be allowed to enjoy freebies.

Meanwhile, Foreign Secretary David Lammy said there is no budget for the prime minister’s clothing in the UK, compared to other countries, and leaders need to look their best on the world stage.

Baroness Harman said Sir Keir doesn’t need to do “sackcloth and ashes… he just needs to say ‘right, with hindsight, I’m going to do things different’… and then move on to the things that people really they care about”.

She added: “I think at the moment he’s just got to put everything in terms of the government service and just get rid of every distraction he possibly can. There’s no right or wrong. So it’s just a question of a choice that he feels that is justified and that people in the country would be happy with.”

Ruth Davidson, former leader of the Scottish Conservatives, went further in her criticism, saying: “This wasn’t like he got some guy to buy him a suit before he went to the G7 and his wife got a frock. It’s that he did it again and again and again.

“So he may not be the sort of person that wants to be seen as grasping, and he may not even think of himself as that, but it’s the layer upon layer of rock here that kind of buries him. It’s not just he did one thing.”