Officers were pictured searching the vehicle’s boot and removing a white sheet.
Large sections of Whitehall were closed to the public and vehicles following the incident, but they were later reopened.
Eyewitness Krish Kandiah, who was delivering a petition at Downing Street when the crash happened, told Sky News there was an “almighty smash at the front gate”.
“Immediately we saw police scramble and armed officers responded swiftly. They shouted at the man to get out. He was wrestled out of the car and put down on the ground and handcuffed,” he said.
Please use Chrome browser for a more accessible video player
1:37
‘Almighty smash at front gate’
Another witness Simon Parry, 44, said: “I heard a bang and looked up and saw loads of police with Tasers shouting at the man.
“A lot of police vehicles came very quickly and were very quick to evacuate the area.”
Mr Parry said the driver had his “face to the floor” as he was being arrested.
He said the man appeared to slow down before he reached the gates of Downing Street, having driven out from the other side of the road.
“There were sniffer dogs and a bomb squad,” he added.
Asked to describe the mood in the immediate aftermath, Mr Parry said: “We saw people that were in a panic running away and we saw people who were excited.”
The Metropolitan Police said in a statement: “At approximately 4.20pm on Thursday 25 May a car collided with the gates of Downing Street on Whitehall.
“Armed officers arrested a man at the scene on suspicion of criminal damage and dangerous driving. He has been taken into custody.”