Ben Freeman, now 9, said his father stopped their car on the bridge, took his sister Darcey from the front seat and threw her over the rail.
In a taped interview played to the Supreme Court jury Ben was asked if Darcey said anything.
"She didn't even scream on her fall,'' he said.
"I didn't hear her scream on the way when she......nothing, nothing, nothing.''
Arthur Freeman, 37, of Hawthorn, has pleaded not guilty to murdering four-year-old Darcey Freeman on January 29, 2009, by throwing her from the West Gate.
She fell 58m to the water and died from drowning. It was to be her first day of school.
Opening his case yesterday defence barrister David Brustman SC told the jury it was not disputed his client killed his daughter but they would have to decide if he was "mad or bad''.
Ben Freeman said that his father carried Darcey like he was carrying a baby and threw his he from the bridge and drove off.
"I said go back and get her. And dad keeps driving along,'' he said.
"Then I said Darcey can't swim ... and then dad would just keep on driving, didn't go back to get her.
"I kept on saying it over and over again and he never did it.''
Ben said they drove to the "weird funny place'' like an airport, which the jury has heard was the Commonwealth Law Courts building in William St.
He said he was getting bored and was asked what his father was doing.
"He was sulking. He was crying in the corner,'' he said.
Witness Barry Nelson said he was driving to work over the Westgate with his wife when he saw a white Toyota Prado 4WD pull over in the emergency lane.
Mr Nelson said he saw a man with a child in his arms and watched him tip the child over the rail.
"Her hair and limbs were flying,'' he said.
Mr Nelson stopped his car and approached the man but he saw no signs of aggression or emotion.
"He was totally neutral. He may have been posting a letter. He may have been walking back to the post box to his vehicle,'' he said.
The trial is continuing before Justice Paul Coghlan.