Cardinal Kasper pulled out of the Pope's four-day UK visit, with the Vatican citing illness.
The trip is expected to contain a number of protests and statements by groups opposed to it.
Cardinal Kasper had made his remarks during an interview with the German magazine Focus.
He reportedly told a the magazine the UK was marked by "a new and aggressive atheism".
On Wednesday, Vatican sources said Cardinal Kasper was suffering from gout and had been advised by his doctors not to travel to the UK.
They also said his "Third World" comment referred to the UK's multicultural society.
Ahead of the Pope's arrival in Edinburgh, Cardinal O'Brien told BBC Radio Scotland: "[The comment] was unfortunate and each and every person's aides sometimes do make awkward, difficult remarks.
"Sometimes we make awkward, difficult remarks ourselves.
"And simply, if we do that sort of thing we apologise for it, and I'm sure Cardinal Kasper will apologise for any intemperate remarks which he made some time ago."
The Catholic Church in England and Wales said Cardinal Kasper's comments were "the personal views of one individual".
Pope Benedict XVI is making the first state visit by a pontiff to the UK.
The invitation has been criticised by a number of groups, including gay and women's rights organisations.
Protest the Pope, an umbrella group bringing together a dozen humanist, atheist, secular and gay rights groups, has said it opposes the idea of the Pope being welcomed to the UK as a head of state, with the UK taxpayer paying for much of the visit.
Pope Benedict has also faced calls to address public concern over the sexual and physical abuse of children by priests.
And reformist Catholics are using the Pope's visit as an opportunity to call on Benedict XVI to open up the debate on priestly celibacy and the ordination of women.