The amount of new public sector borrowing hit a record of £15.9bn for August, according to the Office for National Statistics (ONS).
The larger-than-expected figure came after higher inflation led to a rise in interest payments on index-linked government bonds.
However, the ONS said receipts from taxes were still rising.
The latest figure means borrowing in the first five months of the financial year has reached £58.1bn.
However, the forecast for borrowing for 2010-11 as a whole remains £149bn, down from last year's total of £155bn.
The ONS figures exclude the impact of financial interventions by the government, which reduce overall borrowing because of profit contributions from the part-nationalised banks.
The ONS said the rise in the retail prices index, which is used to set payments on index-linked bonds, meant interest payments almost trebled to £3.8bn last month, compared with £1.3bn in for August a year ago.
The BBC's chief economics correspondent, Hugh Pym, said this rise in interest payments meant the country's finances would be out of balance for some time
A spokesman for the Treasury said the figure underlined the need for the government's forthcoming spending cuts.
"Today's borrowing figures demonstrate just why the government needs to tackle the deficit," he said.
"If the government had not announced decisive action to bring borrowing down, debt interest would have been over £65bn by 2014-15, more than is spent on schools or defence."