27 February 2012 Last updated at 03:20 GMT
Correspondents say the result is a major boost for a leader who had been facing serious internal divisions.
Speaking after the vote, Mr Rudd threw his support behind the prime minister.
A cabinet reshuffle is now expected to fill his position as foreign minister, from which he resigned last week.
Speaking to reporters after the result was announced, Ms Gillard thanked colleagues for their ''overwhelming endorsement'' and stressed that she was confident of leading the Labor Party to a win at the next election in 2013.
''This issue - the leadership question - is now determined,'' she said.
She praised Mr Rudd's achievements, saying that he would be honoured by the nation and party, but firmly put the events of his removal as prime minister behind her.
'Without rancour' Speaking at a separate press conference before Ms Gillard, Mr Rudd congratulated her on ''her strong win today''.
''The caucus has spoken. I accept the verdict without qualification, without rancour,'' he said. "I dedicate myself to working fully for her re-election as prime minister of Australia."
Thanking his family, supporters, staff and colleagues, he said he would ''bear no grudges'' against those who criticised him.
He did not take any questions from the media, but said he would continue serving his community as an MP. He is expected to move to the backbenches.
All eyes are now on how Ms Gillard will reshuffle her cabinet after a handful of ministers came out in support of Mr Rudd. Defence Minister Stephen Smith is seen as the front-runner to fill the position of foreign minister.
She said at her press conference that she would make an announcement in the days to come.
Meanwhile, opposition leader Tony Abbott continued to cast doubts on her leadership, saying that a third of her colleagues and a quarter of parliament have no confidence in her.
Her win today, he said at a media conference after she and Mr Rudd had spoken, was ''merely a stay of execution''.
'Right thing' Mr Rudd came to power with an electoral landslide in 2007 after more than a decade of conservative rule and enjoyed some of the highest popularity ratings for a prime minister in years.
But he went on to lose the confidence of colleagues over his environmental and taxation policies, and was ousted by Ms Gillard who at the time was his deputy. She became Australia's first female prime minister in June 2010.
He returned to the cabinet as foreign minister, but in recent months as Ms Gillard's support fell in opinion polls there was widespread speculation that a leadership bid was imminent.
In his statement, Mr Rudd said that he knew challenging Ms Gillard for the leadership would be tough, but had been the ''right thing'' to do.
''But we Queenslanders are made of other stuff,'' he said.
AnalysisThis was never really an ideological battle of ideas between the two candidates. Both put social welfare and economic fairness first.
Instead, it was much more about the simple matter of power - who has it and how do they wield it.
Whilst Kevin Rudd is popular with the public, his colleagues have much less time for him. They remember him as a somewhat arrogant prime minister who ruled over a chaotic government.
Although Julia Gillard lacks his charm, they view her as a much more competent, collaborative and effective leader.