The mosque is the burial place of the Prophet Muhammad and Medina the holiest city in Islam after Mecca.
Suicide blasts also struck two other Saudi cities on Monday.
The fact that an attack happened in Medina at such a place is likely to leave Muslims around the world aghast, BBC World Service's Middle East editor, Alan Johnston, says.
Suspicion is likely to fall on so-called Islamic State (IS), he adds.
Al-Arabiya gave a different account of the incident, saying the bomber had targeted the security officers by pretending he wanted to break his Ramadan fast with them.
Qari Ziyaad Patel, 36, from South Africa, who was in the mosque, told the Associated Press news agency people had at first thought it was the sound of the cannon fire that marks the breaking of fast.
The ground shook, he said, adding: "The vibrations were very strong. It sounded like a building imploded."
Earlier, at least one explosion rocked Qatif, an eastern city which is home to many minority Shia Muslims.
The blast appeared to target a Shia mosque. The attacker was killed but no other casualties were reported.
Why IS attacks during Ramadan? By Shiraz Maher, King's College, LondonRamadan is traditionally viewed as the most holy and spiritual month in the Islamic calendar, a time of penance and temperance.
Mosques are consequently fuller than usual, typically packed with worshippers seeking divine mercy and blessings.
Juxtaposed alongside that ascetic puritanism is the view of radicals who regard Ramadan as a month of conquest and plunder.
They believe it is an opportune moment to double down on their millenarian war against civilisation and therefore launch more attacks than normal.
Read his full analysis
A suspected suicide bomber also died after detonating a device near the US consulate in the city of Jeddah in the early hours of Monday. Two security officers were slightly injured as they tackled the man, but no-one else was hurt.
No-one has yet said they were behind any of the attacks