Mr Rafsanjani had chaired the Assembly of Experts, which selects the supreme leader and supervises his activities, for the past four years. But he was criticised by hardliners for being close to the opposition.
Ayatollah Mohammad Reza Mahdavi Kani was selected to replace him, the semi-official Mehr news agency reported.
Mr Rafsanjani said that for the sake of keeping the country united, he would not stand against Mr Kani.
“If he was ready and accepted responsibility, I would certainly not nominate myself, so that we could eliminate the differences and this sacred institution would not be damaged by my side,” he said.
He warned that divisiveness was becoming “very serious” and said the country should be “vigilant to keep people united”.
Mr Rafsanjani had chaired the Assembly – an elected body of clerics which has the theoretical power to dismiss the supreme leader – since 2007.
But he was stripped of his role as a leader of Friday prayers after criticising a crackdown on opposition protests after President Mahmoud Ahmadinejad’s disputed 2009 re-election.
Mr Rafsanjani remains the chairman of the Expediency Council, a committee which arbitrates disputes over legislation among state bodies.