Ms Kramp-Karrenbauer took over as head of the Christian Democratic Union (CDU) party in December 2018.
The protégée of Mrs Merkel was widely seen as her favoured successor.
Her decision comes after fresh questions were raised about her ability to command authority in the party.
Last week, defying Ms Kramp-Karrenbauer, some of the CDU's local politicians in the eastern state of Thuringia voted with the far-right Alternative for Germany (AfD) party to install liberal candidate Thomas Kemmerich as regional premier.
Mr Kemmerich later said he would resign and seek new elections in the state, "to remove the stain of the AfD's support for the office of the premiership".
For mainstream parties to collaborate with the AfD is considered a taboo, and no state premier has ever taken office before with the aid of the AfD.
Merkel choice elected CDU leader
The AfD has grown in popularity in recent years but has been condemned for its extreme views on immigration, freedom of speech and the press.
Ms Kramp-Karrenbauer told a news conference on Monday that she would stay on as party leader until a leadership contest, expected in the summer, is held. She also plans to remain in the post of defence minister until the end of the legislative period.
Her decision has shaken up the race to succeed Mrs Merkel, who has led Germany for 15 years. But Ms Kramp-Karrenbauer said she believed her choice would not hurt the stability of the coalition between the CDU and the Social Democrats that runs Germany's federal government.
Turbulent months ahead for Germany
After a series of public blunders, Annegret Kramp-Karrenbauer finally fell over her inability to control her party - and its attitude toward the far right - in Germany's former east.
Like other mainstream parties, the CDU has still not found the formula for winning back voters lured by AfD's xenophobic nationalism - particularly ardent in the state of Thuringia, where the AfD more than doubled its vote in a regional election last year, leading to an inconclusive result.
The episode has dashed Chancellor Merkel's hopes of a smooth transition of power when she steps down next year. Attention is turning to the handful of men jostling to succeed her - at least three of whom would likely steer the CDU to the right-of-centre.
All this as Germany's Green Party edges closer and closer in the polls.
It is going to be a turbulent few months for German politics. Not least because what began as an inconclusive regional election has demonstrated the ability of Germany's far right to cause political chaos at the very highest level.
Ms Kramp-Karrenbauer - also known as AKK - made the surprise announcement that she no longer wanted to lead her CDU party following a meeting with members.
At that meeting, she said that parts of the CDU held an ambiguous position towards the far right and left.
In November, AKK urged critics to support her vision for Germany and threatened to stand down if the party failed to back her at the CDU's annual congress in Leipzig.
Who is AKK?
The 57-year-old joined the CDU in 1981 as a 19-year-old student and, after completing a masters in political science, worked her way up in state-level politics.
She became the first woman to serve as a state minister for internal affairs in 2000 and the first woman to serve as prime minister of Saarland, a position she held from 2011 to 2018.
After being nominated by Chancellor Merkel, AKK was elected as CDU general secretary with a record 98.9% of the vote.
In her first address as CDU leader in 2018, after ending Mrs Merkel's 18-year reign, AKK - seen as a moderate choice for the leadership - called for party unity.
During her farewell speech as party leader, Ms Merkel praised AKK for her 2017 electoral success in Saarland, in a clear hint that the chancellor saw her as her successor.