James Brokenshire has been appointed Crime and Security Minister following the departure of Baroness Neville-Jones.
The move comes after Lady Neville-Jones stepped down on Monday "at her own request" amid reports of tensions within the Government.
Baroness Browning will take over Mr Brokenshire's previous role of Crime Prevention Minister.
Home Secretary Theresa May said: "Maintaining public safety and security remains my top priority and I will continue to take the lead for the department in ensuring the UK's national security and counter-terrorism strategy responds to the evolving terrorist threat.
"To support me in this important work, I am pleased to announce that James Brokenshire will be taking on the role of parliamentary under-secretary for crime and security."
Lady Browning will take on responsibility for the drugs strategy, licensing, community activism and football safety, and will also lead all Home Office business in the House of Lords.
In her resignation letter, Lady Neville-Jones, 71, assured Prime Minister David Cameron that she would "continue to give the Government, the Conservative Party and your leadership strong support".
She added she looked back on her time in the coalition "with some satisfaction".
In his reply, Mr Cameron thanked the former security minister for her "wise advice ... over many years" and admitted he had "asked a lot" when he invited her to "step into the world of active politics".
But the Prime Minister said she could be "very proud" of the part she has played in Government, including drawing up the plans for the National Security Council and National Security Strategy.