Mr Javid, 48, was promoted from the position of Communities Secretary after scandals over the Windrush generation and deportation targets.
Addressing the Windrush scandal is his “most urgent task”, he said.
He added that he wants people caught up in the controversy to be treated with the “decency and fairness they deserve”.
Ms Rudd resigned late last night after admitting “inadvertently” misleading a committee of MPs about Home Office targets for forced removals.
Her Cabinet departure is the fifth since last year’s snap General Election.
Mr Javid has spoken out passionately on the Windrush issue, telling The Sunday Telegraph that he or his family could have suffered because of the scandal. “It could have been me, my mum or my dad,” he said.
Mr Javid was born in Rochdale, Lancashire, to parents of Pakistani descent. His father worked as a bus driver. Later, after moving to Bristol, he attended a state comprehensive school and Exeter University. Before going into politics he worked in banking.
The MP for Bromsgrove served previously as a treasury minister and equalities minister.
He also headed up the culture and business departments, and is now firmly entrenched in clearing up the fallout from the Grenfell Tower fire.
He backed remaining in the EU.
Former Northern Ireland Secretary and immigration minister James Brokenshire is the new Communities Secretary.
Mr Brokenshire stepped down from the cabinet in January after being diagnosed with lung cancer.
In an article for Sky News, he wrote about being told he had a lesion on his right lung.
Penny Mordaunt remains International Development Secretary, but also becomes women and equalities minister – a role Ms Rudd held in addition to her former leadership of the Home Office.
In addition to dealing with the fallout from the Windrush scandal, Mr Javid will have to address concerns about increasing gun and knife crime in England and Wales.
More from Politics
LIVE: Reshuffle as Sajid Javid replaces Amber Rudd
Home Secretary Amber Rudd quits over misleading Parliament
The fall of Amber Rudd – and what it means for PM
Rudd-less May faces perilous week
Windrush must remain the focus despite Amber Rudd’s resignation, MPs warn
Frustration and relief: Inside an immigration centre with the Windrush generation
He will also need to finalise the Government’s refreshed counter-terrorism strategy – expected in the coming weeks.
The official threat level remains at severe, meaning an attack is “highly likely”.