The President announced in a tweet that he had nominated CIA Director Mike Pompeoto take up the role, saying the 54-year-old “will do a fantastic job”.
He went on to reveal he had appointed Gina Haspel as Mr Pompeo’s successor, the first woman to take on the prestigious position.
Mr Trump told reporters that although he and Mr Tillerson “got along”, they had disagreements.
“Rex and I have been talking about this for a long time,” Mr Trump said.
Mike Pompeo, Director of the CIA, will become our new Secretary of State. He will do a fantastic job! Thank you to Rex Tillerson for his service! Gina Haspel will become the new Director of the CIA, and the first woman so chosen. Congratulations to all!
— Donald J. Trump (@realDonaldTrump) March 13, 2018
“We got actually along quite well but we disagreed on things,” he added, saying he thinks the US’ nuclear deal with Iran is “terrible” whereas Mr Tillerson “felt a little bit differently”.
Mr Trump said that he has “total confidence” in Mr Pompeo and that the two share “a similar thought process”.
“I respect his intellect, his process that we’ve all gone through together,” he said. “We have a really good relationship, chemistry, whatever is is, why do people get along.”
Mr Tillerson was sworn in as Secretary of State in February 2017.
In October, he was forced to deny he was considering quitting over tensions between him and the President.
The 65-year-old did not speak to Mr Trump before he was was sacked and has not been given a reason for his dismissal, a top aide said.
“The secretary did not speak to the President this morning and is unaware of the reason, but he is grateful for the opportunity to serve, and still believes strongly that public service is a noble calling and not to be regretted,” undersecretary of state Steve Goldstein said.
In a statement announcing Mr Pompeo’s appointment, Mr Trump said: “I have gotten to know Mike very well over the past 14 months, and I am confident he is the right person for the job at this critical juncture.
“He will continue our programme of restoring America’s standing in the world, strengthening our alliances, confronting our adversaries, and seeking the denuclearisation of the Korean Peninsula.”
Mr Pompeo issued a statement saying he was “looking forward” to representing the President.
“I am deeply grateful to President Trump for permitting me to serve as Director of the Central Intelligence Agency and for this opportunity to serve as Secretary of State,” he said.
“His leadership has made America safer and I look forward to representing him and the American people to the rest of the world to further America’s prosperity.”
While some White House insiders claim Mr Pompeo tends to tell Mr Trump what he wants to hear, others say they have been impressed by his intellect, willingness to listen and advocacy of more robust covert operations.
More from US
Mike Pompeo: What you need to know about the new US Secretary of State
Charles Manson grandson wins battle over cult leader’s body
Trump blocks $117bn Qualcomm takeover citing national security
Trump’s protectionist stance has its dangers for US businesses
OECD warns of trade war threat to global economic rebound
Firefighter, journalist among helicopter victims
In another departure from the White House, John McEntee has been sacked as a personal aide to the President.
Mr McEntee, one of a shrinking group of campaign aides still working in the West Wing, has rejoined Mr Trump’s re-election campaign as an adviser.