A cable, running under the sea, powers the centre and transports the data to the shore and the wider internet.
The centre is a 40ft (12.2m) long white cylinder containing 864 servers which is enough to store five million movies and could be left in the sea for up to five years.
The cylinder is powered by tidal turbines and wave energy converters.
Orkney was chosen as a location because it is a major centre for renewable energy research.
Experts say the cost of cooling the computers will be cut by putting them under water.
The sea offers a free and accessible resource for cooling which is one of the biggest costs for land-based data centres.
If the project is successful, Microsoft hopes to sink groups of five of the cylinders and be able to deploy a data centre offshore in 90 days – doing this on land can take years.
Microsoft said: “More than half of the world’s population lives within about 120 miles of the coast.
“By putting data centres in bodies of water near coastal cities, data would have a short distance to travel to reach coastal communities.”
Cindy Rose, chief executive of the technology giant, said: “Microsoft is exploring the idea that data centres – essentially the backbone of the internet – can be based on the sea floor.
“Phase two of this research project has just begun in the Orkney Islands, where a more eco-friendly data centre was lowered into the water.
“The shipping container-sized prototype, which will be left in the sea for a set period of time before being recovered, can hold data and process information for up to five years without maintenance.
“Despite being as powerful as several thousand high-end consumer PCs, the data centre uses minimal energy as it’s naturally cooled.”
The downside is that if the computers on board break, they cannot be repaired.
More from Microsoft
Brace yourself for another Windows 10 update
Brace yourself: It’s time for another Windows 10 update
The data centre is also very small compared to the giant warehouses used to store the world’s information.
The cylinder was built in France by shipbuilding company Naval and driven to the Orkney Islands, an archipelago of around 70 islands.