The Cloud That Is Facebook

Facebook’s gigantic userbase, which is made up of 901 million users, requires an equally huge number of servers, datacenters and assorted storage and networking gear. Such large needs is expected of a company that has enough clout to go toe to toe with Google. However, unlike Google, Facebook has not relied on their own hardware designed to promote the company’s infrastructure advantage. Instead, Mark Zuckerberg and company decided to open a large part of its approach on hardware IP to the world, via the Open Compute Project.

In a recent meeting at their headquarters, Facebook Senior Vice President of Infrastructure Frank Frankovsky discussed the social media giant’s plans for a new kind of networking gear; basically, Facebook is rethinking the type of approach users have towards networking gear, and how said gear fits in within the data center. Frankovsky predicts that it’s going to be very hard to tell the difference between servers and networking devices over time, in the sense that the top of rack switch will eventually evolve to be more than just an Ethernet switch. According to Facebook’s predictions, top of rack switches will eventually be made with all the necessary server components built in, such as boot devices as well as NiCs.

It’s not just about networking, though. It’s also about a shift in the contents of the data center and taking servers out of the equation – with the roles of storage, servers, and networking being merged into a single datacenter in the future. Facebook’s view runs counter to large equipment vendors such as Cisco, which saw the same blurred line between servers and networking devices, but opted to combine both networking gear and servers into a single proprietary box.

Facebook believes that they have already solved a lot of the problems with regards to building the infrastructure, and that the next wave of innovation must instead come from the efficient operation of the datacenter at scale, and over time. Frankovsky cites the problem of the PC-refresh cycle that occurs every three years, or everytime the major chip vendors release the next generation of processors. This cycle requires people who want to keep up to rip out and replace substantial amount of their gear, which can be expensive especially for a large enterprise. Frankovsky thinks that this will be one day unnecessary for Facebook.

Facebook’s Open Compute Project is developing a server that isn’t actually a server box, but a simple CPU with its own DRAM, on a sled that is slotted into their own Open Rack design. This design makes do without any NICs, which means the CPUs can be replaced without touching the network itself, since it’s already integrated in the rack using a new kind of switch on top.

With Facebook’s Open Compute Project, the next generation of top rack switches will be more akin to IO appliances with built in Network Interface Cards than actual parts of the server. Dell and HP executives have already voiced their support for the Open Compute project, citing that they are supportive of the project’s ability to allow them to focus and engineer in a new area, but we’re wondering if the two manufacturers are sincere about their support, as it’s no secret that their margins will be negatively affected if Open Compute became successful.