Green Hadoop: The Cloud is Going Solar

Researchers at the Polytechnic University of Catalonia and the Rutgers University have recently proposed building a programming framework that can effectively manage a datacenter’s workload in a solar powered system. Dubbed the “Green Hadoop”, the system will also be using the grid, but only as a backup source of power.

The Green Hadoop will mostly be useful when determining when solar energy will be made available for Hadoop batch processing jobs, particularly those that aren’t time sensitive, because the system can schedule them more effectively without affecting any crucial tasks. The Green Hadoop will also handle providing energy from the grid, in cases where non renewable energy must be used due to the urgency of a task, or if there is not enough green energy available to complete said task.

The ideal scenario is for the Green Hadoop to schedule all resource intensive jobs that are not time sensitive to run during peak solar times, while switching to low intensity levels of computing when the datacenter only has grid power available.

The beauty of the Green Hadoop is that it will allow datacenters to maintain uptime even if availability of sunlight is inconsistent, and the worst that could happen is that it will just delay the non-priority computing tasks. It basically functions under the same principles as workloads that are shifted amongst different datacenters spread all over the world in order to take advantage of cheaper and cleaner energy.

According to the researchers, widespread use of Green Hadoop-managed data centers can significantly decrease electricity cost and increase green energy usage. Their findings were published at the recent EuroSys2012 conference.

The Rutgers team is developing their own Green Hadoop managed data center using a small solar powered unit that they built and named “Parasol”. The Parasol includes a container, a solar photovoltaic system, and a battery for storing energy. The datacenter can be switched to different sources of energy (complete grid, off grid, and hybrid) using three switches.

Due to solar power’s inability to provide a 100 percent reliable power source, it remains a controversial way to power data centers. But the technology is rapidly being employed by big name players, with Apple recently building a 20 MW solar panel farm in Maiden, NC in order to power their data centers. Facebook and eBay have their own smaller solar farms as well.

Leave a Reply

Your email address will not be published. Required fields are marked *