Conference Publication Details
Mandatory Fields
Asad Z.;Chaudhry M.;Malone D.
9th Annual IEEE International Systems Conference, SysCon 2015 - Proceedings
CodHoop: A system for optimizing big data processing
Optional Fields
© 2015 IEEE. The rise of the cloud and distributed data-intensive ('Big Data') applications puts pressure on data center networks due to the movement of massive volumes of data. This paper proposes CodHoop a system employing network coding techniques, specifically index coding, as a means of dynamically-controlled reduction in volume of communication. Using Hadoop as a representative of this class of applications, a motivating use-case is presented. The proof-of-concept implementation results exhibit an average advantage of 31% compared to vanilla Hadoop implementation which depending on use-case translates to 31% less energy utilization of the equipment, 31% more jobs that run simultaneously, or to a 31% decrease in job completion time.
Grant Details