A fast map reduce algorithm for exact-repair reconstruction of big-data in cloud storage




Qin, Xue

Journal Title

Journal ISSN

Volume Title



In the distributed cloud storage for big data systems, there is a need for exact repair, high bandwidth codes. Instead of simply replicating the entire data, exact repair only focus on the error ones. The challenge for exact repair in big-data storage is to simultaneously enable the very high bandwidth repair using Map-Reduce, Simple Regenerating Code schemes and to combine with maximally distance separable (MDS) exact repair for the rare, but exceptional outlier error patterns requiring optimum erasure code reconstruction. In this thesis, we apply an optimum fast bandwidth repair algorithm for a big-data source. We build a cloud system framework to place this big-data source. And through the specific allocation we are able to use exact repair reconstruction (simple regeneration code). We also propose an innovation to the Map-Reduce so that we can apply our reconstruction in parallel. With the tremendously fast copy speed in Hadoop system and up to 2/3 code rate for SRC, in both GF(2) and GF(q) field. This cloud system will show up a better performance.


This item is available only to currently enrolled UTSA students, faculty or staff. To download, navigate to Log In in the top right-hand corner of this screen, then select Log in with my UTSA ID.


big data, distributed storage, Error Correction Code, Map Reduce



Electrical and Computer Engineering