Checkpoint and Replication Oriented Fault Tolerant Mechanism for MapReduce Framework
MapReduce is an emerging programming paradigm and an associated implementation for processing and generating big data which has been widely applied in data-intensive systems. In cloud environment, node and task failure is no longer accidental but a common feature of large-scale systems. In MapReduce...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | EJournal Article |
| Published: |
Institute of Advanced Engineering and Science,
2014-02-01.
|
| Subjects: | |
| Online Access: | Get fulltext |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| LEADER | 02218 am a22002893u 4500 | ||
|---|---|---|---|
| 001 | ijeecs3097_1163 | ||
| 042 | |a dc | ||
| 100 | 1 | 0 | |a Liu, Yang |e author |
| 100 | 1 | 0 | |e contributor |
| 700 | 1 | 0 | |a Wei, Wei |e author |
| 700 | 1 | 0 | |a Zhang, Yuhong |e author |
| 245 | 0 | 0 | |a Checkpoint and Replication Oriented Fault Tolerant Mechanism for MapReduce Framework |
| 260 | |b Institute of Advanced Engineering and Science, |c 2014-02-01. | ||
| 520 | |a MapReduce is an emerging programming paradigm and an associated implementation for processing and generating big data which has been widely applied in data-intensive systems. In cloud environment, node and task failure is no longer accidental but a common feature of large-scale systems. In MapReduce framework, although the rescheduling based fault-tolerant method is simple to implement, it failed to fully consider the location of distributed data, the computation and storage overhead. Thus, a single node failure will increase the completion time dramatically. In this paper, a Checkpoint and Replication Oriented Fault Tolerant scheduling algorithm (CROFT) is proposed, which takes both task and node failure into consideration. Preliminary experiments show that with less storage and network overhead.CROFT will significantly reduce the completion time at failure time, and the overall performance of MapReduce can be improved at least over 30% than original mechanism in Hadoop. DOI : http://dx.doi.org/10.11591/telkomnika.v12i2.4324 | ||
| 540 | |a Copyright (c) 2014 Institute of Advanced Engineering and Science | ||
| 540 | |a http://creativecommons.org/licenses/by-nc-nd/4.0 | ||
| 546 | |a eng | ||
| 690 | |a MapReduce; Fault Tolerant; Cloud Computing; Checkpoint | ||
| 655 | 7 | |a info:eu-repo/semantics/article |2 local | |
| 655 | 7 | |a info:eu-repo/semantics/publishedVersion |2 local | |
| 655 | 7 | |2 local | |
| 786 | 0 | |n Indonesian Journal of Electrical Engineering and Computer Science; Vol 12, No 2: February 2014; 1029-1036 | |
| 786 | 0 | |n 2502-4760 | |
| 786 | 0 | |n 2502-4752 | |
| 786 | 0 | |n 10.11591/ijeecs.v12.i2 | |
| 787 | 0 | |n https://ijeecs.iaescore.com/index.php/IJEECS/article/view/3097/1163 | |
| 856 | 4 | 1 | |u https://ijeecs.iaescore.com/index.php/IJEECS/article/view/3097/1163 |z Get fulltext |