Please use this identifier to cite or link to this item: http://buratest.brunel.ac.uk/handle/2438/12574
Full metadata record
DC FieldValueLanguage
dc.contributor.authorKhan, M-
dc.contributor.authorHuang, Z-
dc.contributor.authorLi, M-
dc.contributor.authorTaylor, GA-
dc.contributor.authorKhan, M-
dc.date.accessioned2016-04-25T10:40:42Z-
dc.date.available2016-02-24-
dc.date.available2016-04-25T10:40:42Z-
dc.date.issued2016-
dc.identifier.citationConcurrency Computation: Practice and Experience, 6: pp.42561-42571,(2016)en_US
dc.identifier.issn1532-0626-
dc.identifier.issn1532-0634-
dc.identifier.urihttp://onlinelibrary.wiley.com/doi/10.1002/cpe.3786/abstract;jsessionid=5B3B229799FEF32621FC92B4064A7D64.f03t01-
dc.identifier.urihttp://bura.brunel.ac.uk/handle/2438/12574-
dc.description.abstractHadoop MapReduce has become a major computing technology in support of big data analytics. The Hadoop framework has over 190 configuration parameters, and some of them can have a significant effect on the performance of a Hadoop job. Manually tuning the optimum or near optimum values of these parameters is a challenging task and also a time consuming process. This paper optimizes the performance of Hadoop by automatically tuning its configuration parameter settings. The proposed work first employs gene expression programming technique to build an objective function based on historical job running records, which represents a correlation among the Hadoop configuration parameters. It then employs particle swarm optimization technique, which makes use of the objective function to search for optimal or near optimal parameter settings. Experimental results show that the proposed work enhances the performance of Hadoop significantly compared with the default settings. Moreover, it outperforms both rule-of-thumb settings and the Starfish model in Hadoop performance optimization.en_US
dc.description.sponsorshipThis research is supported by the UK EPSRC under grant EP/K006487/1 and also the National Basic Research Program (973) of China under grant 2014CB340404.en_US
dc.language.isoenen_US
dc.publisherJohn Wiley & Sonsen_US
dc.subjectHadoopen_US
dc.subjectMapreduceen_US
dc.subjectBig data analyticsen_US
dc.subjectGene expression programmingen_US
dc.subjectParticle swarm optimizationen_US
dc.titleOptimizing hadoop parameter settings with gene expression programming guided PSOen_US
dc.typeArticleen_US
dc.identifier.doihttp://dx.doi.org/10.1002/cpe.3786-
dc.relation.isPartOfConcurrency Computation-
pubs.begin-page42561-42571-
pubs.publication-statusAccepted-
pubs.volume6-
Appears in Collections:Dept of Electronic and Computer Engineering Research Papers

Files in This Item:
File Description SizeFormat 
Fulltext.pdf1.41 MBAdobe PDFView/Open


Items in BURA are protected by copyright, with all rights reserved, unless otherwise indicated.