Task Allocation And Scheduling For Big Data Applications In Cloud Computing Environments

Abstract

newline x newlineABSTRACT newlineThe excess volumes of data produced by devices and Internet-based application newlineperiodically, constitute Big data which can be processed and analyzed to newlinedevelop useful applications for specific domains such as social media, newlineE-commerce transactions, etc. Big data describes the tools and technologies newlinerequired to capture, manage, store, distribute and analyze larger-sized datasets newlinehaving different structures with high speed. The cloud infrastructures afford a newlineproper environment for the execution of large-scale big data application. newlineHowever, big data computing in the cloud has its own set of challenges and newlineresearch issues. newlineScheduling is defined as a set of policies to manage the flow of work which will newlinebe executed by computing resources. Scheduling large number of tasks in multi newlinecloud environment is one of the most significant research challenges in the newlinecurrent era. Task scheduling algorithms need to provide high performance and newlineefficient system throughput. newlineFor selecting the best cloud resources for each tasks based on their newlinerequirements, task allocation and reallocation algorithm for Big data cloud newlineapplications is proposed. In the proposed the Reallocation agent dispatches newlinedeallocation and allocation requests to the supervisor based on the physical newlinemachine. The resource allocation agent monitors the resources and chooses VM newlineresiding at the cluster which demands resource reconfiguration. Then it newlinedispatches an allocation or de-allocation request to RA, running in the physical newlinesystem. newlineFor optimizing the overall task execution time, minimizing the response time newlineand cost of execution, Priority-Based Optimized Scheduling (PBOS) algorithm newlineis proposed. In this algorithm, for each incoming task request, the task size and newlineexpected completion time are estimated. Similarly, for each VM on a host, its newlineprocessing capability, current load are calculated. A priority queue for the task newlineand a priority queue for VMs are created. The scheduling algorithm newlinexi newlineintelligently maps each user tasks from the task priorit