HDFS Load Balancer for Video Cloud Storage Service
HUANG Xiao-cheng1,YIN Jian-wei(1,2)
1（School of Computer Science and Technology,Zhejiang University,Hangzhou 310058,China） 2（Modern Service Industry Electornic Service Engineering Research Center in Zhejiang Province,Hangzhou 310058,China）
Abstract：Online video service is an important part of internet service,and storage is the basis of it.As a cloud storage system aiming at storing common files,HDFS is applied by many video service websites,but its load balancer doesn′t take videos′ characteristics of bandwidth consuming when played online into consideration to make full use of bandwidth resource of cluster.In order to solve this problem,proposes a load balancing method called VOBM under video storage circumstance,which anaylzes the relationship of bandwidth consuming of videos when played online between their bitrate,size of block and popularity and built up a new load evaluating model,then on the basis of it this method takes factor of bandwidth consuming into consideration in generating plans and scheduling blocks.Implemented VOBM on the basis of original load balancer in HDFS.Experiments show that VOBM can effectively avoid clustering of high bandwidth consuming data blocks,and it outperforms original balancing method in 90% of cases and manages to decrease bandwidth peak of bottleneck datanode of cluster by 20% in the best case when high bandwidth consuming files act as hot spots in the experiments.
2017, Vol. 38 
