-
摘要: 针对基于Docker容器的分布式云计算下出现负载不均衡问题, 有必要将较高负载服务器中的Docker容器进程迁移到其他相对空闲的服务器上. 而传统的容器迁移算法忽视了容器本身的特征, 从而导致在迁移过程中传输效率低下. 基于此, 利用第三方管理平台和数据预存储阈值机制, 提出一种Docker容器动态迁移预存储算法PF-Docker. 首先将Docker容器内部进程运行相关文件和流动数据预存至云端存储器, 然后通过预存储阈值机制减少流动数据的无效传输, 最后在停机传输阶段将流动数据和冗余数据传输给目的服务器. 实验表明, 该方法在Docker容器迁移中能有效地降低迁移时间, 减少数据传输量, 提高容器的容错率.Abstract: In order to solve the problem of load imbalance in distributed cloud computing based on Docker container, it is necessary to migrate Docker container processes in high load servers to other relatively idle servers. However, the traditional container migration algorithm ignores the characteristics of the container itself, which leads to low transfer efficiency in the migration process. Based on this, this paper proposes a Docker container dynamic migration pre-storage algorithm PF-Docker by using third-party management platform and data pre-storage threshold mechanism. Firstly, relevant files and flowing data of Docker internal process operation are pre-stored to cloud storage, and then invalid transmission of flowing data is reduced through pre-storage threshold mechanism. Finally, flowing data and redundant number are reported to the destination server in the shutdown transmission stage. Experiments show that this method can effectively reduce the migration time, reduce the amount of data transmission and improve the fault tolerance rate of Docker container migration.
-
Key words:
- Docker container /
- pre-storage /
- dynamic migration /
- cloud computing /
- fault tolerance
-
图 3 基于预存储的容器动态迁移框架: PF-Docker
Fig. 3 Prestorage-based dynamic migration framework for containers: PF-Docker
图 7 读写空闲场景下两种Docker动态迁移方法性能对比
Fig. 7 Performance comparison of two Docker dynamic migration methods in read-write idle scenarios
图 10 Stream测试下PF-Docker与LM-KVM对比
Fig. 10 The comparison of PF-Docker and LM-KVM under Stream test
图 11 内核编译负载下Docker与KVM对比
Fig. 11 The comparison of Docker and KVM under the kernel compilation load
图 12 多线程内核编译在不同预存条件下对比
Fig. 12 The comparison of multithreaded kernel compilation under different pre-stored conditions
表 1 实验负载类型表
Table 1 Experimental load type table
负载类型 具体配置 Loop测试 基础镜像为Ubuntu14.04.1, 在Ubuntu操作系统下执行永真循环打印程序(低负载写空闲) Stream基准测试 基础镜像为Ubuntu14.04.1, 通过设置不同的内存读写速率进行测试(高负载写密集) 多线程内核编译测试 在Ubuntu14.04.1上部署内核编译依赖环境, 生成镜像zx/Ubuntu1, 在不同线程下测试(并行程序写密集) 
下载: 导出CSV
-
[1] 郭刚, 于炯, 鲁亮, 英昌甜, 尹路通. 内存云分级存储架构下的数据迁移模型. 计算机应用, 2015, 35(12): 3392-3397Guo Gang, Yu Jiong, Lv Liang, Ying Chang-Tian, Yin Lu-Tong. Data migration model based on RAMCloud hierarchical storage architecture. Journal of Computer Applications, 2015, 35(12): 3392-3397 [2] Filani D. Dynamic data center power management trends, issues, and solutions. Intel Technology Journal, 2008, 12(1): 59-67 doi: 10.1535/itj.1201.06 [3] Shieh A, Kandula S, Greenberg A G, Kim C, Saha B. Sharing the data center network. In: Proceedings of the 8th USENIX Conference on Networked Systems Design and Implementation. Boston, USA: USENIX Association, 2011. 309−322 [4] 夏元清. 云控制系统及其面临的挑战. 自动化学报, 2016, 42(1): 1-12Xia Yuan-Qing. Cloud control systems and their challenges. Acta Automatica Sinica, 2016, 42(1): 1-12 [5] 夏元清, 闫策, 王笑京, 宋向辉. 智能交通信息物理融合云控制系统. 自动化学报, 2019, 45(1): 132-142Xia Yuan-Qing, Yan Ce, Wang Xiao-Jing, Song Xiang-Hui. Intelligent transportation cyber-physical cloud control systems. Acta Automatica Sinica, 2016, 45(1): 132-142 [6] Fenn M, Murphy M A, Martin J, Goasguen S. An evaluation of KVM for use in cloud computing. In: Proceedings of the 2nd International Conference on the Virtual Computing Initiative. 2008. [7] Feng X J, Tang J X, Luo X, Jin Y H. A performance study of live VM migration technologies: VMotion vs XenMotion. In: Proceedings of the Asia Communications and Photonics Conference and Exhibition. Shanghai, China: IEEE, 2011. 1−6 [8] Liu Z B, Qu W Y, Yan T, Li K Q. Hierarchical copy algorithm for Xen live migration. In: Proceedings of the International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery. IEEE, 2010. 361−364 [9] Travostino F, Daspit P, Gommans L, Jog C, De Laat C, Mambretti J, et al. Seamless live migration of virtual machines over the MAN/WAN. Future Generation Computer Systems, 2006, 22(8): 901-907 doi: 10.1016/j.future.2006.03.007 [10] Bradford R, Kotsovinos E, Feldmann A, Schiöberg H. Live wide-area migration of virtual machines including local persistent state. In: Proceedings of the 3rd International Conference on Virtual Execution Environments. San Diego, USA: ACM, 2007. 169−179 [11] Docker. Make better, secure software from the start [Online], available: http://www.docker.com/, March 12, 2018 [12] Rightscale-2018-state of the cloud report [Online], available: https://www.rightscale.com/li/state-of-the-cloud, January 25, 2018 [13] 胡丹琪. 基于云计算的Docker容器动态迁移框架 [硕士学位论文], 中国科学院大学, 中国, 2017.Hu Dan-Qi. Docker Container Dynamic Migration Framework Based on Cloud Computing, Academy of Sciences [Master thesis], University of Science and Technology of China, China, 2017. [14] Chandra R, Zeldovich N, Sapuntzakis C, Lam M S. The collective: A cache-based system management architecture. In: Proceedings of the 2nd Conference on Symposium on Networked Systems Design and Implementation-Volume 2. Boston, USA: USENIX Association, 2005. 259−272 [15] Kozuch M, Satyanarayanan M. Internet suspend/resume. In: Proceedings of the 4th IEEE Workshop on Mobile Computing Systems and Applications. Callicoon, USA: IEEE, 2002. 40−46 [16] Clark C, Fraser K, Hand S, Hansen J G, Jul E, Limpach C, et al. Live migration of virtual machines. In: Proceedings of the 2nd Conference on Symposium on Networked Systems Design and Implementation-Volume 2. Boston, USA: USENIX Association, 2005. 273−286 [17] 周佳祥, 郑纬民, 杨广文. 一种基于进程迁移的自适应双阈值动态负载平衡系统. 清华大学学报(自然科学版), 2000, 40(3): 121-125Zhou Jia-Xiang, Zheng Wei-Min, Yang Guang-Wen. Adaptive dual-threshold dynamic load balancing system based on migrating. Journal of Tsinghua University (Science and Technology), 2000, 40(3): 121-125 [18] 张彬彬, 罗英伟, 汪小林, 王振林, 孙逸峰, 陈昊罡, 等. 虚拟机全系统在线迁移. 电子学报, 2009, 37(4): 894-899Zhang Bin-Bin, Luo Ying-Wei, Wang Xiao-Lin, Wang Zhen-Lin, Sun Yi-Feng, Chen Hao-Gang, et al. Whole-system live migration mechanism for virtual machines. Acta Electronica Sinica, 2009, 37(4): 894-899 [19] 邵曦煜. 基于Ceph的非共享存储虚拟机动态迁移系统的优化 [硕士学位论文], 中国科学技术大学, 中国, 2018.Shao Xi-Yu. Optimization of Non-shared Storage Virtual Machine Live Migration System Based on Ceph [Master thesis], University of Science and Technology of China, China, 2018. [20] 赵佳. 虚拟机动态迁移的关键问题研究 [博士学位论文], 吉林大学, 中国, 2013.Zhao Jia. Research on Live Migration of Virtual Machine [Ph.D. dissertation], Jilin University, China, 2013. [21] 吕小虎, 李沁. 虚拟机磁盘迁移技术研究与实现. 计算机科学, 2009, 36(7): 256-261, 297Lv Xiao-Hu, Li Qin. Research and implementation on migration of virtual machine including Vm-disk. Computer Science, 2009, 36(7): 256-261, 297 [22] Osman S, Subhraveti D, Su G, Nieh J. The design and implementation of Zap: A system for migrating computing environments. ACM SIGOPS Operating Systems Review, 2002, 36(SI): 361-376 doi: 10.1145/844128.844162 [23] CRIU. Checkpoint/restore [Online], available: https://criu.org/Checkpoint/Restore, March 12, 2018 [24] CRIU. P. Haul [Online], available: https://criu.org/P.Haul, March 13, 2018 [25] 禹超. Linux Containers热迁移机制研究 [硕士学位论文], 电子科技大学, 中国, 2015.Yu Chao. Research on the Live Migration Mechanism of Linux Containers [Master thesis], University of Electronic Science and Technology of China, China, 2015. [26] Breitgand D, Kutiel G, Raz D. Cost-aware live migration of services in the cloud. In: Proceedings of the 3rd Annual Haifa Experimental Systems Conference. Haifa, Israel: Association for Computing Machinery, 2010. Article No. 11 [27] Jaikar A, Huang D D, Kim G R, Noh S Y. Power efficient virtual machine migration in a scientific federated cloud. Cluster Computing, 2015, 18(2): 609-618 doi: 10.1007/s10586-015-0425-0 [28] Mogul J C, Farkas K I, Ranganathan P, Pinheiro E S. System and Method for Energy Efficient Data Prefetching, U.S. Patent 7437438, October 14, 2008 [29] Krishnan N, Baron D. A universal parallel two-pass MDL context tree compression algorithm. IEEE Journal of Selected Topics in Signal Processing, 2015, 9(4): 741-748 doi: 10.1109/JSTSP.2015.2403800 [30] LZ4 [Online], available: http://lz4.github.io/lz4, March 25, 2018 -
图(12) / 表(1)
计量
- 文章访问数: 366
- HTML全文浏览量: 206
- PDF下载量: 124
- 被引次数: 0
