Loading... <h1 style="font-size: 2.6em;margin: 21px 0px 10.5px;font-family: inherit;color: inherit">HA专题: 编译安装并配置DRBD</h1> <ul class=" list-paddingleft-2"> <li> <p>前言</p> </li> <li> <p>实验环境</p> </li> <li> <p>DRBD介绍</p> </li> <li> <p>安装前准备工作</p> </li> <li> <p>获取并编译安装DRBD</p> </li> <li> <p>配置DRBD</p> </li> <ul class=" list-paddingleft-2"> <li> <p>分区</p> </li> <li> <p>配置文件</p> </li> <li> <p>启动并设置DRBD</p> </li> <li> <p>格式化并挂载</p> </li> <li> <p>验证冗余性</p> </li> </ul> <li> <p>总结</p> </li> </ul> <h2 style="font-family: inherit;color: inherit;font-size: 2.15em;margin: 1.2em 0px 0.6em">前言</h2> <blockquote><p style="margin-top: 0px;font-size: 1em;margin-bottom: 0px"><span style="font-size: 16px;font-family: 黑体, SimHei">上篇我们讲到使用</span><code><span style="font-size: 16px;font-family: 黑体, SimHei">corosync</span></code><span style="font-size: 16px;font-family: 黑体, SimHei">+</span><code><span style="font-size: 16px;font-family: 黑体, SimHei">pacemaker</span></code><span style="font-size: 16px;font-family: 黑体, SimHei">实现</span><code><span style="font-size: 16px;font-family: 黑体, SimHei">MySQL</span></code><span style="font-size: 16px;font-family: 黑体, SimHei">高可用, 但是</span><code><span style="font-size: 16px;font-family: 黑体, SimHei">NFS</span></code><span style="font-size: 16px;font-family: 黑体, SimHei">容易成为单点故障从而导致数据丢失等严重问题, 我们可以使用</span><code><span style="font-size: 16px;font-family: 黑体, SimHei">drbd</span></code><span style="font-size: 16px;font-family: 黑体, SimHei">来实现</span><code><span style="font-size: 16px;font-family: 黑体, SimHei">MySQL</span></code><span style="font-size: 16px;font-family: 黑体, SimHei">数据的高可用, 本文介绍如何编译安装</span><code><span style="font-size: 16px;font-family: 黑体, SimHei">drbd</span></code><span style="font-size: 16px;font-family: 黑体, SimHei">并配置</span><code><span style="font-size: 16px;font-family: 黑体, SimHei">drbd</span></code><span style="font-size: 16px;font-family: 黑体, SimHei">实现不同主机的分区镜像</span></p> </blockquote> <h2 style="font-family: inherit;color: inherit;font-size: 2.15em;margin: 1.2em 0px 0.6em">实验环境</h2> <p style="margin-top: 0px;margin-bottom: 1.1em"><code>今天实验环境比较简单, 所以就不画图了</code></p> <table> <thead> <tr class="firstRow"> <th>主机</th> <th>IP</th> <th>功用</th> </tr> </thead> <tbody> <tr> <td style="padding: 0.5em;vertical-align: top">node1.anyisalin.com</td> <td style="padding: 0.5em;vertical-align: top">172.16.1.2</td> <td style="padding: 0.5em;vertical-align: top">drbd节点</td> </tr> <tr> <td style="padding: 0.5em;vertical-align: top">node2.anyisalin.com</td> <td style="padding: 0.5em;vertical-align: top">172.16.1.3</td> <td style="padding: 0.5em;vertical-align: top">drbd节点</td> </tr> </tbody> </table> <p style="margin-top: 0px;margin-bottom: 1.1em"><code>注意: 本文实验中所有主机SElinux和iptables都是关闭的</code></p> <h2 style="font-family: inherit;color: inherit;font-size: 2.15em;margin: 1.2em 0px 0.6em">DRBD介绍</h2> <blockquote><p style="margin-top: 0px;font-size: 1em;margin-bottom: 1.1em"><span style="font-family: 黑体, SimHei;font-size: 16px">The Distributed Replicated Block Device (DRBD) is a software-based, shared-nothing, replicated storage solution mirroring the content of block devices (hard disks, partitions, logical volumes etc.) between hosts. </span><span style="font-family: 黑体, SimHei;font-size: 16px">转自官方文档: Chapter 1. DRBD Fundamentals</span></p> <p style="margin-top: 0px;font-size: 1em;margin-bottom: 1.1em"><span style="font-family: 黑体, SimHei;font-size: 16px">分布式文件块(DRBD)是一种基于软件, 无共享的, 复制, 镜像主机间块设备的内容(lvm, partition, disk..)的解决方案</span></p> <p style="margin-top: 0px;font-size: 1em;margin-bottom: 1.1em"><span style="font-family: 黑体, SimHei;font-size: 16px">DRBD镜像数据的特点:</span></p> <ul style="margin-bottom: 1.1em" class=" list-paddingleft-2"> <li> <p><span style="font-family: 黑体, SimHei;font-size: 16px">实时(in real time): 实时复制镜像设备的内容</span></p> </li> <li> <p><span style="font-family: 黑体, SimHei;font-size: 16px">透明(transparently): 对于用户和应用程序镜像数据的过程是透明的</span></p> </li> <li> <p><span style="font-family: 黑体, SimHei;font-size: 16px">同步、异步(synchronously、asynchronously): 同步模式下, 程序会通知所欲主机并在所有的写操作写入所有主机才会写入本地磁盘、异步模式下, 程序会在通知写操作写入其他主机之前就写入本地磁盘</span></p> </li> </ul> <p style="margin-top: 0px;font-size: 1em;margin-bottom: 1.1em"><span style="font-family: 黑体, SimHei;font-size: 16px"><strong>DRBD在Linux内核I/O栈的位置</strong><br /></span><img src="//cto.wang/usr/uploads/2016/07/20160703161239-51.gif" alt="Alt text" longdesc="./drbd.gif" /><img src="//cto.wang/usr/uploads/2016/07/20160703161239-100.png" title="1460436218761843.png" alt="blob.png" /></p> <p style="margin-top: 0px;font-size: 1em;margin-bottom: 1.1em"><span style="font-family: 黑体, SimHei;font-size: 16px"><strong>DRBD的三种协议</strong><br />上面的图可以看出DRBD在I/O栈的位置, 也大体可以了解DRBD的工作流程, 其实DRBD有三种工作模型, 我们称它为三种工作协议, 分别为protcol A, protcol B, protocol C, 我们大体说一下这三种工作协议</span></p> <ul style="margin-bottom: 1.1em" class=" list-paddingleft-2"> <li> <p><span style="font-family: 黑体, SimHei;font-size: 16px">Protocol A: Asynchronous, 也称为异步模型, 当DRBD将数据流传到本机网卡接口时就视为完成传输</span></p> </li> <li> <p><span style="font-family: 黑体, SimHei;font-size: 16px">Protocol B: Semi-Synchronous, 也成为半同步模型, 当DRBD将数据流传到目标主机的网卡接口时视为完成传输</span></p> </li> <li> <p><span style="font-family: 黑体, SimHei;font-size: 16px">Protocol C: Synchronous, 也成为同步模型, 当DRBD将数据流传到目标主机并且目标主机将数据写入DRBD设备时才视其为完成传输</span></p> </li> </ul> <p style="margin-top: 0px;font-size: 1em;margin-bottom: 1.1em"><span style="font-family: 黑体, SimHei;font-size: 16px"><strong>DRBD在用户空间的管理命令</strong></span></p> <ul class=" list-paddingleft-2"> <li> <p><span style="font-family: 黑体, SimHei;font-size: 16px">drbdadm: 在drbd套件用较为高级的程序, 配置方法较为简单, 获取配置文件中的参数来进行配置, 算是drbdsetup和drbdmeta的前端工具</span></p> </li> <li> <p><span style="font-family: 黑体, SimHei;font-size: 16px">drbdsetup: 可以直接配置加载的DRBD模块, 配置较为复杂</span></p> </li> <li> <p><span style="font-family: 黑体, SimHei;font-size: 16px">drbdmeta: 允许创建、转储、还原和修改DRBD元数据结构, 配置更为复杂</span></p> </li> </ul> </blockquote> <h2 style="font-family: inherit;color: inherit;font-size: 2.15em;margin: 1.2em 0px 0.6em">安装前准备工作</h2> <blockquote><p style="margin-top: 0px;font-size: 1em;margin-bottom: 0px"><span style="font-family: 黑体, SimHei;font-size: 16px">配置一个HA集群的前提需要配置时间同步, 双机互信, 主机名解析<br />由于我们以前的博文已经讲解过如何配置:可以查看</span><span style="font-family: 黑体, SimHei;font-size: 16px">HA配置准备工作</span></p> </blockquote> <h2 style="font-family: inherit;color: inherit;font-size: 2.15em;margin: 1.2em 0px 0.6em">获取并编译安装DRBD</h2> <blockquote><p style="margin-top: 0px;font-size: 1em;margin-bottom: 0px"><span style="font-family: 黑体, SimHei;font-size: 16px">我这里系统是</span><code><span style="font-family: 黑体, SimHei;font-size: 16px">CentOS 6.7</span></code><span style="font-family: 黑体, SimHei;font-size: 16px">, 通过官方站点</span><span style="font-family: 黑体, SimHei;font-size: 16px">DRBD DownLoad</span><span style="font-family: 黑体, SimHei;font-size: 16px">下载</span><span style="font-family: 黑体, SimHei;font-size: 16px">drbd-8.4.4.tar.gz</span></p> </blockquote> <pre>[root@node1 ~]# yum groupinstall "Development Tools" "Server Platform Development" -y --nogpgcheck #安装开发包组 [root@node1 ~]# wget http://oss.linbit.com/drbd/8.4/drbd-8.4.4.tar.gz #下载drbd [root@node1 ~]# tar xf drbd-8.4.4.tar.gz -C /usr/src/ [root@node1 ~]# cd /usr/src/drbd-8.4.4/ #编译安装 [root@node1 drbd-8.4.4]# ./configure --prefix=/usr/local/drbd --sysconfdir=/etc/ --with-km [root@node1 drbd-8.4.4]# make DIR=/usr/src/kernels/2.6.32-573.el6.x86_64/ [root@node1 drbd-8.4.4]# make install [root@node1 drbd-8.4.4]# cd drbd [root@node1 drbd]# make KDIR=/usr/src/kernels/2.6.32-573.el6.x86_64/ [root@node1 drbd]# cp drbd.ko /lib/modules/2.6.32-573.el6.x86_64/kernel/lib/ #装载模块 [root@node1 drbd]# depmod [root@node1 drbd]# modprobe drbd [root@node1 drbd]# lsmod | grep drbd drbd 327242 0 libcrc32c 1246 1 drbd node2的操作同上, 由于篇幅原因不做叙述</pre> <h2 style="font-family: inherit;color: inherit;font-size: 2.15em;margin: 1.2em 0px 0.6em">配置DRBD</h2> <h3 style="font-family: inherit;color: inherit;font-size: 1.7em;margin: 1.2em 0px 0.6em">分区</h3> <blockquote><p style="margin-top: 0px;font-size: 1em;margin-bottom: 0px"><code><span style="font-family: 黑体, SimHei;font-size: 16px">注意:创建完分区后不能对其进行格式化, 下面的操作在node1和node2都需要执行</span></code></p> </blockquote> <pre>[root@node1 ~]# fdisk /dev/sdb Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel Building a new DOS disklabel with disk identifier 0xa5927bb4. Changes will remain in memory only, until you decide to write them. After that, of course, the previous content won't be recoverable. Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite) WARNING: DOS-compatible mode is deprecated. It's strongly recommended to switch off the mode (command 'c') and change display units to sectors (command 'u'). Command (m for help): n Command action e extended p primary partition (1-4) p Partition number (1-4): 1 First cylinder (1-2610, default 1): Using default value 1 Last cylinder, +cylinders or +size{K,M,G} (1-2610, default 2610): +10G Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. Syncing disks. [root@node1 ~]# partx -a /dev/sdb BLKPG: Device or resource busy error adding partition 1 [root@node1 ~]# partx -a /dev/sdb BLKPG: Device or resource busy error adding partition 1</pre> <h3 style="font-family: inherit;color: inherit;font-size: 1.7em;margin: 1.2em 0px 0.6em">配置文件</h3> <blockquote><p style="margin-top: 0px;font-size: 1em;margin-bottom: 0px"><span style="font-family: 黑体, SimHei;font-size: 16px">配置文件如下, </span><code><span style="font-family: 黑体, SimHei;font-size: 16px">/etc/drbd.d/glob-common.conf</span></code><span style="font-family: 黑体, SimHei;font-size: 16px">, 并复制到node2上</span></p> </blockquote> <pre>global { usage-count no; # minor-count dialog-refresh disable-ip-verification } common { protocol C; handlers { pri-on-incon-degr "/usr/lib/drbd/notify-pri-on-incon-degr.sh; /usr/lib/drbd/notify-emergency-reboot.sh; echo b > /proc/sysrq-trigger ; reboot -f"; pri-lost-after-sb "/usr/lib/drbd/notify-pri-lost-after-sb.sh; /usr/lib/drbd/notify-emergency-reboot.sh; echo b > /proc/sysrq-trigger ; reboot -f"; local-io-error "/usr/lib/drbd/notify-io-error.sh; /usr/lib/drbd/notify-emergency-shutdown.sh; echo o > /proc/sysrq-trigger ; halt -f"; # fence-peer "/usr/lib/drbd/crm-fence-peer.sh"; # split-brain "/usr/lib/drbd/notify-split-brain.sh root"; # out-of-sync "/usr/lib/drbd/notify-out-of-sync.sh root"; # before-resync-target "/usr/lib/drbd/snapshot-resync-target-lvm.sh -p 15 -- -c 16k"; # after-resync-target /usr/lib/drbd/unsnapshot-resync-target-lvm.sh; } startup { #wfc-timeout 120; #degr-wfc-timeout 120; } disk { on-io-error detach; #fencing resource-only; } net { cram-hmac-alg "sha1"; shared-secret "mydrbdlab"; } syncer { rate 1000M; }</pre> <blockquote><p style="margin-top: 0px;font-size: 1em;margin-bottom: 0px">配置文件如下, <code>/etc/drbd.d/data.res</code>, 并复制到node2上</p> </blockquote> <pre>resource data { on node1.anyisalin.com { device /dev/drbd0; disk /dev/sdb1; address 172.16.1.2:7789; meta-disk internal; } on node2.anyisalin.com { device /dev/drbd0; disk /dev/sdb1; address 172.16.1.3:7789; meta-disk internal; } }</pre> <h3 style="font-family: inherit;color: inherit;font-size: 1.7em;margin: 1.2em 0px 0.6em">启动并设置DRBD</h3> <blockquote><p style="margin-top: 0px;font-size: 1em;margin-bottom: 0px"><span style="font-family: 黑体, SimHei;font-size: 16px">首先我们先在</span><code><span style="font-family: 黑体, SimHei;font-size: 16px">node1</span></code><span style="font-family: 黑体, SimHei;font-size: 16px">和</span><code><span style="font-family: 黑体, SimHei;font-size: 16px">node2</span></code><span style="font-family: 黑体, SimHei;font-size: 16px">分别初始化资源</span></p> </blockquote> <pre>[root@node1 drbd.d]# drbdadm create-md data #node1运行 [root@node2 drbd.d]# drbdadm create-md data #node2运行</pre> <blockquote><p style="margin-top: 0px;font-size: 1em;margin-bottom: 0px"><span style="font-size: 16px;font-family: 黑体, SimHei">同时在</span><code><span style="font-size: 16px;font-family: 黑体, SimHei">node1</span></code><span style="font-size: 16px;font-family: 黑体, SimHei">和</span><code><span style="font-size: 16px;font-family: 黑体, SimHei">node2</span></code><span style="font-size: 16px;font-family: 黑体, SimHei">上启动</span><code><span style="font-size: 16px;font-family: 黑体, SimHei">drbd</span></code></p> </blockquote> <pre>[root@node1 drbd.d]# service drbd start #node1运行 [root@node2 drbd.d]# service drbd start #node2运行</pre> <blockquote><p style="margin-top: 0px;font-size: 1em;margin-bottom: 0px"><span style="font-family: 黑体, SimHei;font-size: 16px">设置</span><code><span style="font-family: 黑体, SimHei;font-size: 16px">node1</span></code><span style="font-family: 黑体, SimHei;font-size: 16px">为主节点</span></p> </blockquote> <pre>[root@node1 drbd.d]# drbdadm primary --force data [root@node1 drbd.d]# cat /proc/drbd #可以看到同步过程, 手慢了没截到图 # 同步完成后 0: cs:Connected ro:Primary/Secondary ds:UpToDate/Diskless C r----- ns:0 nr:0 dw:0 dr:792 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:10490052</pre> <h3 style="font-family: inherit;color: inherit;font-size: 1.7em;margin: 1.2em 0px 0.6em">格式化并挂载</h3> <blockquote><p style="margin-top: 0px;font-size: 1em;margin-bottom: 0px"><span style="font-size: 16px;font-family: 黑体, SimHei">我们需要对drbd设备格式化并挂载才能使用, </span><code><span style="font-size: 16px;font-family: 黑体, SimHei">P.S</span></code><span style="font-size: 16px;font-family: 黑体, SimHei"> drbd正常情况下只有主节点能够挂载</span></p> </blockquote> <pre>[root@node1 drbd.d]# mkfs.ext4 /dev/drbd0 #格式化drbd设备 [root@node1 drbd.d]# mkdir /data [root@node1 drbd.d]# mount /dev/drbd0 /data/ #挂载到data目录 [root@node1 drbd.d]# cd /data/ [root@node1 data]# touch node1.anyisalin.com #创建文件在data目录</pre> <h3 style="font-family: inherit;color: inherit;font-size: 1.7em;margin: 1.2em 0px 0.6em">验证冗余性</h3> <blockquote><p style="margin-top: 0px;font-size: 1em;margin-bottom: 0px"><span style="font-family: 黑体, SimHei;font-size: 16px">我们将</span><code><span style="font-family: 黑体, SimHei;font-size: 16px">node1</span></code><span style="font-family: 黑体, SimHei;font-size: 16px">设置为</span><code><span style="font-family: 黑体, SimHei;font-size: 16px">seondary</span></code><span style="font-family: 黑体, SimHei;font-size: 16px">并将</span><code><span style="font-family: 黑体, SimHei;font-size: 16px">node2</span></code><span style="font-family: 黑体, SimHei;font-size: 16px">设置为</span><code><span style="font-family: 黑体, SimHei;font-size: 16px">primary</span></code></p> </blockquote> <pre>[root@node1 ~]# umount /data/ #首先要卸载drbd0 [root@node1 ~]# drbdadm secondary data #设置为secondary [root@node2 ~]# drbdadm primary data #设置node2为primary [root@node2 ~]# cat /proc/drbd #现在为Primary 2016-04-12 10:29:41 0: cs:Connected ro:Primary/Secondary ds:Diskless/UpToDate C r----- ns:0 nr:664 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0 [root@node2 ~]# mkdir /data [root@node2 ~]# mount /dev/drbd0 /data/ #挂载到/data [root@node2 ~]# cd /data/ [root@node2 data]# ls #还是能查看到node1创建的文件 lost+found node1.anyisalin.com [root@node2 data]# touch node2.anyisalin.com</pre> <h2 style="font-family: inherit;color: inherit;font-size: 2.15em;margin: 1.2em 0px 0.6em">总结</h2> <blockquote><p style="margin-top: 0px;font-size: 1em;margin-bottom: 1.1em"><span style="font-family: 黑体, SimHei;font-size: 16px">本篇文章的</span><code><span style="font-family: 黑体, SimHei;font-size: 16px">DRBD</span></code><span style="font-family: 黑体, SimHei;font-size: 16px">还不算特别完整, 只能通过手动来切换主从, 我们下篇文章介绍如何将</span><code><span style="font-family: 黑体, SimHei;font-size: 16px">DRBD</span></code><span style="font-family: 黑体, SimHei;font-size: 16px">配置成集群资源并结合</span><code><span style="font-family: 黑体, SimHei;font-size: 16px">MySQL</span></code><span style="font-family: 黑体, SimHei;font-size: 16px">, 实现</span><code><span style="font-family: 黑体, SimHei;font-size: 16px">MySQL</span></code><span style="font-family: 黑体, SimHei;font-size: 16px">数据库的高可用</span></p> <p style="margin-top: 0px;font-size: 1em;margin-bottom: 0px"><span style="font-family: 黑体, SimHei;font-size: 16px">作者水平很低, 如果有错误及时指出, 如果你觉得本文写的好请点一波赞~(≧▽≦)/~<br />作者: AnyISaIln QQ: 1449472454<br />感谢: MageEdu</span></p> </blockquote> 最后修改:2021 年 12 月 10 日 10 : 53 AM © 允许规范转载 赞赏 如果觉得我的文章对你有用,请随意赞赏 赞赏作者 支付宝微信