## 1、准备工作 ##
之前已经介绍过了 MongoDB 在[滴滴云](https://blog.didiyun.com)上的搭建。参考：[<<在滴滴云 DC2 云服务器上搭建 MongoDB 实战>>](http://blog.itpub.net/31559758/viewspace-2284946/)

但是生产环境中，通常需要更高的稳定性要求，单机版的 MongoDB 服务器并不能满足，因此我们需要高可用的 MongoDB 集群。

### 硬件准备 ###
本次只是模拟集群的搭建。因此直接采用上一次的搭建环境，通过不同的端口模拟集群。

## 2、简介 ##
MongoDB 的集群搭建方式主要有三种方式：Master-Slave，Replica Sets，Sharding。

其中 Master-Slave 官方已不再支持，故本文不再介绍。 本文主要介绍 Replica Sets 方式搭建高可用集群。

### 2.1 Replica Sets ###

#### 2.1.1 介绍 ####

副本集是一组维护相同数据集的 Mongod 实例。副本集包含多个数据承载节点和一个仲裁节点（可选），在数据承载节点中，一个且只有一个成员被视为主节点，而其他节点则被视为辅助节点。

主节点接收所有写入操作，一个副本集只能有一个主实例能够写入，主节点记录所有变更到它的 oplog。

![](https://blog.didiyun.com/wp-content/uploads/2019/02/1-1.png)
 
辅助节点复制主节点的 oplog 并将操作应用于数据集。

![](https://blog.didiyun.com/wp-content/uploads/2019/02/2-1.png)

也可以将一个额外的实例作为仲裁者添加到副本集。仲裁员不维护数据集，仲裁器的目的是通过响应其他副本集成员的心跳和选择请求来维护副本集中的仲裁。因为它们不存储数据集，所以仲裁器是提供副本集仲裁功能的一种好方法。

与具有数据集的完全功能副本集成员相比，仲裁器的资源成本更低，如果副本集的成员数为偶数，则添加一个仲裁器以在初选中获得多数票。

![](https://blog.didiyun.com/wp-content/uploads/2019/02/3-1.png)

当一个主服务器在超过配置的周期（默认为 10 秒）内未与该组的其他成员通信时，符合条件的辅助服务器将要求选择将其自身指定为新的主服务器。集群试图完成新的初选并恢复正常操作。

![](https://blog.didiyun.com/wp-content/uploads/2019/02/4-1.png)

#### 2.1.2 搭建过程 ####

登录之前搭建单机服务的机器，并切换到 root 账户：

```
ssh dc2-user@ip
sudo -i
```

创建配置文件：

```
vi /etc/mongod_27017.conf
```

文件内容如下：

```
# mongod_27017.conf

# for documentation of all options, see:
#   http://docs.mongodb.org/manual/reference/configuration-options/

# where to write logging data.
systemLog:
  destination: file
  logAppend: true
  # log的路径与上边创建的路径一一致
  path: /data/log/mongodb/mongod_27017.log

# Where and how to store data.
storage:
	# db储存的路径与上边创建的路径一一致
  dbPath: /data/db/mongo_27017
  journal:
    enabled: true
#  engine:
#  mmapv1:
#  wiredTiger:

# how the process runs
processManagement:
  fork: true  # fork and run in background
  pidFilePath: /var/run/mongodb/mongod_27017.pid  # location of pidfile
  timeZoneInfo: /usr/share/zoneinfo

# network interfaces
net:
  port: 27017
  bindIp: 127.0.0.1  # Enter 0.0.0.0,:: to bind to all IPv4 and IPv6 addresses or, alternatively, use the net.bindIpAll setting.

#security:

#operationProfiling:

replication:
   oplogSizeMB: 1024
   replSetName: test_mongo_replica_set
   secondaryIndexPrefetch: all
   enableMajorityReadConcern: true
#sharding:

## Enterprise-Only Options

#auditLog:

#snmp:

```

```
vi /etc/mongod_27018.conf
```

文件内容如下：

```
# mongod_27018.conf

# for documentation of all options, see:
#   http://docs.mongodb.org/manual/reference/configuration-options/

# where to write logging data.
systemLog:
  destination: file
  logAppend: true
  # log的路径与上边创建的路径一一致
  path: /data/log/mongodb/mongod_27018.log

# Where and how to store data.
storage:
	# db储存的路径与上边创建的路径一一致
  dbPath: /data/db/mongo_27018
  journal:
    enabled: true
#  engine:
#  mmapv1:
#  wiredTiger:

# how the process runs
processManagement:
  fork: true  # fork and run in background
  pidFilePath: /var/run/mongodb/mongod_27018.pid  # location of pidfile
  timeZoneInfo: /usr/share/zoneinfo

# network interfaces
net:
  port: 27018
  bindIp: 127.0.0.1  # Enter 0.0.0.0,:: to bind to all IPv4 and IPv6 addresses or, alternatively, use the net.bindIpAll setting.

#security:

#operationProfiling:

replication:
   oplogSizeMB: 1024
   replSetName: test_mongo_replica_set
   secondaryIndexPrefetch: all
   enableMajorityReadConcern: true
#sharding:

## Enterprise-Only Options

#auditLog:

#snmp:
```

```
vi /etc/mongod_27019.conf
```

文件内容如下：

```
# mongod_27019.conf

# for documentation of all options, see:
#   http://docs.mongodb.org/manual/reference/configuration-options/

# where to write logging data.
systemLog:
  destination: file
  logAppend: true
  # log的路径与上边创建的路径一一致
  path: /data/log/mongodb/mongod_27019.log

# Where and how to store data.
storage:
	# db储存的路径与上边创建的路径一一致
  dbPath: /data/db/mongo_27019
  journal:
    enabled: true
#  engine:
#  mmapv1:
#  wiredTiger:

# how the process runs
processManagement:
  fork: true  # fork and run in background
  pidFilePath: /var/run/mongodb/mongod_27019.pid  # location of pidfile
  timeZoneInfo: /usr/share/zoneinfo

# network interfaces
net:
  port: 27019
  bindIp: 127.0.0.1  # Enter 0.0.0.0,:: to bind to all IPv4 and IPv6 addresses or, alternatively, use the net.bindIpAll setting.

#security:

#operationProfiling:

replication:
   oplogSizeMB: 1024
   replSetName: test_mongo_replica_set
   secondaryIndexPrefetch: all
   enableMajorityReadConcern: true
#sharding:

## Enterprise-Only Options

#auditLog:

#snmp:

```

根据配置文件创建对应的目录：

```
mkdir /data/db/mongo_27017 && mkdir /data/db/mongo_27018 && mkdir /data/db/mongo_27019
```

启动对应的服务：

```
mongod -f /etc/mongod_27017.conf
mongod -f /etc/mongod_27018.conf
mongod -f /etc/mongod_27019.conf

```
见到如下输出即为启动成功

```
2019-01-28T22:23:39.965+0800 I CONTROL  [main] Automatically disabling TLS 1.0, to force-enable TLS 1.0 specify --sslDisabledProtocols 'none'
about to fork child process, waiting until server is ready for connections.
forked process: 11004
child process started successfully, parent exiting
```

使用 MongoDB shell 连接服务：

```
mongo
```

查看副本集的配置情况：

```
rs.conf()
```

此时我们还未配置副本集应显示如下：

```
2019-01-28T22:25:51.406+0800 E QUERY    [js] Error: Could not retrieve replica set config: {
	"operationTime" : Timestamp(0, 0),
	"ok" : 0,
	"errmsg" : "no replset config has been received",
	"code" : 94,
	"codeName" : "NotYetInitialized",
	"$clusterTime" : {
		"clusterTime" : Timestamp(0, 0),
		"signature" : {
			"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
			"keyId" : NumberLong(0)
		}
	}
} :
rs.conf@src/mongo/shell/utils.js:1448:11
@(shell):1:1
```

使用命令rs.initiate()来启动一个新的副本集：

```
rs.initiate()
```

显示如下：

```
{
	"info2" : "no configuration specified. Using a default configuration for the set",
	"me" : "127.0.0.1:27017",
	"ok" : 1,
	"operationTime" : Timestamp(1548685575, 1),
	"$clusterTime" : {
		"clusterTime" : Timestamp(1548685575, 1),
		"signature" : {
			"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
			"keyId" : NumberLong(0)
		}
	}
}
```

使用命令rs.add("HOST_NAME:PORT")添加副本集成员：

```
rs.add("127.0.0.1:27018")

```

显示如下：

```
{
	"ok" : 1,
	"operationTime" : Timestamp(1548685681, 1),
	"$clusterTime" : {
		"clusterTime" : Timestamp(1548685681, 1),
		"signature" : {
			"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
			"keyId" : NumberLong(0)
		}
	}
}
```

```
rs.add("127.0.0.1:27019")

```

显示如下：

```
{
	"ok" : 1,
	"operationTime" : Timestamp(1548685690, 1),
	"$clusterTime" : {
		"clusterTime" : Timestamp(1548685690, 1),
		"signature" : {
			"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
			"keyId" : NumberLong(0)
		}
	}
}
```

再次执行：

```
rs.conf()
```

此时我们还未配置副本集应显示如下：

```
{
	"_id" : "test_mongo_replica_set",
	"version" : 3,
	"protocolVersion" : NumberLong(1),
	"writeConcernMajorityJournalDefault" : true,
	"members" : [
		{
			"_id" : 0,
			"host" : "127.0.0.1:27017",
			"arbiterOnly" : false,
			"buildIndexes" : true,
			"hidden" : false,
			"priority" : 1,
			"tags" : {

},
			"slaveDelay" : NumberLong(0),
			"votes" : 1
		},
		{
			"_id" : 1,
			"host" : "127.0.0.1:27018",
			"arbiterOnly" : false,
			"buildIndexes" : true,
			"hidden" : false,
			"priority" : 1,
			"tags" : {

},
			"slaveDelay" : NumberLong(0),
			"votes" : 1
		},
		{
			"_id" : 2,
			"host" : "127.0.0.1:27019",
			"arbiterOnly" : false,
			"buildIndexes" : true,
			"hidden" : false,
			"priority" : 1,
			"tags" : {

},
			"slaveDelay" : NumberLong(0),
			"votes" : 1
		}
	],
	"settings" : {
		"chainingAllowed" : true,
		"heartbeatIntervalMillis" : 2000,
		"heartbeatTimeoutSecs" : 10,
		"electionTimeoutMillis" : 10000,
		"catchUpTimeoutMillis" : -1,
		"catchUpTakeoverDelayMillis" : 30000,
		"getLastErrorModes" : {

},
		"getLastErrorDefaults" : {
			"w" : 1,
			"wtimeout" : 0
		},
		"replicaSetId" : ObjectId("5c4f11068679c0236588f2bd")
	}
}
```

执行 db.isMaster() 查看是否为主节点：

```
db.isMaster() 
```

发现本身 27017 端口已经设置为了主节点：

```
{
	"hosts" : [
		"127.0.0.1:27017",
		"127.0.0.1:27018",
		"127.0.0.1:27019"
	],
	"setName" : "test_mongo_replica_set",
	"setVersion" : 3,
	"ismaster" : true,
	"secondary" : false,
	"primary" : "127.0.0.1:27017",
	"me" : "127.0.0.1:27017",
	"electionId" : ObjectId("7fffffff0000000000000001"),
	"lastWrite" : {
		"opTime" : {
			"ts" : Timestamp(1548685717, 1),
			"t" : NumberLong(1)
		},
		"lastWriteDate" : ISODate("2019-01-28T14:28:37Z"),
		"majorityOpTime" : {
			"ts" : Timestamp(1548685717, 1),
			"t" : NumberLong(1)
		},
		"majorityWriteDate" : ISODate("2019-01-28T14:28:37Z")
	},
	"maxBsonObjectSize" : 16777216,
	"maxMessageSizeBytes" : 48000000,
	"maxWriteBatchSize" : 100000,
	"localTime" : ISODate("2019-01-28T14:28:45.684Z"),
	"logicalSessionTimeoutMinutes" : 30,
	"minWireVersion" : 0,
	"maxWireVersion" : 7,
	"readOnly" : false,
	"ok" : 1,
	"operationTime" : Timestamp(1548685717, 1),
	"$clusterTime" : {
		"clusterTime" : Timestamp(1548685717, 1),
		"signature" : {
			"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
			"keyId" : NumberLong(0)
		}
	}
}
```

#### 2.1.3 验证效果 ####

连接主实例：

```
mongo  --port 27017
```

查看主状态：

```
 db.isMaster()
```

显示如下：
```
 {
	"hosts" : [
		"127.0.0.1:27017",
		"127.0.0.1:27018",
		"127.0.0.1:27019"
	],
	"setName" : "test_mongo_replica_set",
	"setVersion" : 3,
	"ismaster" : true,
	"secondary" : false,
	"primary" : "127.0.0.1:27017",
	"me" : "127.0.0.1:27017",
	"electionId" : ObjectId("7fffffff0000000000000001"),
	"lastWrite" : {
		"opTime" : {
			"ts" : Timestamp(1548685717, 1),
			"t" : NumberLong(1)
		},
		"lastWriteDate" : ISODate("2019-01-28T14:28:37Z"),
		"majorityOpTime" : {
			"ts" : Timestamp(1548685717, 1),
			"t" : NumberLong(1)
		},
		"majorityWriteDate" : ISODate("2019-01-28T14:28:37Z")
	},
	"maxBsonObjectSize" : 16777216,
	"maxMessageSizeBytes" : 48000000,
	"maxWriteBatchSize" : 100000,
	"localTime" : ISODate("2019-01-28T14:28:45.684Z"),
	"logicalSessionTimeoutMinutes" : 30,
	"minWireVersion" : 0,
	"maxWireVersion" : 7,
	"readOnly" : false,
	"ok" : 1,
	"operationTime" : Timestamp(1548685717, 1),
	"$clusterTime" : {
		"clusterTime" : Timestamp(1548685717, 1),
		"signature" : {
			"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
			"keyId" : NumberLong(0)
		}
	}
}
```

写入数据：

```
use mongo_test
db.mongo_test.insert({"name":"滴滴云MongoDBTest1","id":1})
```

读取数据：

```
db.mongo_test.find().pretty()
```

显示如下：

```
{
	"_id" : ObjectId("5c4f156792a30dbd4d11afdc"),
	"name" : "滴滴云MongoDBTest1",
	"id" : 1
}

```

再开一个窗口连接副本实例：

```
mongo  --port 27018
```

查看主状态：

```
db.isMaster()
```

显示如下：

```
{
	"hosts" : [
		"127.0.0.1:27017",
		"127.0.0.1:27018",
		"127.0.0.1:27019"
	],
	"setName" : "test_mongo_replica_set",
	"setVersion" : 3,
	"ismaster" : false,
	"secondary" : true,
	"primary" : "127.0.0.1:27017",
	"me" : "127.0.0.1:27018",
	"lastWrite" : {
		"opTime" : {
			"ts" : Timestamp(1548686587, 1),
			"t" : NumberLong(1)
		},
		"lastWriteDate" : ISODate("2019-01-28T14:43:07Z"),
		"majorityOpTime" : {
			"ts" : Timestamp(1548686587, 1),
			"t" : NumberLong(1)
		},
		"majorityWriteDate" : ISODate("2019-01-28T14:43:07Z")
	},
	"maxBsonObjectSize" : 16777216,
	"maxMessageSizeBytes" : 48000000,
	"maxWriteBatchSize" : 100000,
	"localTime" : ISODate("2019-01-28T14:43:12.576Z"),
	"logicalSessionTimeoutMinutes" : 30,
	"minWireVersion" : 0,
	"maxWireVersion" : 7,
	"readOnly" : false,
	"ok" : 1,
	"operationTime" : Timestamp(1548686587, 1),
	"$clusterTime" : {
		"clusterTime" : Timestamp(1548686587, 1),
		"signature" : {
			"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
			"keyId" : NumberLong(0)
		}
	}
}
```

模拟主实例故障 ( kill 进程) ：

```
ps aux|grep mongo

kill 对应的进程id

```

查看副本实例的状态（此处会有几秒的延迟，多执行几次查看效果)：

```
db.isMaster()
```

如发现如下显示，再执行几次，直到 ismaster 变为 true：

```
{
	"hosts" : [
		"127.0.0.1:27017",
		"127.0.0.1:27018",
		"127.0.0.1:27019"
	],
	"setName" : "test_mongo_replica_set",
	"setVersion" : 3,
	"ismaster" : false,
	"secondary" : true,
	"primary" : "127.0.0.1:27018",
	"me" : "127.0.0.1:27018",
	"electionId" : ObjectId("7fffffff0000000000000002"),
	"lastWrite" : {
		"opTime" : {
			"ts" : Timestamp(1548686967, 1),
			"t" : NumberLong(1)
		},
		"lastWriteDate" : ISODate("2019-01-28T14:49:27Z"),
		"majorityOpTime" : {
			"ts" : Timestamp(1548686967, 1),
			"t" : NumberLong(1)
		},
		"majorityWriteDate" : ISODate("2019-01-28T14:49:27Z")
	},
	"maxBsonObjectSize" : 16777216,
	"maxMessageSizeBytes" : 48000000,
	"maxWriteBatchSize" : 100000,
	"localTime" : ISODate("2019-01-28T14:49:40.069Z"),
	"logicalSessionTimeoutMinutes" : 30,
	"minWireVersion" : 0,
	"maxWireVersion" : 7,
	"readOnly" : false,
	"ok" : 1,
	"operationTime" : Timestamp(1548686967, 1),
	"$clusterTime" : {
		"clusterTime" : Timestamp(1548686979, 1),
		"signature" : {
			"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
			"keyId" : NumberLong(0)
		}
	}
}
```

即：

```
{
	"hosts" : [
		"127.0.0.1:27017",
		"127.0.0.1:27018",
		"127.0.0.1:27019"
	],
	"setName" : "test_mongo_replica_set",
	"setVersion" : 3,
	"ismaster" : true,
	"secondary" : false,
	"primary" : "127.0.0.1:27018",
	"me" : "127.0.0.1:27018",
	"electionId" : ObjectId("7fffffff0000000000000002"),
	"lastWrite" : {
		"opTime" : {
			"ts" : Timestamp(1548686981, 1),
			"t" : NumberLong(2)
		},
		"lastWriteDate" : ISODate("2019-01-28T14:49:41Z"),
		"majorityOpTime" : {
			"ts" : Timestamp(1548686981, 1),
			"t" : NumberLong(2)
		},
		"majorityWriteDate" : ISODate("2019-01-28T14:49:41Z")
	},
	"maxBsonObjectSize" : 16777216,
	"maxMessageSizeBytes" : 48000000,
	"maxWriteBatchSize" : 100000,
	"localTime" : ISODate("2019-01-28T14:49:51.062Z"),
	"logicalSessionTimeoutMinutes" : 30,
	"minWireVersion" : 0,
	"maxWireVersion" : 7,
	"readOnly" : false,
	"ok" : 1,
	"operationTime" : Timestamp(1548686981, 1),
	"$clusterTime" : {
		"clusterTime" : Timestamp(1548686981, 1),
		"signature" : {
			"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
			"keyId" : NumberLong(0)
		}
	}
}
```

读取刚才写入的数据：

```
use mongo_test
db.mongo_test.find().pretty()
```

显示如下：

```
{
	"_id" : ObjectId("5c4f156792a30dbd4d11afdc"),
	"name" : "滴滴云MongoDBTest1",
	"id" : 1
}
```

验证成功。

ITPub博客

在滴滴云上搭建 MongoDB 集群（一）：

最新文章

ITPub博客

在滴滴云上搭建 MongoDB 集群 （一）：

最新文章

在滴滴云上搭建 MongoDB 集群（一）：