mongodb的集群部署有3种方式,这里记录我在使用Replica Set(副本集)部署时的步骤。

首先这种方式至少需要3个节点——主、备、仲裁节点。主备节点存储数据,仲裁节点不存储数据,负责决定主节点挂掉之后哪个备节点升级为主节点。客户端同时连接主节点与备节点,不连接仲裁节点。

安装mongodb,CentOS下安装rpmfusion源后使用yum安装即可:yum install mongodb mongodb-server

安装完成后,修改/etc/mongodb.conf 这个配置文件。

1
2
3
4
5
6
7
8
9
10
11
bind_ip = 192.168.4.50
port = 27017
fork = true
pidfilepath = /var/run/mongodb/mongodb.pid
logpath = /var/log/mongodb/mongodb.log
dbpath =/var/lib/mongodb
journal = true
directoryperdb=true
logappend=true
replSet=testrs
noprealloc=true

其中注意修改bind_ip为各个节点的IP,下面的4行是我添加的:

  1. directoryperdb:为每一个数据库按照数据库名建立文件夹存放
  2. logappend:以追加的方式记录日志
  3. replSet:replica set的名字
  4. noprealloc:不预先分配存储

修改完成后,使用service mongod start即可启动服务。

随便链接一个节点后,进行配置:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
[root@pxc_bj63 ~]# service mongod start
Starting mongod: [ OK ]
[root@pxc_bj63 ~]# clear
[root@pxc_bj63 ~]# mongo 192.168.4.63:27017
MongoDB shell version: 2.4.12
connecting to: 192.168.4.63:27017/test
> use admin
switched to db admin
> conf = {_id:"testrs",members:[{_id:0,host:"192.168.4.63:27017",priority:2},{_id:1,host:"192.168.4.64:27017",priority:1},{_id:2,host:"192.168.4.50:27017",arbiterOnly:true}]};
{
"_id" : "testrs",
"members" : [
{
"_id" : 0,
"host" : "192.168.4.63:27017",
"priority" : 2
},
{
"_id" : 1,
"host" : "192.168.4.64:27017",
"priority" : 1
},
{
"_id" : 2,
"host" : "192.168.4.50:27017",
"arbiterOnly" : true
}
]
}

注意最初的_id的值就是我们配置文件中设定的replSet的值,arbiterOnly是大写的O,配置完成后进行初始化:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
> rs.initiate(conf)
{
"info" : "Config now saved locally. Should come online in about a minute.",
"ok" : 1
}
> rs.status()
{
"set" : "testrs",
"date" : ISODate("2015-03-03T03:17:25Z"),
"myState" : 1,
"members" : [
{
"_id" : 0,
"name" : "192.168.4.63:27017",
"health" : 1,
"state" : 1,
"stateStr" : "PRIMARY",
"uptime" : 460,
"optime" : Timestamp(1425352584, 1),
"optimeDate" : ISODate("2015-03-03T03:16:24Z"),
"self" : true
},
{
"_id" : 1,
"name" : "192.168.4.64:27017",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 60,
"optime" : Timestamp(1425352584, 1),
"optimeDate" : ISODate("2015-03-03T03:16:24Z"),
"lastHeartbeat" : ISODate("2015-03-03T03:17:25Z"),
"lastHeartbeatRecv" : ISODate("2015-03-03T03:17:25Z"),
"pingMs" : 0,
"syncingTo" : "192.168.4.63:27017"
},
{
"_id" : 2,
"name" : "192.168.4.50:27017",
"health" : 1,
"state" : 7,
"stateStr" : "ARBITER",
"uptime" : 58,
"lastHeartbeat" : ISODate("2015-03-03T03:17:25Z"),
"lastHeartbeatRecv" : ISODate("2015-03-03T03:17:24Z"),
"pingMs" : 1
}
],
"ok" : 1
}
testrs:PRIMARY>

插入一条数据试试:

1
2
3
testrs:PRIMARY> use test
switched to db test
testrs:PRIMARY> db.testdb.insert({"test1":"i'm master"})

目前,备节点是无法查询数据的:

1
2
3
4
5
6
7
8
9
10
11
12
13
[root@pxc_bj_64 ~]# mongo 192.168.4.64
MongoDB shell version: 2.4.12
connecting to: 192.168.4.64/test
Welcome to the MongoDB shell.
For interactive help, type "help".
For more comprehensive documentation, see
http://docs.mongodb.org/
Questions? Try the support group
http://groups.google.com/group/mongodb-user
testrs:SECONDARY> use test;
switched to db test
testrs:SECONDARY> show tables;
Tue Mar 3 11:41:00.274 error: { "$err" : "not master and slaveOk=false", "code" : 13435 } at src/mongo/shell/query.js:128

需要进行配置:

1
2
3
testrs:SECONDARY> db.getMongo().setSlaveOk();
testrs:SECONDARY> db.testdb.find();
{ "_id" : ObjectId("54f52c9073b3842303717165"), "test1" : "i'm master" }

设定setSlaveOK后便可以进行数据查询了。

现在,停止主库的服务,再查看整个集群的状态

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
testrs:SECONDARY> rs.status();
{
"set" : "testrs",
"date" : ISODate("2015-03-03T03:48:40Z"),
"myState" : 1,
"members" : [
{
"_id" : 0,
"name" : "192.168.4.63:27017",
"health" : 0,
"state" : 8,
"stateStr" : "(not reachable/healthy)",
"uptime" : 0,
"optime" : Timestamp(1425353872, 1),
"optimeDate" : ISODate("2015-03-03T03:37:52Z"),
"lastHeartbeat" : ISODate("2015-03-03T03:48:39Z"),
"lastHeartbeatRecv" : ISODate("2015-03-03T03:47:39Z"),
"pingMs" : 0
},
{
"_id" : 1,
"name" : "192.168.4.64:27017",
"health" : 1,
"state" : 1,
"stateStr" : "PRIMARY",
"uptime" : 2317,
"optime" : Timestamp(1425353872, 1),
"optimeDate" : ISODate("2015-03-03T03:37:52Z"),
"self" : true
},
{
"_id" : 2,
"name" : "192.168.4.50:27017",
"health" : 1,
"state" : 7,
"stateStr" : "ARBITER",
"uptime" : 1905,
"lastHeartbeat" : ISODate("2015-03-03T03:48:39Z"),
"lastHeartbeatRecv" : ISODate("2015-03-03T03:48:40Z"),
"pingMs" : 1
}
],
"ok" : 1
}

此时原来的备节点变成了主节点,此时我们新插入一条数据:

1
2
3
4
testrs:PRIMARY> db.testdb.insert({"test2":"i'm new master"})
testrs:PRIMARY> db.testdb.find();
{ "_id" : ObjectId("54f52c9073b3842303717165"), "test1" : "i'm master" }
{ "_id" : ObjectId("54f52f70abb951d25573981e"), "test2" : "i'm new master" }

重启原来的主节点并查看状态:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
[root@pxc_bj63 ~]# mongo 192.168.4.63
MongoDB shell version: 2.4.12
connecting to: 192.168.4.63/test
testrs:PRIMARY> rs.status()
{
"set" : "testrs",
"date" : ISODate("2015-03-03T03:50:57Z"),
"myState" : 1,
"members" : [
{
"_id" : 0,
"name" : "192.168.4.63:27017",
"health" : 1,
"state" : 1,
"stateStr" : "PRIMARY",
"uptime" : 59,
"optime" : Timestamp(1425354608, 1),
"optimeDate" : ISODate("2015-03-03T03:50:08Z"),
"self" : true
},
{
"_id" : 1,
"name" : "192.168.4.64:27017",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 59,
"optime" : Timestamp(1425354608, 1),
"optimeDate" : ISODate("2015-03-03T03:50:08Z"),
"lastHeartbeat" : ISODate("2015-03-03T03:50:56Z"),
"lastHeartbeatRecv" : ISODate("2015-03-03T03:50:57Z"),
"pingMs" : 0,
"lastHeartbeatMessage" : "syncing to: 192.168.4.63:27017",
"syncingTo" : "192.168.4.63:27017"
},
{
"_id" : 2,
"name" : "192.168.4.50:27017",
"health" : 1,
"state" : 7,
"stateStr" : "ARBITER",
"uptime" : 57,
"lastHeartbeat" : ISODate("2015-03-03T03:50:56Z"),
"lastHeartbeatRecv" : ISODate("2015-03-03T03:50:56Z"),
"pingMs" : 0
}
],
"ok" : 1
}

查询数据也同步过来了:

1
2
3
4
5
6
7
8
9
10
11
testrs:PRIMARY> db.testdb.find();
{ "_id" : ObjectId("54f52c9073b3842303717165"), "test1" : "i'm master" }
{ "_id" : ObjectId("54f52f70abb951d25573981e"), "test2" : "i'm new master" }
</pre>
当主节点恢复后,备节点查询数据时会提示:
<pre class="lang:sh decode:true ">testrs:PRIMARY> db.testdb.find();
Tue Mar 3 11:53:00.354 DBClientCursor::init call() failed
Error: error doing query: failed
Tue Mar 3 11:53:00.355 trying reconnect to 192.168.4.64:27017
Tue Mar 3 11:53:00.356 reconnect 192.168.4.64:27017 ok
testrs:SECONDARY>

如果要向集群中新增节点,则进行如下操作:

1
2
3
cfg = rs.conf()
cfg.members[3] = {_id:3,host:"192.168.4.42:27017",priority:0.5}
rs.reconfig(cfg)

新的备份节点配置之前是无法读数据的,记得设定setSlaveOK,不过虽然可读但备份节点还是不可写的。如果想修改配置也同理,首先得到当前配置信息,以修改优先级为例:

1
2
3
4
5
cfg = rs.conf()
cfg.members[0].priority = 0.5
cfg.members[1].priority = 2
cfg.members[2].priority = 2
rs.reconfig(cfg)

一定记得reconfig后新配置才会生效。