| 主機名 | 角色 | ip地址 |
| node01 | router | 192.168.0.41 |
| node02/node03/node04 | config server replication set |
192.168.0.42 192.168.0.43 192.168.0.44 |
| node05/node06/node07 | shard1 replication set |
192.168.0.45 192.168.0.46 192.168.0.47 |
| node08/node09/node10 | shard2 replication set |
192.168.0.48 192.168.0.49 192.168.0.50 |
基礎環境，各server做時間同步，關閉防火墻，關閉selinux，ssh互信，主機名解析
主機名解析
[root@node01 ~]# cat /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 192.168.0.99 time.test.org time-node 192.168.0.41 node01.test.org node01 192.168.0.42 node02.test.org node02 192.168.0.43 node03.test.org node03 192.168.0.44 node04.test.org node04 192.168.0.45 node05.test.org node05 192.168.0.46 node06.test.org node06 192.168.0.47 node07.test.org node07 192.168.0.48 node08.test.org node08 192.168.0.49 node09.test.org node09 192.168.0.50 node10.test.org node10 192.168.0.51 node11.test.org node11 192.168.0.52 node12.test.org node12 [root@node01 ~]#
準備好基礎環境以后，配置mongodb yum源
[root@node01 ~]# cat /etc/yum.repos.d/mongodb.repo [mongodb-org] name = MongoDB Repository baseurl = https://mirrors.aliyun.com/mongodb/yum/redhat/7/mongodb-org/4.4/x86_64/ gpgcheck = 1 enabled = 1 gpgkey = https://www.mongodb.org/static/pgp/server-4.4.asc [root@node01 ~]#
將mongodb yum源復制給其他節點
[root@node01 ~]# for i in {02..10} ; do scp /etc/yum.repos.d/mongodb.repo node$i:/etc/yum.repos.d/; done
mongodb.repo 100% 206 247.2KB/s 00:00
mongodb.repo 100% 206 222.3KB/s 00:00
mongodb.repo 100% 206 118.7KB/s 00:00
mongodb.repo 100% 206 164.0KB/s 00:00
mongodb.repo 100% 206 145.2KB/s 00:00
mongodb.repo 100% 206 119.9KB/s 00:00
mongodb.repo 100% 206 219.2KB/s 00:00
mongodb.repo 100% 206 302.1KB/s 00:00
mongodb.repo 100% 206 289.3KB/s 00:00
[root@node01 ~]#
在每個節點上安裝mongodb-org這個包
for i in {01..10} ;
do ssh node$i ' yum -y install mongodb-org ';
done
在config server 和shard節點上創建數據目錄和日志目錄，并將其屬主和屬組更改為mongod
[root@node01 ~]# for i in {02..10} ; do ssh node$i 'mkdir -p /mongodb/{data,log} chown -R mongod.mongod /mongodb/ ls -ld /mongodb'; done
drwxr-xr-x 4 mongod mongod 29 Nov 11 22:47 /mongodb
drwxr-xr-x 4 mongod mongod 29 Nov 11 22:45 /mongodb
drwxr-xr-x 4 mongod mongod 29 Nov 11 22:45 /mongodb
drwxr-xr-x 4 mongod mongod 29 Nov 11 22:45 /mongodb
drwxr-xr-x 4 mongod mongod 29 Nov 11 22:45 /mongodb
drwxr-xr-x 4 mongod mongod 29 Nov 11 22:45 /mongodb
drwxr-xr-x 4 mongod mongod 29 Nov 11 22:45 /mongodb
drwxr-xr-x 4 mongod mongod 29 Nov 11 22:45 /mongodb
drwxr-xr-x 4 mongod mongod 29 Nov 11 22:45 /mongodb
[root@node01 ~]#
配置shard1 replication set
[root@node05 ~]# cat /etc/mongod.conf systemLog: destination: file logAppend: true path: /mongodb/log/mongod.log storage: dbPath: /mongodb/data/ journal: enabled: true processManagement: fork: true pidFilePath: /var/run/mongodb/mongod.pid timeZoneInfo: /usr/share/zoneinfo net: bindIp: 0.0.0.0 sharding: clusterRole: shardsvr replication: replSetName: shard1_replset [root@node05 ~]# scp /etc/mongod.conf node06:/etc/ mongod.conf 100% 360 394.5KB/s 00:00 [root@node05 ~]# scp /etc/mongod.conf node07:/etc/ mongod.conf 100% 360 351.7KB/s 00:00 [root@node05 ~]#
配置shard2 replication set
[root@node08 ~]# cat /etc/mongod.conf systemLog: destination: file logAppend: true path: /mongodb/log/mongod.log storage: dbPath: /mongodb/data/ journal: enabled: true processManagement: fork: true pidFilePath: /var/run/mongodb/mongod.pid timeZoneInfo: /usr/share/zoneinfo net: bindIp: 0.0.0.0 sharding: clusterRole: shardsvr replication: replSetName: shard2_replset [root@node08 ~]# scp /etc/mongod.conf node09:/etc/ mongod.conf 100% 360 330.9KB/s 00:00 [root@node08 ~]# scp /etc/mongod.conf node10:/etc/ mongod.conf 100% 360 385.9KB/s 00:00 [root@node08 ~]#
啟動shard1 replication set和shard2 replication set
[root@node05 ~]# systemctl start mongod.service
[root@node05 ~]# ss -tnl
State Recv-Q Send-Q Local Address:Port Peer Address:Port
LISTEN 0 128 *:22 *:*
LISTEN 0 100 127.0.0.1:25 *:*
LISTEN 0 128 *:27018 *:*
LISTEN 0 128 :::22 :::*
LISTEN 0 100 ::1:25 :::*
[root@node05 ~]#for i in {06..10} ; do ssh node$i 'systemctl start mongod.service ss -tnl';done
State Recv-Q Send-Q Local Address:Port Peer Address:Port
LISTEN 0 128 *:22 *:*
LISTEN 0 100 127.0.0.1:25 *:*
LISTEN 0 128 *:27018 *:*
LISTEN 0 128 :::22 :::*
LISTEN 0 100 ::1:25 :::*
State Recv-Q Send-Q Local Address:Port Peer Address:Port
LISTEN 0 128 *:22 *:*
LISTEN 0 100 127.0.0.1:25 *:*
LISTEN 0 128 *:27018 *:*
LISTEN 0 128 :::22 :::*
LISTEN 0 100 ::1:25 :::*
State Recv-Q Send-Q Local Address:Port Peer Address:Port
LISTEN 0 128 *:22 *:*
LISTEN 0 100 127.0.0.1:25 *:*
LISTEN 0 128 *:27018 *:*
LISTEN 0 128 :::22 :::*
LISTEN 0 100 ::1:25 :::*
State Recv-Q Send-Q Local Address:Port Peer Address:Port
LISTEN 0 128 *:22 *:*
LISTEN 0 100 127.0.0.1:25 *:*
LISTEN 0 128 *:27018 *:*
LISTEN 0 128 :::22 :::*
LISTEN 0 100 ::1:25 :::*
State Recv-Q Send-Q Local Address:Port Peer Address:Port
LISTEN 0 128 *:22 *:*
LISTEN 0 100 127.0.0.1:25 *:*
LISTEN 0 128 *:27018 *:*
LISTEN 0 128 :::22 :::*
LISTEN 0 100 ::1:25 :::*
[root@node05 ~]#
提示：默認不指定shard監聽端口，它默認就監聽在27018端口，所以啟動shard節點后，請確保27018端口正常監聽即可；
連接node05的mongodb 初始化shard1_replset副本集
> rs.initiate(
... {
... _id : "shard1_replset",
... members: [
... { _id : 0, host : "node05:27018" },
... { _id : 1, host : "node06:27018" },
... { _id : 2, host : "node07:27018" }
... ]
... }
... )
{
"ok" : 1,
"$clusterTime" : {
"clusterTime" : Timestamp(1605107401, 1),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
},
"operationTime" : Timestamp(1605107401, 1)
}
shard1_replset:SECONDARY>
連接node08的mongodb 初始化shard2_replset副本集
> rs.initiate(
... {
... _id : "shard2_replset",
... members: [
... { _id : 0, host : "node08:27018" },
... { _id : 1, host : "node09:27018" },
... { _id : 2, host : "node10:27018" }
... ]
... }
... )
{
"ok" : 1,
"$clusterTime" : {
"clusterTime" : Timestamp(1605107644, 1),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
},
"operationTime" : Timestamp(1605107644, 1)
}
shard2_replset:OTHER>
配置configserver replication set
[root@node02 ~]# cat /etc/mongod.conf systemLog: destination: file logAppend: true path: /mongodb/log/mongod.log storage: dbPath: /mongodb/data/ journal: enabled: true processManagement: fork: true pidFilePath: /var/run/mongodb/mongod.pid timeZoneInfo: /usr/share/zoneinfo net: bindIp: 0.0.0.0 sharding: clusterRole: configsvr replication: replSetName: cfg_replset [root@node02 ~]# scp /etc/mongod.conf node03:/etc/mongod.conf mongod.conf 100% 358 398.9KB/s 00:00 [root@node02 ~]# scp /etc/mongod.conf node04:/etc/mongod.conf mongod.conf 100% 358 270.7KB/s 00:00 [root@node02 ~]#
啟動config server
[root@node02 ~]# systemctl start mongod.service [root@node02 ~]# ss -tnl State Recv-Q Send-Q Local Address:Port Peer Address:Port LISTEN 0 128 *:27019 *:* LISTEN 0 128 *:22 *:* LISTEN 0 100 127.0.0.1:25 *:* LISTEN 0 128 :::22 :::* LISTEN 0 100 ::1:25 :::* [root@node02 ~]# ssh node03 'systemctl start mongod.service ss -tnl' State Recv-Q Send-Q Local Address:Port Peer Address:Port LISTEN 0 128 *:27019 *:* LISTEN 0 128 *:22 *:* LISTEN 0 100 127.0.0.1:25 *:* LISTEN 0 128 :::22 :::* LISTEN 0 100 ::1:25 :::* [root@node02 ~]# ssh node04 'systemctl start mongod.service ss -tnl' State Recv-Q Send-Q Local Address:Port Peer Address:Port LISTEN 0 128 *:27019 *:* LISTEN 0 128 *:22 *:* LISTEN 0 100 127.0.0.1:25 *:* LISTEN 0 128 :::22 :::* LISTEN 0 100 ::1:25 :::* [root@node02 ~]#
提示：config server 默認在不指定端口的情況監聽在27019這個端口，啟動后，請確保該端口處于正常監聽；
連接node02的mongodb，初始化cfg_replset 副本集
> rs.initiate(
... {
... _id: "cfg_replset",
... configsvr: true,
... members: [
... { _id : 0, host : "node02:27019" },
... { _id : 1, host : "node03:27019" },
... { _id : 2, host : "node04:27019" }
... ]
... }
... )
{
"ok" : 1,
"$gleStats" : {
"lastOpTime" : Timestamp(1605108177, 1),
"electionId" : ObjectId("000000000000000000000000")
},
"lastCommittedOpTime" : Timestamp(0, 0),
"$clusterTime" : {
"clusterTime" : Timestamp(1605108177, 1),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
},
"operationTime" : Timestamp(1605108177, 1)
}
cfg_replset:SECONDARY>
配置router
[root@node01 ~]# cat /etc/mongos.conf systemLog: destination: file path: /var/log/mongodb/mongos.log logAppend: true processManagement: fork: true net: bindIp: 0.0.0.0 sharding: configDB: "cfg_replset/node02:27019,node03:27019,node04:27019" [root@node01 ~]#
提示：configDB必須是副本集名稱/成員監聽地址：port的形式，成員至少要寫一個；
啟動router
[root@node01 ~]# mongos -f /etc/mongos.conf about to fork child process, waiting until server is ready for connections. forked process: 1510 child process started successfully, parent exiting [root@node01 ~]# ss -tnl State Recv-Q Send-Q Local Address:Port Peer Address:Port LISTEN 0 128 *:22 *:* LISTEN 0 100 127.0.0.1:25 *:* LISTEN 0 128 *:27017 *:* LISTEN 0 128 :::22 :::* LISTEN 0 100 ::1:25 :::* [root@node01 ~]#
連接mongos，添加shard1 replication set 和shard2 replication set
mongos> sh.addShard("shard1_replset/node05:27018,node06:27018,node07:27018")
{
"shardAdded" : "shard1_replset",
"ok" : 1,
"operationTime" : Timestamp(1605109085, 3),
"$clusterTime" : {
"clusterTime" : Timestamp(1605109086, 1),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
}
}
mongos> sh.addShard("shard2_replset/node08:27018,node09:27018,node10:27018")
{
"shardAdded" : "shard2_replset",
"ok" : 1,
"operationTime" : Timestamp(1605109118, 2),
"$clusterTime" : {
"clusterTime" : Timestamp(1605109118, 3),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
}
}
mongos>
提示：添加shard 副本集也是需要指明副本集名稱/成員的格式添加；
到此分片集群就配置好了
查看sharding 集群狀態
mongos> sh.status()
--- Sharding Status ---
sharding version: {
"_id" : 1,
"minCompatibleVersion" : 5,
"currentVersion" : 6,
"clusterId" : ObjectId("5fac01dd8d6fa3fe899662c8")
}
shards:
{ "_id" : "shard1_replset", "host" : "shard1_replset/node05:27018,node06:27018,node07:27018", "state" : 1 }
{ "_id" : "shard2_replset", "host" : "shard2_replset/node08:27018,node09:27018,node10:27018", "state" : 1 }
active mongoses:
"4.4.1" : 1
autosplit:
Currently enabled: yes
balancer:
Currently enabled: yes
Currently running: yes
Collections with active migrations:
config.system.sessions started at Wed Nov 11 2020 23:43:14 GMT+0800 (CST)
Failed balancer rounds in last 5 attempts: 0
Migration Results for the last 24 hours:
45 : Success
databases:
{ "_id" : "config", "primary" : "config", "partitioned" : true }
config.system.sessions
shard key: { "_id" : 1 }
unique: false
balancing: true
chunks:
shard1_replset 978
shard2_replset 46
too many chunks to print, use verbose if you want to force print
mongos>
提示：可以看到當前分片集群中有兩個shard 副本集，分別是shard1_replset和shard2_replset；以及一個config server
對testdb數據庫啟用sharding功能
mongos> sh.enableSharding("testdb")
{
"ok" : 1,
"operationTime" : Timestamp(1605109993, 9),
"$clusterTime" : {
"clusterTime" : Timestamp(1605109993, 9),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
}
}
mongos> sh.status()
--- Sharding Status ---
sharding version: {
"_id" : 1,
"minCompatibleVersion" : 5,
"currentVersion" : 6,
"clusterId" : ObjectId("5fac01dd8d6fa3fe899662c8")
}
shards:
{ "_id" : "shard1_replset", "host" : "shard1_replset/node05:27018,node06:27018,node07:27018", "state" : 1 }
{ "_id" : "shard2_replset", "host" : "shard2_replset/node08:27018,node09:27018,node10:27018", "state" : 1 }
active mongoses:
"4.4.1" : 1
autosplit:
Currently enabled: yes
balancer:
Currently enabled: yes
Currently running: no
Failed balancer rounds in last 5 attempts: 0
Migration Results for the last 24 hours:
214 : Success
databases:
{ "_id" : "config", "primary" : "config", "partitioned" : true }
config.system.sessions
shard key: { "_id" : 1 }
unique: false
balancing: true
chunks:
shard1_replset 810
shard2_replset 214
too many chunks to print, use verbose if you want to force print
{ "_id" : "testdb", "primary" : "shard2_replset", "partitioned" : true, "version" : { "uuid" : UUID("454aad2e-b397-4c88-b5c4-c3b21d37e480"), "lastMod" : 1 } }
mongos>
提示：在對某個數據庫啟動sharding功能后，它會給我們分片一個主shard所謂主shard是用來存放該數據庫下沒有做分片的colleciton；對于分片的collection會分散在各個shard上；
啟用對testdb庫下的peoples集合啟動sharding，并指明在age字段上做基于范圍的分片
mongos> sh.shardCollection("testdb.peoples",{"age":1})
{
"collectionsharded" : "testdb.peoples",
"collectionUUID" : UUID("ec095411-240d-4484-b45d-b541c33c3975"),
"ok" : 1,
"operationTime" : Timestamp(1605110694, 11),
"$clusterTime" : {
"clusterTime" : Timestamp(1605110694, 11),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
}
}
mongos> sh.status()
--- Sharding Status ---
sharding version: {
"_id" : 1,
"minCompatibleVersion" : 5,
"currentVersion" : 6,
"clusterId" : ObjectId("5fac01dd8d6fa3fe899662c8")
}
shards:
{ "_id" : "shard1_replset", "host" : "shard1_replset/node05:27018,node06:27018,node07:27018", "state" : 1 }
{ "_id" : "shard2_replset", "host" : "shard2_replset/node08:27018,node09:27018,node10:27018", "state" : 1 }
active mongoses:
"4.4.1" : 1
autosplit:
Currently enabled: yes
balancer:
Currently enabled: yes
Currently running: no
Failed balancer rounds in last 5 attempts: 0
Migration Results for the last 24 hours:
408 : Success
databases:
{ "_id" : "config", "primary" : "config", "partitioned" : true }
config.system.sessions
shard key: { "_id" : 1 }
unique: false
balancing: true
chunks:
shard1_replset 616
shard2_replset 408
too many chunks to print, use verbose if you want to force print
{ "_id" : "testdb", "primary" : "shard2_replset", "partitioned" : true, "version" : { "uuid" : UUID("454aad2e-b397-4c88-b5c4-c3b21d37e480"), "lastMod" : 1 } }
testdb.peoples
shard key: { "age" : 1 }
unique: false
balancing: true
chunks:
shard2_replset 1
{ "age" : { "$minKey" : 1 } } -->> { "age" : { "$maxKey" : 1 } } on : shard2_replset Timestamp(1, 0)
mongos>
提示：如果對應的collection存在，我們還需要先對collection創建shardkey索引，然后在使用sh.shardCollection()來對colleciton啟用sharding功能；基于范圍做分片，我們可以在多個字段上做；
基于hash做分片
mongos> sh.shardCollection("testdb.peoples1",{"name":"hashed"})
{
"collectionsharded" : "testdb.peoples1",
"collectionUUID" : UUID("f6213da1-7c7d-4d5e-8fb1-fc554efb9df2"),
"ok" : 1,
"operationTime" : Timestamp(1605111014, 2),
"$clusterTime" : {
"clusterTime" : Timestamp(1605111014, 2),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
}
}
mongos> sh.status()
--- Sharding Status ---
sharding version: {
"_id" : 1,
"minCompatibleVersion" : 5,
"currentVersion" : 6,
"clusterId" : ObjectId("5fac01dd8d6fa3fe899662c8")
}
shards:
{ "_id" : "shard1_replset", "host" : "shard1_replset/node05:27018,node06:27018,node07:27018", "state" : 1 }
{ "_id" : "shard2_replset", "host" : "shard2_replset/node08:27018,node09:27018,node10:27018", "state" : 1 }
active mongoses:
"4.4.1" : 1
autosplit:
Currently enabled: yes
balancer:
Currently enabled: yes
Currently running: yes
Collections with active migrations:
config.system.sessions started at Thu Nov 12 2020 00:10:16 GMT+0800 (CST)
Failed balancer rounds in last 5 attempts: 0
Migration Results for the last 24 hours:
480 : Success
databases:
{ "_id" : "config", "primary" : "config", "partitioned" : true }
config.system.sessions
shard key: { "_id" : 1 }
unique: false
balancing: true
chunks:
shard1_replset 543
shard2_replset 481
too many chunks to print, use verbose if you want to force print
{ "_id" : "testdb", "primary" : "shard2_replset", "partitioned" : true, "version" : { "uuid" : UUID("454aad2e-b397-4c88-b5c4-c3b21d37e480"), "lastMod" : 1 } }
testdb.peoples
shard key: { "age" : 1 }
unique: false
balancing: true
chunks:
shard2_replset 1
{ "age" : { "$minKey" : 1 } } -->> { "age" : { "$maxKey" : 1 } } on : shard2_replset Timestamp(1, 0)
testdb.peoples1
shard key: { "name" : "hashed" }
unique: false
balancing: true
chunks:
shard1_replset 2
shard2_replset 2
{ "name" : { "$minKey" : 1 } } -->> { "name" : NumberLong("-4611686018427387902") } on : shard1_replset Timestamp(1, 0)
{ "name" : NumberLong("-4611686018427387902") } -->> { "name" : NumberLong(0) } on : shard1_replset Timestamp(1, 1)
{ "name" : NumberLong(0) } -->> { "name" : NumberLong("4611686018427387902") } on : shard2_replset Timestamp(1, 2)
{ "name" : NumberLong("4611686018427387902") } -->> { "name" : { "$maxKey" : 1 } } on : shard2_replset Timestamp(1, 3)
mongos>
提示：基于hash做分片只能在一個字段上做，不能指定多個字段；從上面的狀態信息可以看到testdb.peoples被分到了shard2上，peoples1一部分分到了shard1，一部分分到了shard2上；所以在peoples中插入多少條數據，它都會寫到shard2上，在peoples1中插入數據會被寫入到shard1和shard2上；
驗證：在peoples1 集合上插入數據，看看是否將數據分片到不同的shard上呢？
在mongos上插入數據
mongos> use testdb
switched to db testdb
mongos> for (i=1;i=10000;i++) db.peoples1.insert({name:"people"+i,age:(i%120),classes:(i%20)})
WriteResult({ "nInserted" : 1 })
mongos>
在shard1上查看數據
shard1_replset:PRIMARY> show dbs admin 0.000GB config 0.001GB local 0.001GB testdb 0.000GB shard1_replset:PRIMARY> use testdb switched to db testdb shard1_replset:PRIMARY> show tables peoples1 shard1_replset:PRIMARY> db.peoples1.find().count() 4966 shard1_replset:PRIMARY>
提示：在shard1上可以看到對應collection保存了4966條數據；
在shard2上查看數據
shard2_replset:PRIMARY> show dbs admin 0.000GB config 0.001GB local 0.011GB testdb 0.011GB shard2_replset:PRIMARY> use testdb switched to db testdb shard2_replset:PRIMARY> show tables peoples peoples1 shard2_replset:PRIMARY> db.peoples1.find().count() 5034 shard2_replset:PRIMARY>
提示：在shard2上可以看到有peoples集合和peoples1集合，其中peoples1集合保存了5034條數據；shard1和shard2總共就保存了我們剛才插入的10000條數據；
ok,到此mongodb的分片集群就搭建，測試完畢了；
到此這篇關于分布式文檔存儲數據庫之MongoDB分片集群的文章就介紹到這了,更多相關MongoDB分片集群內容請搜索腳本之家以前的文章或繼續瀏覽下面的相關文章希望大家以后多多支持腳本之家！
