ITPub博客

首页 > 数据库 > NoSQL > mongodb 3.0.2与wiredTiger存储引擎安装测试

mongodb 3.0.2与wiredTiger存储引擎安装测试

原创 NoSQL 作者:天空的鹰 时间:2015-10-27 12:19:10 0 删除 编辑
mongodb3.0版本,除了之前默认的MMAP存储引擎,还新推出了wiredTiger存储引擎。
据官网测试报告称,mongodb 3.0版本使用wiredTiger存储引擎后,写性能能够提高7-10倍,数据压缩30%-80%,运维降低95%,这三个方面都还是很有吸引力的。
为了进行mongodb 2.6和mongodb 3.0的性能,先搭建一个mongodb 3.0环境,然后用YCSB工具进行测试。

mongodb 3.0.2 与wiredTiger的安装步骤如下:

1. 下载和安装mongodb 3.0软件
从mongodb官网下载mongodb 3.0的安装介质 mongodb-linux-x86_64-rhel62-3.0.2.tgz ,然后登陆的服务器上执行下面的命令安装:
cd /usr/local/
rz mongodb-linux-x86_64-rhel62-3.0.2.tgz
tar -zxvf mongodb-linux-x86_64-rhel62-3.0.2.tgz
ln -s mongodb-linux-x86_64-rhel62-3.0.2 mongodb30
chown -R mongodb:mongodb mongodb-linux-x86_64-rhel62-3.0.2 mongodb30


2. 准备mongodb实例目录和配置文件
创建mongodb实例所需目录:
rm -fR /data/mongodb30/
mkdir -p /data/mongodb30/{db,logs}
touch /data/mongodb30/logs/mongodb.log
chown -R mongodb:mongodb /data/mongodb30/


创建配置文件,配置启动参数,尤其注意添加wriedTiger存储引擎相关参数:
vim /data/mongodb30/mongodb30.conf


storageEngine = wiredTiger
wiredTigerCacheSizeGB = 2
syncdelay = 30
wiredTigerCollectionBlockCompressor = snappy
port=38019
dbpath=/data/mongodb30/db
oplogSize=2048
logpath=/data/mongodb30/logs/mongodb.log
logappend=true
fork=true
rest=true
journal = true


上面的参数中,标红的参数是wriedTiger相关参数,storageEngine 是设置存储引擎;wiredTigerCacheSizeGB 是设置mongodb存储引擎所用的内容,默认为系统内存的50%;syncdelay 是设置从内存同步到硬盘的时间间隔,默认为60秒,可以设置的少一些,在mongodb故障时,丢失的日志会少一些;wiredTigerCollectionBlockCompressor 是设定压缩策略 snappy 是一种压缩速度非常快的压缩策略。
由此可见,在mongodb 3.0中想要启用wriedTiger存储引擎,只需要增加该引擎的几个参数即可。


注:mongodb实例的启动,可以用命令行启动,也可以用配置文件启动;为了方便操作,并且可以保存启动的的参数,建议使用配置文件的方式启动mongodb实例。


修改实例目录和文件的属主为mongodb:
chown -R mongodb:mongodb /data/mongodb30/


3. 启动mongodb实例,并登录测试
使用mongod命令,以配置文件的方式启动mongodb实例:
/usr/local/mongodb30/bin/mongod --config /data/mongodb30/mongodb30.conf


确认启动后的进程状态:
ps -ef | grep mongo


使用mongo命令登录到mongodb实例中:
/usr/local/mongodb30/bin/mongo --port=38019



实际操作过程记录如下:


点击(此处)折叠或打开

  1. [root@test07 local]#
  2. [root@test07 local]# rm -fR /data/mongodb30/
  3. [root@test07 local]# mkdir -p /data/mongodb30/{db,logs}
  4. [root@test07 local]# touch /data/mongodb30/logs/mongodb.log
  5. [root@test07 local]# chown -R mongodb:mongodb /data/mongodb30/
  6. [root@test07 local]#
  7. [root@test07 local]# vim /data/mongodb30/mongodb30.conf
  8. [root@test07 local]#
  9. [root@test07 local]#
  10. [root@test07 local]# cat /data/mongodb30/mongodb30.conf
  11. storageEngine = wiredTiger
  12. wiredTigerCacheSizeGB = 2
  13. syncdelay = 30
  14. wiredTigerCollectionBlockCompressor = Snappy
  15. port=38019
  16. dbpath=/data/mongodb30/db
  17. oplogSize=2048
  18. logpath=/data/mongodb30/logs/mongodb.log
  19. logappend=true
  20. fork=true
  21. rest=true
  22. journal = true
  23. [root@test07 local]#
  24. [root@test07 local]# chown -R mongodb:mongodb /data/mongodb30/
  25. [root@test07 local]#
  26. [root@test07 local]# su - mongodb
  27. [mongodb@test07 ~]$
  28. [mongodb@test07 ~]$ /usr/local/mongodb30/bin/mongod --config /data/mongodb30/mongodb30.conf
  29. 2015-10-27T11:06:12.889+0800 F CONTROL Failed global initialization: BadValue storage.wiredTiger.collectionConfig.blockCompressor must be a string of the format: (none/snappy/zlib)
  30. [mongodb@test07 ~]$ sed -i 's/Snappy/snappy/g' /data/mongodb30/mongodb30.conf
  31. [mongodb@test07 ~]$
  32. [mongodb@test07 ~]$ /usr/local/mongodb30/bin/mongod --config /data/mongodb30/mongodb30.conf
  33. 2015-10-27T11:07:04.640+0800 I CONTROL ** WARNING: --rest is specified without --httpinterface,
  34. 2015-10-27T11:07:04.640+0800 I CONTROL ** enabling http interface
  35. about to fork child process, waiting until server is ready for connections.
  36. forked process: 3098
  37. child process started successfully, parent exiting
  38. [mongodb@test07 ~]$
  39. [mongodb@test07 ~]$ ps -ef | grep mysql
  40. mongodb 3344 3063 0 11:42 pts/0 00:00:00 grep mysql
  41. [mongodb@test07 ~]$
  42. [mongodb@test07 ~]$
  43. [mongodb@test07 ~]$ ps -ef|grep mongodb
  44. mongodb 2953 1 0 11:00 ? 00:00:16 /usr/local/mongodb26/bin/mongod --config /data/mongodb26/mongodb26.conf
  45. root 3062 2802 0 11:05 pts/0 00:00:00 su - mongodb
  46. mongodb 3063 3062 0 11:05 pts/0 00:00:00 -bash
  47. mongodb 3098 1 0 11:07 ? 00:00:13 /usr/local/mongodb30/bin/mongod --config /data/mongodb30/mongodb30.conf
  48. mongodb 3346 3063 0 11:42 pts/0 00:00:00 ps -ef
  49. mongodb 3347 3063 0 11:42 pts/0 00:00:00 grep mongodb
  50. [mongodb@test07 ~]$
  51. [mongodb@test07 ~]$
  52. [mongodb@test07 ~]$ /usr/local/mongodb30/bin/mongo --port=38019
  53. MongoDB shell version: 3.0.2
  54. connecting to: 127.0.0.1:38019/test
  55. Server has startup warnings:
  56. 2015-10-27T11:07:04.640+0800 I CONTROL ** WARNING: --rest is specified without --httpinterface,
  57. 2015-10-27T11:07:04.640+0800 I CONTROL ** enabling http interface
  58. >
  59. > show dbs;
  60. local 0.000GB
  61. > use admin;
  62. switched to db admin
  63. > show collections;
  64. >
  65. >
  66. > db.serverStatus()
  67. {
  68.         "host" : "test07:38019",
  69.         "version" : "3.0.2",
  70.         "process" : "mongod",
  71.         "pid" : NumberLong(3098),
  72.         "uptime" : 2697,
  73.         "uptimeMillis" : NumberLong(2697265),
  74.         "uptimeEstimate" : 2457,
  75.         "localTime" : ISODate("2015-10-27T03:52:01.900Z"),
  76.         "asserts" : {
  77.                 "regular" : 0,
  78.                 "warning" : 0,
  79.                 "msg" : 0,
  80.                 "user" : 0,
  81.                 "rollovers" : 0
  82.         },
  83.         "connections" : {
  84.                 "current" : 1,
  85.                 "available" : 818,
  86.                 "totalCreated" : NumberLong(2)
  87.         },
  88.         "cursors" : {
  89.                 "note" : "deprecated, use server status metrics",
  90.                 "clientCursors_size" : 0,
  91.                 "totalOpen" : 0,
  92.                 "pinned" : 0,
  93.                 "totalNoTimeout" : 0,
  94.                 "timedOut" : 0
  95.         },
  96.         "extra_info" : {
  97.                 "note" : "fields vary by platform",
  98.                 "heap_usage_bytes" : 31212656,
  99.                 "page_faults" : 0
  100.         },
  101.         "globalLock" : {
  102.                 "totalTime" : NumberLong("2697266000"),
  103.                 "currentQueue" : {
  104.                         "total" : 0,
  105.                         "readers" : 0,
  106.                         "writers" : 0
  107.                 },
  108.                 "activeClients" : {
  109.                         "total" : 8,
  110.                         "readers" : 0,
  111.                         "writers" : 0
  112.                 }
  113.         },
  114.         "locks" : {
  115.                 "Global" : {
  116.                         "acquireCount" : {
  117.                                 "r" : NumberLong(724),
  118.                                 "w" : NumberLong(2),
  119.                                 "W" : NumberLong(4)
  120.                         }
  121.                 },
  122.                 "Database" : {
  123.                         "acquireCount" : {
  124.                                 "r" : NumberLong(723),
  125.                                 "R" : NumberLong(1),
  126.                                 "W" : NumberLong(2)
  127.                         }
  128.                 },
  129.                 "Collection" : {
  130.                         "acquireCount" : {
  131.                                 "r" : NumberLong(722)
  132.                         }
  133.                 }
  134.         },
  135.         "network" : {
  136.                 "bytesIn" : 1371,
  137.                 "bytesOut" : 3116,
  138.                 "numRequests" : 18
  139.         },
  140.         "opcounters" : {
  141.                 "insert" : 0,
  142.                 "query" : 1,
  143.                 "update" : 0,
  144.                 "delete" : 0,
  145.                 "getmore" : 0,
  146.                 "command" : 19
  147.         },
  148.         "opcountersRepl" : {
  149.                 "insert" : 0,
  150.                 "query" : 0,
  151.                 "update" : 0,
  152.                 "delete" : 0,
  153.                 "getmore" : 0,
  154.                 "command" : 0
  155.         },
  156.         "storageEngine" : {
  157.                 "name" : "wiredTiger"
  158.         },
  159.         "wiredTiger" : {
  160.                 "uri" : "statistics:",
  161.                 "LSM" : {
  162.                         "sleep for LSM checkpoint throttle" : 0,
  163.                         "sleep for LSM merge throttle" : 0,
  164.                         "rows merged in an LSM tree" : 0,
  165.                         "application work units currently queued" : 0,
  166.                         "merge work units currently queued" : 0,
  167.                         "tree queue hit maximum" : 0,
  168.                         "switch work units currently queued" : 0,
  169.                         "tree maintenance operations scheduled" : 0,
  170.                         "tree maintenance operations discarded" : 0,
  171.                         "tree maintenance operations executed" : 0
  172.                 },
  173.                 "async" : {
  174.                         "number of allocation state races" : 0,
  175.                         "number of operation slots viewed for allocation" : 0,
  176.                         "current work queue length" : 0,
  177.                         "number of flush calls" : 0,
  178.                         "number of times operation allocation failed" : 0,
  179.                         "maximum work queue length" : 0,
  180.                         "number of times worker found no work" : 0,
  181.                         "total allocations" : 0,
  182.                         "total compact calls" : 0,
  183.                         "total insert calls" : 0,
  184.                         "total remove calls" : 0,
  185.                         "total search calls" : 0,
  186.                         "total update calls" : 0
  187.                 },
  188.                 "block-manager" : {
  189.                         "mapped bytes read" : 0,
  190.                         "bytes read" : 12288,
  191.                         "bytes written" : 126976,
  192.                         "mapped blocks read" : 0,
  193.                         "blocks pre-loaded" : 0,
  194.                         "blocks read" : 3,
  195.                         "blocks written" : 27
  196.                 },
  197.                 "cache" : {
  198.                         "tracked dirty bytes in the cache" : 0,
  199.                         "tracked bytes belonging to internal pages in the cache" : 1245,
  200.                         "bytes currently in the cache" : 14644,
  201.                         "tracked bytes belonging to leaf pages in the cache" : 2147482403,
  202.                         "maximum bytes configured" : 2147483648,
  203.                         "tracked bytes belonging to overflow pages in the cache" : 0,
  204.                         "bytes read into cache" : 0,
  205.                         "bytes written from cache" : 20178,
  206.                         "pages evicted by application threads" : 0,
  207.                         "checkpoint blocked page eviction" : 0,
  208.                         "unmodified pages evicted" : 0,
  209.                         "page split during eviction deepened the tree" : 0,
  210.                         "modified pages evicted" : 0,
  211.                         "pages selected for eviction unable to be evicted" : 0,
  212.                         "pages evicted because they exceeded the in-memory maximum" : 0,
  213.                         "pages evicted because they had chains of deleted items" : 0,
  214.                         "failed eviction of pages that exceeded the in-memory maximum" : 0,
  215.                         "hazard pointer blocked page eviction" : 0,
  216.                         "internal pages evicted" : 0,
  217.                         "maximum page size at eviction" : 0,
  218.                         "eviction server candidate queue empty when topping up" : 0,
  219.                         "eviction server candidate queue not empty when topping up" : 0,
  220.                         "eviction server evicting pages" : 0,
  221.                         "eviction server populating queue, but not evicting pages" : 0,
  222.                         "eviction server unable to reach eviction goal" : 0,
  223.                         "pages split during eviction" : 0,
  224.                         "pages walked for eviction" : 0,
  225.                         "eviction worker thread evicting pages" : 0,
  226.                         "in-memory page splits" : 0,
  227.                         "percentage overhead" : 8,
  228.                         "tracked dirty pages in the cache" : 0,
  229.                         "pages currently held in the cache" : 10,
  230.                         "pages read into cache" : 0,
  231.                         "pages written from cache" : 16
  232.                 },
  233.                 "connection" : {
  234.                         "pthread mutex condition wait calls" : 59275,
  235.                         "files currently open" : 7,
  236.                         "memory allocations" : 21765,
  237.                         "memory frees" : 19667,
  238.                         "memory re-allocations" : 64,
  239.                         "total read I/Os" : 8,
  240.                         "pthread mutex shared lock read-lock calls" : 1336,
  241.                         "pthread mutex shared lock write-lock calls" : 2958,
  242.                         "total write I/Os" : 141
  243.                 },
  244.                 "cursor" : {
  245.                         "cursor create calls" : 568,
  246.                         "cursor insert calls" : 24,
  247.                         "cursor next calls" : 12,
  248.                         "cursor prev calls" : 3,
  249.                         "cursor remove calls" : 0,
  250.                         "cursor reset calls" : 119,
  251.                         "cursor search calls" : 604,
  252.                         "cursor search near calls" : 3,
  253.                         "cursor update calls" : 0
  254.                 },
  255.                 "data-handle" : {
  256.                         "connection dhandles swept" : 0,
  257.                         "connection candidate referenced" : 0,
  258.                         "connection sweeps" : 269,
  259.                         "connection time-of-death sets" : 1,
  260.                         "session dhandles swept" : 0,
  261.                         "session sweep attempts" : 56
  262.                 },
  263.                 "log" : {
  264.                         "log buffer size increases" : 0,
  265.                         "total log buffer size" : 1048576,
  266.                         "log bytes of payload data" : 8051,
  267.                         "log bytes written" : 18560,
  268.                         "yields waiting for previous log file close" : 0,
  269.                         "total size of compressed records" : 5113,
  270.                         "total in-memory size of compressed records" : 9524,
  271.                         "log records too small to compress" : 91,
  272.                         "log records not compressed" : 5,
  273.                         "log records compressed" : 6,
  274.                         "maximum log file size" : 104857600,
  275.                         "pre-allocated log files prepared" : 1,
  276.                         "number of pre-allocated log files to create" : 1,
  277.                         "pre-allocated log files used" : 0,
  278.                         "log read operations" : 0,
  279.                         "log release advances write LSN" : 102,
  280.                         "records processed by log scan" : 0,
  281.                         "log scan records requiring two reads" : 0,
  282.                         "log scan operations" : 0,
  283.                         "consolidated slot closures" : 0,
  284.                         "logging bytes consolidated" : 0,
  285.                         "consolidated slot joins" : 0,
  286.                         "consolidated slot join races" : 0,
  287.                         "slots selected for switching that were unavailable" : 0,
  288.                         "record size exceeded maximum" : 0,
  289.                         "failed to find a slot large enough for record" : 0,
  290.                         "consolidated slot join transitions" : 0,
  291.                         "log sync operations" : 92,
  292.                         "log sync_dir operations" : 1,
  293.                         "log server thread advances write LSN" : 0,
  294.                         "log write operations" : 102
  295.                 },
  296.                 "reconciliation" : {
  297.                         "page reconciliation calls" : 16,
  298.                         "page reconciliation calls for eviction" : 0,
  299.                         "split bytes currently awaiting free" : 0,
  300.                         "split objects currently awaiting free" : 0
  301.                 },
  302.                 "session" : {
  303.                         "open cursor count" : 54,
  304.                         "open session count" : 59
  305.                 },
  306.                 "thread-yield" : {
  307.                         "page acquire busy blocked" : 0,
  308.                         "page acquire eviction blocked" : 0,
  309.                         "page acquire locked blocked" : 0,
  310.                         "page acquire read blocked" : 0,
  311.                         "page acquire time sleeping (usecs)" : 0
  312.                 },
  313.                 "transaction" : {
  314.                         "transaction begins" : 50,
  315.                         "transaction checkpoints" : 89,
  316.                         "transaction checkpoint generation" : 89,
  317.                         "transaction checkpoint currently running" : 0,
  318.                         "transaction checkpoint max time (msecs)" : 16,
  319.                         "transaction checkpoint min time (msecs)" : 0,
  320.                         "transaction checkpoint most recent time (msecs)" : 0,
  321.                         "transaction checkpoint total time (msecs)" : 52,
  322.                         "transactions committed" : 3,
  323.                         "transaction failures due to cache overflow" : 0,
  324.                         "transaction range of IDs currently pinned by a checkpoint" : 0,
  325.                         "transaction range of IDs currently pinned" : 1,
  326.                         "transactions rolled back" : 46
  327.                 },
  328.                 "concurrentTransactions" : {
  329.                         "write" : {
  330.                                 "out" : 0,
  331.                                 "available" : 128,
  332.                                 "totalTickets" : 128
  333.                         },
  334.                         "read" : {
  335.                                 "out" : 1,
  336.                                 "available" : 127,
  337.                                 "totalTickets" : 128
  338.                         }
  339.                 }
  340.         },
  341.         "writeBacksQueued" : false,
  342.         "mem" : {
  343.                 "bits" : 64,
  344.                 "resident" : 37,
  345.                 "virtual" : 339,
  346.                 "supported" : true,
  347.                 "mapped" : 0,
  348.                 "mappedWithJournal" : 0
  349.         },
  350.         "metrics" : {
  351.                 "commands" : {
  352.                         "getLog" : {
  353.                                 "failed" : NumberLong(0),
  354.                                 "total" : NumberLong(2)
  355.                         },
  356.                         "isMaster" : {
  357.                                 "failed" : NumberLong(0),
  358.                                 "total" : NumberLong(10)
  359.                         },
  360.                         "listCollections" : {
  361.                                 "failed" : NumberLong(0),
  362.                                 "total" : NumberLong(1)
  363.                         },
  364.                         "listDatabases" : {
  365.                                 "failed" : NumberLong(0),
  366.                                 "total" : NumberLong(1)
  367.                         },
  368.                         "replSetGetStatus" : {
  369.                                 "failed" : NumberLong(2),
  370.                                 "total" : NumberLong(2)
  371.                         },
  372.                         "serverStatus" : {
  373.                                 "failed" : NumberLong(0),
  374.                                 "total" : NumberLong(1)
  375.                         },
  376.                         "whatsmyuri" : {
  377.                                 "failed" : NumberLong(0),
  378.                                 "total" : NumberLong(2)
  379.                         }
  380.                 },
  381.                 "cursor" : {
  382.                         "timedOut" : NumberLong(0),
  383.                         "open" : {
  384.                                 "noTimeout" : NumberLong(0),
  385.                                 "pinned" : NumberLong(0),
  386.                                 "total" : NumberLong(0)
  387.                         }
  388.                 },
  389.                 "document" : {
  390.                         "deleted" : NumberLong(0),
  391.                         "inserted" : NumberLong(0),
  392.                         "returned" : NumberLong(0),
  393.                         "updated" : NumberLong(0)
  394.                 },
  395.                 "getLastError" : {
  396.                         "wtime" : {
  397.                                 "num" : 0,
  398.                                 "totalMillis" : 0
  399.                         },
  400.                         "wtimeouts" : NumberLong(0)
  401.                 },
  402.                 "operation" : {
  403.                         "fastmod" : NumberLong(0),
  404.                         "idhack" : NumberLong(0),
  405.                         "scanAndOrder" : NumberLong(0),
  406.                         "writeConflicts" : NumberLong(0)
  407.                 },
  408.                 "queryExecutor" : {
  409.                         "scanned" : NumberLong(0),
  410.                         "scannedObjects" : NumberLong(0)
  411.                 },
  412.                 "record" : {
  413.                         "moves" : NumberLong(0)
  414.                 },
  415.                 "repl" : {
  416.                         "apply" : {
  417.                                 "batches" : {
  418.                                         "num" : 0,
  419.                                         "totalMillis" : 0
  420.                                 },
  421.                                 "ops" : NumberLong(0)
  422.                         },
  423.                         "buffer" : {
  424.                                 "count" : NumberLong(0),
  425.                                 "maxSizeBytes" : 268435456,
  426.                                 "sizeBytes" : NumberLong(0)
  427.                         },
  428.                         "network" : {
  429.                                 "bytes" : NumberLong(0),
  430.                                 "getmores" : {
  431.                                         "num" : 0,
  432.                                         "totalMillis" : 0
  433.                                 },
  434.                                 "ops" : NumberLong(0),
  435.                                 "readersCreated" : NumberLong(0)
  436.                         },
  437.                         "preload" : {
  438.                                 "docs" : {
  439.                                         "num" : 0,
  440.                                         "totalMillis" : 0
  441.                                 },
  442.                                 "indexes" : {
  443.                                         "num" : 0,
  444.                                         "totalMillis" : 0
  445.                                 }
  446.                         }
  447.                 },
  448.                 "storage" : {
  449.                         "freelist" : {
  450.                                 "search" : {
  451.                                         "bucketExhausted" : NumberLong(0),
  452.                                         "requests" : NumberLong(0),
  453.                                         "scanned" : NumberLong(0)
  454.                                 }
  455.                         }
  456.                 },
  457.                 "ttl" : {
  458.                         "deletedDocuments" : NumberLong(0),
  459.                         "passes" : NumberLong(44)
  460.                 }
  461.         },
  462.         "ok" : 1
  463. }
  464. >

来自 “ ITPUB博客 ” ,链接:https://blog.itpub.net/24638123/viewspace-1816510/ ,如需转载,请注明出处,否则将追究法律责任。

请登录后发表评论 登录
全部评论
多年数据库运维和架构经验,对 MySQL、Oracle、PostgreSQL、Greenplum、MongoDB 等多种常见数据库有丰富的运维实践经验,擅长数据库架构设计、维护优化、数据流转、Shell 和 Python 开发;乐于技术交流,网名 yumushui 进行了大量的技术总结和思考分享。

注册时间:2015-05-21

  • 博文量
    20
  • 访问量
    52617