Here is a step by step to set it up. I made it short and concise based on CFEngine Enterprise 3.5.3.
For a full version, please visit this online documentation from 10Gen.
http://docs.mongodb.org/manual/core/replication-introduction/
1. Prepare 3 machines
eg. (hub1, IP = 10.0.0.4), (hub2, IP = 10.0.0.5), (hub3, IP = 10.0.0.6)
2. Install CFEngine hub package on all 3 machines and all of them should have same key pairs and license file for our better life in key management
3. Set up mongodb replica set (do (a)(b) on all 3 machines. (c)(d)(e)(f)(g)(h) only on hub1 which is our primary in this case.)
a. In /var/cfengine/masterfiles/update/mongod.conf, uncomment the "# replSet = <replicaset-name>" and give appropriate name eg. "cfenginers0". Also we need to remove "bind_ip" for networking connections
# bind_ip 127.0.0.1
replSet = cfenginers0
b. Start mongod as follows:
/var/cfengine/bin/mongod --dbpath /var/cfengine/state --config /var/cfengine/masterfiles/update/mongod.conf
c. Connect to one of the mongd instances, lets say 10.0.0.4
/var/cfengine/bin/mongo
d. Initialize replication with default config - this also will add 10.0.0.4 to the replica set
rs.initiate()
e. Add the other two members ***
rs.add("10.0.0.5")
rs.add("10.0.0.6")
f. Allows the current connection to allow read operations to run on secondary members
rs.slaveOk()
g. Display the replica set configuration object
rs.conf()
h. Confirm successful replica set configuration after a while (less than a min)
rs.status()
4. Bootstrap each hub to itself
/var/cfenigne/bin/cf-agent -B 10.0.0.4
/var/cfengine/bin/cf-agent -B 10.0.0.5
/var/cfengine/bin/cf-agent -B 10.0.0.6
5. Bootstrap hub for clients is hub1 so /var/cfengine/ppkeys should be duplicated to hub2 and hub3 from time to time. Otherwise, clients cannot talk to both of them when CFEngien on hub1 failed. (assuming trustkey => "false"; in copy_from body).*** Note that before running rs.add() command, mongod processes should be running on secondary nodes