Wednesday, April 16, 2025

MongoDB Sharding Setup


0) Preparation:

Create a directory for Config server:

mkdir D:\MongoDB\config

Start the config server:

mongod --configsvr --replSet configRs --bind_ip 192.168.56.1 port 27020 --dbpath D:\MongoDB\config --logpath D:\MongoDB\config\mongo.log --fork

Connect to the config server:

mongo --port 27020

1) Create replicaSet:

rs.initiate(

{

_id:"configRS",

configsvr:true,

members: [

{ _id:0,host:"192.168.56.1:27020"}

]

}

)

Check the status:

rs.status()

use mymongodb;

db.employees.insert(name:"Matt");

-- You will receive the message "can't create user databases on a --configsvr instance"

show dbs;


2) Configure sharded members:

a) Shard1:

mongod --bind_ip 192.168.56.1 port 27030 --replSet shardset1 --shardsvr --dbpath D:\MongoDB\rs1 --oplogSize 200 

--logpath D:\MongoDB\rs1\mongo.log --fork

connect to server:

mongo --port 27030


Initiate the replicaSet:

rs.initiate(

   {

_id:"shardset1",

members: [

   { _id:0,host:"192.168.56.1:27030"}

]

   }

)

b)  Shard2:

mongod --bind_ip 192.168.56.1 port 27040 --replSet shardset2 --shardsvr --dbpath D:\MongoDB\rs2 --oplogSize 200 --logpath D:\MongoDB\rs2\mongo.log --fork

connect to server

mongo --port 27040

Initiate the replicaSet:

rs.initiate(

   {

_id:"shardset2",

members: [

   { _id:0,host:"192.168.56.1:27040"}

]

   }

)

 c) Shard3:

mongod --bind_ip 192.168.56.1 port 27050 --replSet shardset3 --shardsvr --dbpath D:\MongoDB\rs3 --oplogSize 200 --logpath D:\MongoDB\rs3\mongo.log --fork

connect to server

mongo --port 27050

Initiate the replicaSet:

rs.initiate(

   {

_id:"shardset3",

members: [

   { _id:0,host:"192.168.56.1:27050"}

]

   }

)

3) Connect to the config server

mongo --port 27020

sh.status()  


Note the "shards:" portion, balancer , databases


4) Start the mongos process, its a routing process; Mongos Default port is 27017

mongos --configdb configRS/192.168.56.1:27020 --bind_ip 192.168.56.1 --logpath D:\MongoDB\mongos.log --fork

sh.status() # Note the "active mongoses:"

Then add the shards


5) Adding shards from Mongos Server (Should not be from Config server)

sh.addShard("shardset1/192.168.56.1:27030");

sh.addShard("shardset2/192.168.56.1:27040");

sh.addShard("shardset3/192.168.56.1:27050");


use mymongodb;

db.employees.insert(name:"Matt");


Enabling sharding:

sh.enableSharding("mymongodb");

sh.status() # Now you can see partitioned is true


Changing Chunk size for testing:

use config;

db.settings.save({_id: "chunksize", value: 1});


Insert data

for (var i=0;1<=100000;i++) {db.employees.insert({"empname": "Employee"+i, "created_at": new Date()});}

db.employees,createIndex({empname:1});

sh.shardCollection("mymongodb.employees",{empname:1});


To see sharding distribution info on a single collection:

db.employees.getShardDistribution()














Thursday, April 10, 2025

Cassandra and ScyllaDB

Simple comparison of Cassandra and ScyllaDB


FeatureCassandraScyllaDB
ImplementationJavaC++
PerformanceGood, but higher latenciesHigh performance, low latencies
Resource UtilizationRequires more nodesFewer nodes needed
AdministrationManual tuning requiredAutomated tuning
CompatibilityNativeAPI-compatible with Cassandra
Data ModelTables, rows, columnsTables, rows, columns
Query LanguageCQLCQL
Unique FeaturesStandard featuresMaterialized views, global secondary indexes
Use CasesHigh write throughput, scalabilityData-intensive apps, low latency

Reconfiguring MongoDB ReplicaSets

 Reconfiguring ReplicaSet:

For example, port has to change for one of a member.

1) Stop the member and restart using new port number

2) Modify the member with new port number

conf = rs.config()

conf.members[2].host="192.168.1.52:27019"

3) Apply the reconfiguration

rs.reconfig(conf)   or    rs.reconfig(conf, {force:true})  # Force option is only if it's really necessary

4) Verify the changes

rs.config() 

rs.status()

  Working with Databases Using SQLite in Python Introduction Databases are essential for storing, organizing, and retrieving data efficien...