Did you know that the mongoDB shell is a full-featured JavaScript interpreter, capable of running arbitrary JavaScript programs? To illustrate this, let’s perform some basic math:

> x = 200
 200
 > x / 5;
 40

We can also leverage all of the standard JavaScript libraries:

> Math.sin(Math.PI / 2);
 1
 > new Date("2010/1/1");
 "Fri Jan 01 2010 00:00:00 GMT-0500 (EST)"
 > "Hello, World!".replace("World", "MongoDB"); Hello, MongoDB!

We can even define and call JavaScript functions. In previous posts we explained how to kill slow queries, there we explained how to define a function in MongoDB and store that function inside of MongoDB.


#mongodb #mongo #shell #javascript #js

You can read more about mongoDB functions here...

OperationalError: (2013, 'Lost connection to MySQL server during query')

Usually it indicates network connectivity trouble and you should check the condition of your network if this error occurs frequently. If the error message includes “during query,” this is probably the case you are experiencing.

Sometimes the “during query” form happens when millions of rows are being sent as part of one or more queries. If you know that this is happening, you should try increasing net_read_timeout from its default of 30 seconds to 60 seconds or longer, sufficient for the data transfer to complete.

More rarely, it can happen when the client is attempting the initial connection to the server. In this case, if your connect_timeout value is set to only a few seconds, you may be able to resolve the problem by increasing it to ten seconds, perhaps more if you have a very long distance or slow connection. You can determine whether you are experiencing this more uncommon cause by using SHOW GLOBAL STATUS LIKE 'Aborted_connects'. It will increase by one for each initial connection attempt that the server aborts. You may see “reading authorization packet” as part of the error message; if so, that also suggests that this is the solution that you need.

If the cause is none of those just described, you may be experiencing a problem with BLOB values that are larger than max_allowed_packet, which can cause this error with some clients. Sometime you may see an ER_NET_PACKET_TOO_LARGE error, and that confirms that you need to increase max_allowed_packet.

#database #mysql #OperationalError #connection

How to check MySQL version:

If you have local access to mySQL server you can check it by:

mysql -v

If you have remote access to MySQL you can first login to mysql using mysql -u USERNAME -p PASS -h REMOTE_HOST. And then after successful login issue the below command:

SELECT version();

#mysql #version

#programmer_jokes #fun

To get time in epoch in linux`/`OS X you can use date with formatting like below:

date +%s

The output of the command is similar to the following output:

1518517636

#linux #osx #date #epoch

دوستانی که در مورد تدریس خصوصی سوال کرده بودند، خدمتشون عارض هستم که تا آخر سال متاسفانه فرصتش وجود نداره و کمی برای آخر سال سرمون شلوغ هستش. ان شالله بعد از عید در صورتی که فشار کاری کمتر بشه خدمتتون میگم. ممنونم بابت پیام های امید بخشتون برای ادامه کار. 💐

#تدریس #خصوصی

In order to connect directly to MongoDB from your host:

mongo YOUR_REMOTE_MONGO_SERVER:27017

If your MongoDB port is different, use your desired port rather than 27017.


The interesting thing about this command is that you can give database name that you want to work on:

mongo YOUR_REMOTE_MONGO_SERVER:27017/YOUR_DB

Now after connecting if you use db command you should see your current db:

rs0:PRIMARY> db
 YOUR_DB

rs0:PRIMARY will be shown when you use replication. Your case may be different.

#database #mongodb #mongo

How to print a sentence when users on the server run mongo shell?

For DBAs to limit some dangerous functionalities like dropping a database or it can be a helpful message. Or a greeting message. Or even printing the default database that he is already connected to.

By default mongoDB looks for a file named .mongorc.js in your home directory. So create a file ~/.mongorc.js and put a content like below inside of it:

print("Hello! Welcome to Alireza company :)");
 print("Your database is set to: " + db);

That's it! Save the file and run mongo in your terminal, the output should be similar to the following:

$ mongo
 MongoDB shell version v3.6.2
 connecting to: mongodb://127.0.0.1:27017
 MongoDB server version: 3.6.2
 Your database is set to: test
 Hello! Welcome to Alireza company :)
 >

In order to disable dangerous functionalities:

var no = function() { print("oops! You are not allowed to drop anything!!");};

 // Prevent dropping databases
 db.dropDatabase = DB.prototype.dropDatabase = no;

 // Prevent dropping collections
 DBCollection.prototype.drop =no;

 // Prevent dropping indexes
 DBCollection.prototype.dropIndex = no;

Sample output:

> db.dropDatabase('bi')
 oops! You are not allowed to drop anything!!.

NOTE: You can disable loading your .mongorc.js by using the --norc option when starting the shell.

#mongodb #mongo #shell #mongorc

#Google on #valentine.

In order to connect to MongoDB replica set in Python you can give all server node addersses to MongoClient. Addresses passed to MongoClient() are called the seeds. As long as at least one of the seeds is online, MongoClient discovers all the members in the replica set, and determines which is the current primary and which are secondaries or arbiters.


Sample usages:

>>> MongoClient('localhost', replicaset='foo')
     MongoClient(host=['localhost:27017'], replicaset='foo', ...)
 >>> MongoClient('localhost:27018', replicaset='foo')
     MongoClient(['localhost:27018'], replicaset='foo', ...)
 >>> MongoClient('localhost', 27019, replicaset='foo')
     MongoClient(['localhost:27019'], replicaset='foo', ...)
 >>> MongoClient('mongodb://localhost:27017,localhost:27018/?replicaSet=foo')
     MongoClient(['localhost:27017', 'localhost:27018'], replicaset='foo', ...)

Read full details here:

- http://api.mongodb.com/python/current/examples/high_availability.html#connecting-to-a-replica-set


#database #mongodb #mongo #replica_set #replication #pymongo #arbiter #master #primary #slave

Secondary Reads

By default an instance of MongoClient sends queries to the primary member of the replica set. To use secondaries for queries we have to change the read preference:

>>> client = MongoClient(
 ...     'localhost:27017',
 ...     replicaSet='foo',
 ...     readPreference='secondaryPreferred')
 >>> client.read_preference
 SecondaryPreferred(tag_sets=None)

Now all queries will be sent to the secondary members of the set. If there are no secondary members the primary will be used as a fallback. If you have queries you would prefer to never send to the primary you can specify that using the secondary read preference.

#mongodb #replica_set #replication #secondary #slave #pymongo

If you are building a docker image using docker build and you want to force building the package use --no-cache:

docker build --no-cache

#docker #build #no_cache #cache #force

Today we had hundreds of connections to MongoDB that compared to previous days was so high and one of our modules initiating the connections every time a new request was initiated. To address the issue, I used a singleton design pattern to prevent creating many connections. Although pymongo has its own connection pool, we reduced the connections by 40 to 60!

Singleton design pattern makes sure to only create a new object from your class only ONCE:

class Singleton(object):
     _instance = None

     def __new__(class_, *args, **kwargs):
         if not isinstance(class_._instance, class_):
             class_._instance = object.__new__(class_, *args, **kwargs)

         return class_._instance

__new__ will be called automatically when a new instance of the class is initiated. Here we check whether _instance is set or not, in case it is set we will return the old instance and will not create a new object.

Now you just need to send Singleton as parent class to your classes:

class MongoConnnection(Singleton):
     def __init__(self):
         pass

     def get_connection(self):
         pass

Create as many object as you want from MongoConnnection. And now when you print the initiated class you will see one memory address not different ones.

#mongodb #mongo #singleton #design_pattern #__new__

How to determine fragmentation in MongoDB?

use mydb
 var s = db.my_collection.stats()
 var frag = s.storageSize / (s.size + s.totalIndexSize)

NOTE: A frag value larger than 1 indicates some level of fragmentation.


#mongodb #mongo #stats #fragmentation

http://learnmongodbthehardway.com/schema/introduction/

#mongodb #mongo #tutorial

In order to check the filesystem of the partition and where that partition maps to that address use:

mount -l

It will show for example that you have a partition that mounted by NFS or that a partition is ext4 and so on:

/dev/mapper/vg-var on /var type ext4 (rw,relatime,stripe=384,data=ordered)

#linux #mount #ext4 #nfs #filesystem #partition

For optimal performance in terms of the storage layer, use disks backed by RAID-10. RAID-5 and RAID-6 do not typically provide sufficient performance to support a MongoDB deployment.

Avoid RAID-0 with MongoDB deployments. While RAID-0 provides good write performance, it also provides limited availability and can lead to reduced performance on read operations, particularly when using Amazon’s EBS volumes.


#mongodb #mongo #disk #raid #SSD

How to detect unused indexes on MongoDB collections?

db.your_collection.aggregate([{$indexStats:{}}]).pretty()

ops displays operations per second on a specific index. If ops is very low compared to other indexes you can drop the index.

#mongodb #mongo #index #unused_indexes

#mongodb #backlog #road_map

The good news is MongoDB will support transactions from version 4.0 like any other DBMS systems like MySQL, Oracle,...