Create and assin a self-signed certificate with the bash script below:

- https://gist.github.com/alirezastack/30c8849e4add4329dcc2633fbb06a638

#mongodb #ssl #self_signed

Simple bash script to take nightly MongoDB backups:

#!/bin/sh
 DIR=`date +%m%d%y`
 DEST=/db_backups/$DIR
 mkdir $DEST
 mongodump -h <your_database_host> -d <your_database_name> -u <username> -p <password> -o $DEST

NOTE: db_backups folder shoud already be created by mkdir /db_backups.


Put it in a crontab for nightly backups. First open crotab:

sudo crontab -e

Create a new line (entry) in crontab and paste the below cron task:

45 1 * * * ../../scripts/db_backup.sh

NOTE: here our script is called db_backup.sh, should you use your own script name here. and make it executable by chmod +x /your/ full_path/scripts/db_backup.sh


#mongodb #backup #cron #cronjob #coderwall #mongodump #bash

How to ignore extra fields for schema validation in Mongoengine?

Some records currently have extra fields that are not included in my model schema (by error, but I want to handle these cases). When I try to query the DB and transform the records into the schema, I get the following error:

FieldDoesNotExist
The field 'X' does not exist on the document 'Y'

For ignoring this error when having extra fields while getting data, set strict to False in your meta dictionary.

class User(Document):
    email = StringField(required=True, unique=True)
    password = StringField()
    meta = {'strict': False}

#mongodb #mongo #python #mongoengine #strict #FieldDoesNotExist

In MongoDB you can remove duplicate documents based on a specific field:

db.yourCollection.aggregate([
     { "$group": {
         "_id": { "yourDuplicateKey": "$yourDuplicateKey" },
         "dups": { "$push": "$_id" },
         "count": { "$sum": 1 }
     }},
     { "$match": { "count": { "$gt": 1 } }}
 ]).forEach(function(doc) {
     doc.dups.shift();
     db.yourCollection.remove({ "_id": {"$in": doc.dups }});
 });

It uses aggregation to group by based on the given key then add its _id into dups field and its count in count field. It will project fields with count of more than 1 using $match. At the end loops over each document and remove all duplicate fields except the first one (`shift` will cause this behaviour).

#mongodb #mongo #duplicates #duplication

How do you query is not null in MongoDB?

db.mycollection.find({ 'fieldname' : { $ne: null } });

#database #mongodb #query #ne #null #not_null

How to configure a Delayed Replica Set Member?

Let's assume that our member is third in the array of replica members:

cfg = rs.conf()
 cfg.members[2].priority = 0
 cfg.members[2].hidden = true
 cfg.members[2].slaveDelay = 3600
 rs.reconfig(cfg)

The priority is set to 0 (preventing to be elected as primary).

The hidden to true in order to hide the node from clients querying the database.

And finally slaveDelay to number of seconds that we want it to get behind of Primary Node.

The use case for this is to have a replication that is used for analytical purposes or used for backup and so on.

#mongodb #mongo #replica #replication #primary #delayed_replica_set #slaveDelay

How to add self-signed certificates to replica set nodes?

https://medium.com/@rossbulat/deploy-a-3-node-mongodb-3-6-replica-set-with-x-509-authentication-self-signed-certificates-d539fda94db4

#mongo #mongodb #ssl #self_signed #openssl

In order to see how much time your mongoDB slave is behind the primary node:

rs0:SECONDARY> db.printSlaveReplicationInfo()
 source: mongo.mongo.com:27017
     syncedTo: Mon Nov 12 2018 06:33:40 GMT+0000 (UTC)
     -4 secs (0 hrs) behind the primary

#mongodb #mongo #slave #printSlaveReplicationInfo #replica #replication

How to check MongoDB replication lag in Icinga2 and get notified when it is over 15 seconds?

We assume here that you have a replica set in place. First download the python script for our nagios plugin:

cd /usr/lib/nagios/plugins
 git clone git://github.com/mzupan/nagios-plugin-mongodb.git

Now the Icinga2 part. You first need to create a command for replication lag check:

cd /etc/icinga2/conf.d/commands

Create a new file replication_lag.conf:

object CheckCommand "check_replication_lag" {
     import "plugin-check-command"
     command = [ PluginDir + "/nagios-plugin-mongodb/check_mongodb.py", "-A", "replication_lag" ]
     arguments = {
         "-H" = "$mongo_host$"
         "-P" = "$mongo_port$"
     }
 }

Create a new file in services folder called replication_lag.conf:

apply Service for (display_name => config in host.vars.replication) {
     import "generic-service"
     check_command = "check_replication_lag"
     vars += config
     assign where host.vars.replication
 }

This service gets enabled where it finds replication in host config. Now in secondary mongoDB hosts configuration add the below part:

vars.replication["Secondary DB"] = {
     mongo_host = "slave.example.com"
     mongo_port = 27017
 }

#sysadmin #icinga2 #mongodb #replication #replication_lag #nagios_plugin

https://blog.shippable.com/why-we-moved-from-nosql-mongodb-to-postgressql

#mongoDB #posgreSQL

In MongoDB you can compare one field to another using $expr:

db.users.find({ $expr: { $eq: ["$created_at", "$updated_at"] } })

Here we get users that their updated_at field is equal to created_at field, here it means that user has not yet updated his profile.

#mongodb #mongo #expr #find

MongoDB server Load Average: 0.5 (It can reach 16)
Database Size: 100GB (It is compressed in MySQL it reaches 300 GB in size!)
Req/Sec: 500

Our server seems hungry for more requests and more data.

#mongodb #mongo #awesomeness

In MongoDB you can use $regex in order to find something based on a regex pattern:

my_col.find({'name': { $regex: '^ali.*' } })

It will find all users that their names start with ali. Now let's say you want to search users based on their phone country code which has a + in its number like +98901.... You need to escape the + character but escape it twice:

my_col.find({'phone': { $regex: '^\\+98.*' } })

#mongoDB #pymongo #regex

Is there a way to create ObjectID from an INT in MongoDB?

import bson

def object_id_from_int(n):
    s = str(n)
    s = '0' * (24 - len(s)) + s
    return bson.ObjectId(s)

def int_from_object_id(obj):
    return int(str(obj))

n = 12345
obj = object_id_from_int(n)
n = int_from_object_id(obj)
print(repr(obj))  # ObjectId('000000000000000000012345')
print(n)  # 12345

#mongodb #objectid #pymongo #python #bson #int

https://medium.com/@xoor/indexing-mongodb-with-elasticsearch-2c428b676343

#mongodb #elastic #elasticsearch #indexing #fulltext #fulltext_search #search #ngram #medium

Did you know you can use jsonSchema in MongoDB to search for documents?

Let's say you have users collection with data below:

{ "_id" : ObjectId("5f64bd1eca8806f2c04fcbe3"), "customer_id" : 100, "username" : "john" }
{ "_id" : ObjectId("5f64bd1eca8806f2c04fcbe5"), "customer_id" : 206, "username" : "new_customer" }
{ "_id" : ObjectId("60420df441558d6671cf54f2"), "customer_id" : "123", "username" : "Ali" }

Now let's say you want to find all documents that has a customer_id of type string instead of int.
In Mongo shell:

let ms = {required: ["customer_id"], properties: {customer_id: {bsonType: "string"}}}

This schema says look for documents that have customer_id field with string type. To search:

> db.customers.find({$jsonSchema: ms})
{ "_id" : ObjectId("60420df441558d6671cf54f2"), "customer_id" : "123", "username" : "Ali" }

Interesting, right? :)

#database #mongodb #jsonSchema #json_schema