Multi-key Indexes and Arrays: MongoDB

We have learnt the basics of multi-key indexes in MongoDB. Lets look at an example to demonstrate the multi-key indexing on arrays.

arrays multi key index mongodb Multi key Indexes and Arrays: MongoDB

foo: database name
name: collection name

Insert a document

1
2
3
4
5
6
7
8
MongoDB shell version: 2.6.1
connecting to: test
 
> use foo
switched to db foo
 
> db.name.insert({a: 1, b: 2, c: 3});
WriteResult({ "nInserted" : 1 })

Here we insert {a: 1, b: 2, c: 3} into “name” collection.

Multi-key Indexes and Arrays: MongoDB


You need to a flashplayer enabled browser to view this YouTube video

YouTube Link: http://www.youtube.com/watch?v=VGHSmjVmnzs [Watch the Video In Full Screen.]



Basic Cursor

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
> db.name.find({a: 1, b: 2})
{ "_id" : ObjectId("53d8982b79142c385cddc607"), "a" : 1, "b" : 2, "c" : 3 }
 
> db.name.find({a: 1, b: 2}).explain()
{
        "cursor" : "BasicCursor",
        "isMultiKey" : false,
        "n" : 1,
        "nscannedObjects" : 1,
        "nscanned" : 1,
        "nscannedObjectsAllPlans" : 1,
        "nscannedAllPlans" : 1,
        "scanAndOrder" : false,
        "indexOnly" : false,
        "nYields" : 0,
        "nChunkSkips" : 0,
        "millis" : 0,
        "server" : "Satish-PC:27017",
        "filterSet" : false
}

We find() the document using fields “a” and “b” and the query/command returns a basic cursor, as we do not have indexing on them.

Related Read: index creation: MongoDB

Lets create index on a and b

1
2
3
4
5
6
7
> db.name.ensureIndex({a: 1, b: 1});
{
        "createdCollectionAutomatically" : false,
        "numIndexesBefore" : 1,
        "numIndexesAfter" : 2,
        "ok" : 1
}

Previous there was only 1 index i.e., on “_id” Now there are 2 indexes – “_id” and “{a: 1, b: 1}”

Btree Cursor with multi-key as false

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
> db.name.find({a: 1, b: 2}).explain()
{
        "cursor" : "BtreeCursor a_1_b_1",
        "isMultiKey" : false,
        "n" : 1,
        "nscannedObjects" : 1,
        "nscanned" : 1,
        "nscannedObjectsAllPlans" : 1,
        "nscannedAllPlans" : 1,
        "scanAndOrder" : false,
        "indexOnly" : false,
        "nYields" : 0,
        "nChunkSkips" : 0,
        "millis" : 0,
        "indexBounds" : {
                "a" : [
                        [
                                1,
                                1
                        ]
                ],
                "b" : [
                        [
                                2,
                                2
                        ]
                ]
        },
        "server" : "Satish-PC:27017",
        "filterSet" : false
}

After creating the index on “a” and “b”, chain explain() method on the same command, and it shows you that, now it returns a Btree Cursor.

Lets insert another document

1
2
> db.name.insert({a: [0, 1, 2], b: 2, c: 3});
WriteResult({ "nInserted" : 1 })

Lets insert an array as value to field “a” and scalar values to “b” and “c”.

Btree Cursor with Multi-key true

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
> db.name.find({a: 1, b: 2})
{ "_id" : ObjectId("53d8982b79142c385cddc607"), 
  "a" : 1, "b" : 2, "c" : 3 }
{ "_id" : ObjectId("53d8986f79142c385cddc608"), 
  "a" : [ 0, 1, 2 ], "b" : 2, "c": 3 }
 
> db.name.find({a: 1, b: 2}).explain()
{
        "cursor" : "BtreeCursor a_1_b_1",
        "isMultiKey" : true,
        "n" : 2,
        "nscannedObjects" : 2,
        "nscanned" : 2,
        "nscannedObjectsAllPlans" : 2,
        "nscannedAllPlans" : 2,
        "scanAndOrder" : false,
        "indexOnly" : false,
        "nYields" : 0,
        "nChunkSkips" : 0,
        "millis" : 0,
        "indexBounds" : {
                "a" : [
                        [
                                1,
                                1
                        ]
                ],
                "b" : [
                        [
                                2,
                                2
                        ]
                ]
        },
        "server" : "Satish-PC:27017",
        "filterSet" : false
}

Now append explain() method to our command, it shows us that it returns a Btree Cursor and multi-key as true. MongoDB engine need to match every element of the array present in field “a” with the scalar value of field “b”. Hence it uses Multi-Key indexing.

Multi-Key Condition in MongoDB

1
2
3
4
5
6
7
8
9
> db.name.insert({a: [0, 1, 2], b: [3, 4], c: 3});
WriteResult({
        "nInserted" : 0,
        "writeError" : {
                "code" : 10088,
                "errmsg" : "insertDocument :: caused by :: 10088 cannot 
                            index parallel arrays [b] [a]"
        }
})

It’s difficult to match every combination of the array elements present inside both “a” and “b” fields. If both keys/indexes has its value as an array, then it gets complicated. Thus, mongoDB doesn’t allow both keys to be arrays. Either one of them must be a scalar value.

Get Index and Delete Index: MongoDB

We learnt the uses of having an index/key on our collection and how to create the index. Now, in this video tutorial lets learn how to get index on individual collection and how to drop / remove / delete the index we’ve created.

getIndex dropIndex mongodb Get Index and Delete Index: MongoDB

Related Read: index creation: MongoDB

temp: Database name
no, another: collection names
We’ve 10 Million documents inside “no” collection.

Sample document

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
> use temp
switched to db temp
> show collections
no
system.indexes
 
> db.no.find({"student_id": {$lt: 3}}).pretty()
{
        "_id" : ObjectId("53c9020abcdd1ea7fb833cda"),
        "student_id" : 0,
        "name" : "Satish"
}
{
        "_id" : ObjectId("53c9020abcdd1ea7fb833cdb"),
        "student_id" : 1,
        "name" : "Satish"
}
{
        "_id" : ObjectId("53c9020abcdd1ea7fb833cdc"),
        "student_id" : 2,
        "name" : "Satish"
}

Fetch Index and Drop / remove Index: MongoDB


You need to a flashplayer enabled browser to view this YouTube video

YouTube Link: http://www.youtube.com/watch?v=qYHIRWHS_5I [Watch the Video In Full Screen.]



We shall take a look at “system.indexes” collection

1
2
3
4
5
> db.system.indexes.find()
{ "v" : 1, "key" : { "_id" : 1 }, 
           "name" : "_id_", "ns" : "temp.no" }
{ "v" : 1, "key" : { "student_id" : 1 }, 
           "name" : "student_id_1", "ns" : "temp.no" }

Create “another” collection

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
> db.another.insert({"name": "Satish", "age": 27});
WriteResult({ "nInserted" : 1 })
 
> show collections
another
no
system.indexes
 
> db.system.indexes.find()
{ "v" : 1, "key" : { "_id" : 1 }, 
           "name" : "_id_", "ns" : "temp.no" }
{ "v" : 1, "key" : { "student_id" : 1 }, 
           "name" : "student_id_1", "ns" : "temp.no" }
{ "v" : 1, "key" : { "_id" : 1 }, 
           "name" : "_id_", "ns" : "temp.another" }

After creating “another” collection, mongoDB engine generates default key on its “_id” field. And “system.indexes” shows all the keys present inside the database for all the collections it has. This can get messy if we have large number of collections – which we do in even slightly bigger projects.

To get index on individual collection

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
> db.another.getIndexes()
[
        {
                "v" : 1,
                "key" : {
                        "_id" : 1
                },
                "name" : "_id_",
                "ns" : "temp.another"
        }
]
 
> db.no.getIndexes()
[
        {
                "v" : 1,
                "key" : {
                        "_id" : 1
                },
                "name" : "_id_",
                "ns" : "temp.no"
        },
        {
                "v" : 1,
                "key" : {
                        "student_id" : 1
                },
                "name" : "student_id_1",
                "ns" : "temp.no"
        }
]

We can make use of getIndex() method to fetch or get indexes / keys present on individual collection.

Removing / deleting / dropping – index / key

1
2
3
4
5
6
7
8
9
10
11
12
13
14
> db.no.dropIndex({"student_id": 1});
{ "nIndexesWas" : 2, "ok" : 1 }
 
> db.no.getIndexes()
[
        {
                "v" : 1,
                "key" : {
                        "_id" : 1
                },
                "name" : "_id_",
                "ns" : "temp.no"
        }
]

make use of dropIndex() method and pass-in the index object similar to that used while creating the index. This shall drop the index.

index creation: MongoDB

Lets learn to create index and to optimize the database in MongoDB.

Creating “Database”: “temp”, “Collection”: “no”, and inserting 10 Million documents inside it

1
2
3
4
5
use temp
switched to db temp
 
for(i=0; i< = 10000000; i++)
db.no.insert({"student_id": i, "name": "Satish"});

Since Mongo Shell is built out of JavaScript, you can pass in any valid Javascript code to it. So we write a for loop and insert 10 Million documents inside “no” collection.

creating index mongodb index creation: MongoDB

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
MongoDB shell version: 2.6.1
connecting to: test
> show dbs
admin    (empty)
daily    0.078GB
local    0.078GB
nesting  0.078GB
school   0.078GB
temp     3.952GB
test     0.078GB
> use temp
switched to db temp
> show collections
no
system.indexes
> db.no.find().pretty()
{
        "_id" : ObjectId("53c9020abcdd1ea7fb833cda"),
        "student_id" : 0,
        "name" : "Satish"
}
{
        "_id" : ObjectId("53c9020abcdd1ea7fb833cdb"),
        "student_id" : 1,
        "name" : "Satish"
}
{
        "_id" : ObjectId("53c9020abcdd1ea7fb833cdc"),
        "student_id" : 2,
        "name" : "Satish"
}
{
        "_id" : ObjectId("53c9020abcdd1ea7fb833cdd"),
        "student_id" : 3,
        "name" : "Satish"
}
{
        "_id" : ObjectId("53c9020abcdd1ea7fb833cde"),
        "student_id" : 4,
        "name" : "Satish"
}
{
        "_id" : ObjectId("53c9020abcdd1ea7fb833cdf"),
        "student_id" : 5,
        "name" : "Satish"
}
{
        "_id" : ObjectId("53c9020abcdd1ea7fb833ce0"),
        "student_id" : 6,
        "name" : "Satish"
}
{
        "_id" : ObjectId("53c9020abcdd1ea7fb833ce1"),
        "student_id" : 7,
        "name" : "Satish"
}
{
        "_id" : ObjectId("53c9020abcdd1ea7fb833ce2"),
        "student_id" : 8,
        "name" : "Satish"
}
{
        "_id" : ObjectId("53c9020abcdd1ea7fb833ce3"),
        "student_id" : 9,
        "name" : "Satish"
}
{
        "_id" : ObjectId("53c9020abcdd1ea7fb833ce4"),
        "student_id" : 10,
        "name" : "Satish"
}
{
        "_id" : ObjectId("53c9020abcdd1ea7fb833ce5"),
        "student_id" : 11,
        "name" : "Satish"
}
{
        "_id" : ObjectId("53c9020abcdd1ea7fb833ce6"),
        "student_id" : 12,
        "name" : "Satish"
}
{
        "_id" : ObjectId("53c9020abcdd1ea7fb833ce7"),
        "student_id" : 13,
        "name" : "Satish"
}
{
        "_id" : ObjectId("53c9020abcdd1ea7fb833ce8"),
        "student_id" : 14,
        "name" : "Satish"
}
{
        "_id" : ObjectId("53c9020abcdd1ea7fb833ce9"),
        "student_id" : 15,
        "name" : "Satish"
}
{
        "_id" : ObjectId("53c9020abcdd1ea7fb833cea"),
        "student_id" : 16,
        "name" : "Satish"
}
{
        "_id" : ObjectId("53c9020abcdd1ea7fb833ceb"),
        "student_id" : 17,
        "name" : "Satish"
}
{
        "_id" : ObjectId("53c9020abcdd1ea7fb833cec"),
        "student_id" : 18,
        "name" : "Satish"
}
{
        "_id" : ObjectId("53c9020abcdd1ea7fb833ced"),
        "student_id" : 19,
        "name" : "Satish"
}
Type "it" for more
 
> it

"no" collection has 10 Million record, but it won't fetch you all records at once, as it would take a lot of time and resources of your computer! So it only fetches 20 records at a time. You can iterate through next 20 documents by using command "it".

index creation: MongoDB


You need to a flashplayer enabled browser to view this YouTube video

YouTube Link: http://www.youtube.com/watch?v=zK_mRyiNs-I [Watch the Video In Full Screen.]



1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
> db.no.find({"student_id": 5}).pretty()
{
        "_id" : ObjectId("53c9020abcdd1ea7fb833cdf"),
        "student_id" : 5,
        "name" : "Satish"
}
 
 
> db.no.findOne({"student_id": 5});
{
        "_id" : ObjectId("53c9020abcdd1ea7fb833cdf"),
        "student_id" : 5,
        "name" : "Satish"
}
> db.no.find({"student_id": 5000000}).pretty()
{
        "_id" : ObjectId("53c90ca6bcdd1ea7fbcf881a"),
        "student_id" : 5000000,
        "name" : "Satish"
}

find() method scans through all the documents present in the collection to find multiple matches for the condition. So in above case, find() method scans through 10 Million documents, hence returns the result slowly. Where as findOne() method stops scanning the collection as soon as it finds the first matching document, so findOne() returns result faster than find() method.

Related Read:
Multi-key Index: MongoDB
index / key: MongoDB

Creating index

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
> show collections
no
system.indexes
 
> db.system.indexes.find()
{ "v" : 1, "key" : { "_id" : 1 }, "name" : "_id_", "ns" : "temp.no" }
 
> db.no.ensureIndex({"student_id": 1});
{
        "createdCollectionAutomatically" : false,
        "numIndexesBefore" : 1,
        "numIndexesAfter" : 2,
        "ok" : 1
}
 
> db.system.indexes.find()
{ "v" : 1, "key" : { "_id" : 1 }, 
                     "name" : "_id_", "ns" : "temp.no" }
{ "v" : 1, "key" : { "student_id" : 1 }, 
                     "name" : "student_id_1", "ns" : "temp.no" }

We create index on "student_id". It takes little time to create the index, as we have 10 Million documents inside "no" collection.

After creating index on "student_id", run the same command and you'll get the results instantly - maybe it takes 0.01 ms, but the delay can't be noticed.
Why does it return results faster after creating index on "student_id"? Watch this short video lesson to know it: index / key: MongoDB

1
2
3
4
5
6
7
8
9
10
11
12
13
> db.no.find({"student_id": 5000000}).pretty()
{
        "_id" : ObjectId("53c90ca6bcdd1ea7fbcf881a"),
        "student_id" : 5000000,
        "name" : "Satish"
}
> db.no.find({"student_id": 10000000}).pretty()
{
        "_id" : ObjectId("53c914adbcdd1ea7fb1bd35a"),
        "student_id" : 10000000,
        "name" : "Satish"
}
>

So the querys/commands can be optimized by creating indexes on frequently accessed fields.

index / key: MongoDB

Lets look at some basics of indexes in MongoDB.

If we have 3 fields in a document – name, age, sex
We could make name or age or sex or (name, age) or (name, age, sex) as index.

index key mongodb index / key: MongoDB

Assume that we make a index out of (name, age, sex)
In this case, we need to use the keys from left to right.

If we use “name” in our command, it makes use of the index.
If we use (“name”, “age”) in our command, it makes use of the index.
If we use (“name”, “age”, “sex”) in our command, it makes use of the index.

If we use “age” in our command, it can’t use the index for its operation.
If we use (“age”, “sex”) in our command, it can’t use the index for its operation.

If we use (“name”, “sex”) in our command, it simply uses “name” field and ignores “sex” field.

indexes / keys: MongoDB


You need to a flashplayer enabled browser to view this YouTube video

YouTube Link: http://www.youtube.com/watch?v=NiP-zX-jPaY [Watch the Video In Full Screen.]



For convenience, mongoDB adds “_id” field to each document inserted. “_id” is unique across the collection. And index is automatically created on “_id”.

Since index information is stored in “system.indexes” collection – it consumes disk too. So we need to make sure to add indexes to only those fields which we access frequently. Also note that, each time a document is inserted, “system.indexes” collection must be updated with the new index information, which takes time, bandwidth and disk space. So we need to be careful while creating indexes.

Multi-key Index: MongoDB

Lets learn about multikey indexes and how they are efficient in MongoDB.

multi key index mongodb Multi key Index: MongoDB

database name: daily
collections: gadgets, users

Insert documents into gadgets collection

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
> use daily
switched to db daily
 
> db.gadgets.insert({_id: 1, gadget: "Nexus 5"});
WriteResult({ "nInserted" : 1 })
 
> db.gadgets.insert({_id: 2, gadget: "iPhone"});
WriteResult({ "nInserted" : 1 })
 
> db.gadgets.insert({_id: 3, gadget: "iPad"});
WriteResult({ "nInserted" : 1 })
 
> db.gadgets.insert({_id: 4, gadget: "Nexus 7"});
WriteResult({ "nInserted" : 1 })
 
> db.gadgets.find()
{ "_id" : 1, "gadget" : "Nexus 5" }
{ "_id" : 2, "gadget" : "iPhone" }
{ "_id" : 3, "gadget" : "iPad" }
{ "_id" : 4, "gadget" : "Nexus 7" }

Here we have 4 documents with “_id” as 1, 2, 3, 4.

Insert documents into users collection

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
> db.users.insert({name: "Satish", 
                   products: [db.gadgets.find()[0]._id,
                              db.gadgets.find()[3]._id]});
WriteResult({ "nInserted" : 1 })
 
> db.users.insert({name: "Kiran", 
                   products: [db.gadgets.find()[0]._id, 
                              db.gadgets.find()[3]._id, 
                              db.gadgets.find()[2]._id]});
WriteResult({ "nInserted" : 1 })
 
> db.users.insert({name: "Jyothi", products: [1, 2]});
WriteResult({ "nInserted" : 1 })
 
> db.users.find().pretty()
{
        "_id" : ObjectId("53c7a30efd7d3c9597ca2593"),
        "name" : "Satish",
        "products" : [
                1,
                4
        ]
}
{
        "_id" : ObjectId("53c7a337fd7d3c9597ca2594"),
        "name" : "Kiran",
        "products" : [
                1,
                4,
                3
        ]
}
{
        "_id" : ObjectId("53c7a34efd7d3c9597ca2595"),
        "name" : "Jyothi",
        "products" : [
                1,
                2
        ]
}

Here we insert documents into “users” collection and embed the “_id” value of “gadgets” collection as array elements of “products” key.

Multi-key Index: MongoDB


You need to a flashplayer enabled browser to view this YouTube video

YouTube Link: http://www.youtube.com/watch?v=9C1x-xrN_qQ [Watch the Video In Full Screen.]



Command to fetch documents with array element 1 and 4

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
> db.users.find({products: {$all: [1, 4]}}).pretty();
{
        "_id" : ObjectId("53c7a30efd7d3c9597ca2593"),
        "name" : "Satish",
        "products" : [
                1,
                4
        ]
}
{
        "_id" : ObjectId("53c7a337fd7d3c9597ca2594"),
        "name" : "Kiran",
        "products" : [
                1,
                4,
                3
        ]
}

Both these documents have array values 1 and 4 in them.

Related Read: Comparison Operators: MongoDB

system.indexes content

1
2
3
4
5
6
7
8
> show collections
gadgets
system.indexes
users
 
> db.system.indexes.find()
{ "v" : 1, "key" : { "_id" : 1 }, "name" : "_id_", "ns" : "daily.gadgets" }
{ "v" : 1, "key" : { "_id" : 1 }, "name" : "_id_", "ns" : "daily.users" }

Here we have only 1 key and it’s on “_id”.

Basic Cursor

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
> db.users.find({products: {$all: [1, 4]}}).explain();
{
        "cursor" : "BasicCursor",
        "isMultiKey" : false,
        "n" : 2,
        "nscannedObjects" : 3,
        "nscanned" : 3,
        "nscannedObjectsAllPlans" : 3,
        "nscannedAllPlans" : 3,
        "scanAndOrder" : false,
        "indexOnly" : false,
        "nYields" : 0,
        "nChunkSkips" : 0,
        "millis" : 0,
        "server" : "Satish-PC:27017",
        "filterSet" : false
}

If we chain explain() method to our command, we can know some details about the command and also evaluate about its efficiency. The above command doesn’t have multi-key enabled and it’s a Basic Cursor.

Lets create index on products field

1
2
3
4
5
6
7
> db.users.ensureIndex({"products": 1});
{
        "createdCollectionAutomatically" : false,
        "numIndexesBefore" : 1,
        "numIndexesAfter" : 2,
        "ok" : 1
}

This creates index on field “products”.

Now the system.indexes content

1
2
3
4
5
6
> db.system.indexes.find()
{ "v" : 1, "key" : { "_id" : 1 }, "name" : "_id_", "ns" : "daily.gadgets" }
{ "v" : 1, "key" : { "_id" : 1 }, "name" : "_id_", "ns" : "daily.users" }
{ "v" : 1, "key" : { "products" : 1 }, 
                     "name" : "products_1", 
                     "ns" : "daily.users" }

So now we have 2 keys, “_id” and “products”.

BTree Cursor

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
> db.users.find({products: {$all: [1, 4]}}).explain();
{
        "cursor" : "BtreeCursor products_1",
        "isMultiKey" : true,
        "n" : 2,
        "nscannedObjects" : 3,
        "nscanned" : 3,
        "nscannedObjectsAllPlans" : 3,
        "nscannedAllPlans" : 8,
        "scanAndOrder" : false,
        "indexOnly" : false,
        "nYields" : 0,
        "nChunkSkips" : 0,
        "millis" : 0,
        "indexBounds" : {
                "products" : [
                        [
                                1,
                                1
                        ]
                ]
        },
        "server" : "Satish-PC:27017",
        "filterSet" : false
}

Now if we run the same command chained with explain() method, we could see that its a Btree Cursor and multi-key index is true.

Note: Btree Cursors are faster and efficient than Basic Cursors.

Removing Documents: MongoDB

Lets learn how to remove documents from the collection using remove() and drop() methods.

remove drop mongodb Removing Documents: MongoDB

test database, names collection

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
> db.names.find().pretty()
{
        "_id" : ObjectId("53c6392a2eea8062e084cb57"),
        "Company" : "Google",
        "Product" : "Nexus",
        "No" : 1
}
{
        "_id" : ObjectId("53c639392eea8062e084cb58"),
        "Company" : "Apple",
        "Product" : "Mac",
        "No" : 2
}
{
        "_id" : ObjectId("53c63b26b003603dfdcf8c52"),
        "Company" : "Xiaomi",
        "Product" : "Mi3",
        "No" : 3
}
{
        "_id" : ObjectId("53c63bd1b003603dfdcf8c53"),
        "Product" : "Smart Watch",
        "No" : 4,
        "Company" : "Sony"
}

We have 4 documents in “names” collection.

remove() method, with simple condition

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
> db.names.remove({"No": 4});
WriteResult({ "nRemoved" : 1 })
 
> db.names.find().pretty()
{
        "_id" : ObjectId("53c6392a2eea8062e084cb57"),
        "Company" : "Google",
        "Product" : "Nexus",
        "No" : 1,
        "IT" : "true"
}
{
        "_id" : ObjectId("53c639392eea8062e084cb58"),
        "Company" : "Apple",
        "Product" : "Mac",
        "No" : 2,
        "IT" : "true"
}
{
        "_id" : ObjectId("53c63b26b003603dfdcf8c52"),
        "Company" : "Xiaomi",
        "Product" : "Mi3",
        "No" : 3,
        "IT" : "true"
}

Here the document with “No: 4″ was removed from the collection.

Removing Documents: MongoDB


You need to a flashplayer enabled browser to view this YouTube video

YouTube Link: http://www.youtube.com/watch?v=so0_5Tdb_bQ [Watch the Video In Full Screen.]



remove() method, with comparison condition

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
> db.names.remove({"No": {$gt: 2}});
WriteResult({ "nRemoved" : 1 })
 
> db.names.find().pretty()
{
        "_id" : ObjectId("53c6392a2eea8062e084cb57"),
        "Company" : "Google",
        "Product" : "Nexus",
        "No" : 1,
        "IT" : "true"
}
{
        "_id" : ObjectId("53c639392eea8062e084cb58"),
        "Company" : "Apple",
        "Product" : "Mac",
        "No" : 2,
        "IT" : "true"
}

Here whatever documents which has “No” greater than 2 got removed.

remove() method to remove all documents from the collection

1
2
3
4
5
6
7
8
9
10
11
> db.names.remove({});
WriteResult({ "nRemoved" : 2 })
 
> db.names.find().pretty()
 
> show collections
names
system.indexes
 
> db.system.indexes.find().pretty()
{ "v" : 1, "key" : { "_id" : 1 }, "name" : "_id_", "ns" : "test.names" }

If we pass empty argument to remove() method, it matches with all the documents present in the collection, hence removes all the documents one-by-one.

But it doesn’t remove the contents/documents/index present in “system.indexes” collection.

drop() method

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
> db.names.find()
 
> db.names.insert({"Company": "Apple", "Product": "iPhone", "No": 1});
WriteResult({ "nInserted" : 1 })
 
> db.names.insert({"Company": "Google", "Product": "Nexus", "No": 2});
WriteResult({ "nInserted" : 1 })
 
> db.names.find().pretty()
{
        "_id" : ObjectId("53c6c5d95879b1ff1f0b8356"),
        "Company" : "Apple",
        "Product" : "iPhone",
        "No" : 1
}
{
        "_id" : ObjectId("53c6392a2eea8062e084cb57"),
        "Company" : "Google",
        "Product" : "Nexus",
        "No" : 1,
        "IT" : "true"
}
 
> db.names.drop();
true
 
> db.names.find().pretty()
 
> db.system.indexes.find().pretty()

Since “names” collection was empty, we inserted 2 documents into it. Now we applied drop() method on the collection, which drops all the document present in the collections at once. It also removes the document/index/content present inside “system.indexes” collection.

Note: If you want to remove/drop all the documents present inside the collection, make use of drop() method, as it removes all the documents at once, its more efficient than remove({}) method which removes documents one by one. Use remove() method, when you want to remove one or a set of documents from the collection.

Multi-Update: MongoDB

Lets learn how to update all the documents present in a collection using update() method, using the option multi: true

multi update mongodb Multi Update: MongoDB

test database, names collection

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
> db.names.find().pretty()
{
        "_id" : ObjectId("53c6392a2eea8062e084cb57"),
        "Company" : "Google",
        "Product" : "Nexus",
        "No" : 1
}
{
        "_id" : ObjectId("53c639392eea8062e084cb58"),
        "Company" : "Apple",
        "Product" : "Mac",
        "No" : 2
}
{
        "_id" : ObjectId("53c63b26b003603dfdcf8c52"),
        "Company" : "Xiaomi",
        "Product" : "Mi3",
        "No" : 3
}
{
        "_id" : ObjectId("53c63bd1b003603dfdcf8c53"),
        "Product" : "Smart Watch",
        "No" : 4,
        "Company" : "Sony"
}

We have 4 documents in “names” collection.

find() method

1
2
3
4
5
> db.names.find({}, {"Company": 1, "_id": 0}).pretty()
{ "Company" : "Google" }
{ "Company" : "Apple" }
{ "Company" : "Xiaomi" }
{ "Company" : "Sony" }

If we leave the first argument of find() method empty, it matches with all the documents of the collection. Hence it fetched all the Company names from the documents.

Multi-Update: MongoDB


You need to a flashplayer enabled browser to view this YouTube video

YouTube Link: http://www.youtube.com/watch?v=yuVBs4hcYOQ [Watch the Video In Full Screen.]



update() method

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
> db.names.update({}, {$set: {"IT": "true"}});
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
 
> db.names.find().pretty()
{
        "_id" : ObjectId("53c6392a2eea8062e084cb57"),
        "Company" : "Google",
        "Product" : "Nexus",
        "No" : 1,
        "IT" : "true"
}
{
        "_id" : ObjectId("53c639392eea8062e084cb58"),
        "Company" : "Apple",
        "Product" : "Mac",
        "No" : 2
}
{
        "_id" : ObjectId("53c63b26b003603dfdcf8c52"),
        "Company" : "Xiaomi",
        "Product" : "Mi3",
        "No" : 3
}
{
        "_id" : ObjectId("53c63bd1b003603dfdcf8c53"),
        "Product" : "Smart Watch",
        "No" : 4,
        "Company" : "Sony"
}

But in case of update() method, if the first argument is left empty, it randomly matches to only 1 document in the collection.

Related Read: Update with SET Operator: MongoDB

unset the IT field

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
> db.names.update({"IT": {$exists: true}}, {$unset: {"IT": "true"}});
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
 
> db.names.find().pretty()
{
        "_id" : ObjectId("53c6392a2eea8062e084cb57"),
        "Company" : "Google",
        "Product" : "Nexus",
        "No" : 1
}
{
        "_id" : ObjectId("53c639392eea8062e084cb58"),
        "Company" : "Apple",
        "Product" : "Mac",
        "No" : 2
}
{
        "_id" : ObjectId("53c63b26b003603dfdcf8c52"),
        "Company" : "Xiaomi",
        "Product" : "Mi3",
        "No" : 3
}
{
        "_id" : ObjectId("53c63bd1b003603dfdcf8c53"),
        "Product" : "Smart Watch",
        "No" : 4,
        "Company" : "Sony"
}

Here we select the document which has a field called “IT” and remove it from that document.

Related Read:
$exists, $type, $regex operators: MongoDB
Update with UNSET Operator: MongoDB

multi-update true

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
> db.names.update({}, {$set: {"IT": "true"}}, {multi: true});
WriteResult({ "nMatched" : 4, "nUpserted" : 0, "nModified" : 4 })
 
> db.names.find().pretty()
{
        "_id" : ObjectId("53c6392a2eea8062e084cb57"),
        "Company" : "Google",
        "Product" : "Nexus",
        "No" : 1,
        "IT" : "true"
}
{
        "_id" : ObjectId("53c639392eea8062e084cb58"),
        "Company" : "Apple",
        "Product" : "Mac",
        "No" : 2,
        "IT" : "true"
}
{
        "_id" : ObjectId("53c63b26b003603dfdcf8c52"),
        "Company" : "Xiaomi",
        "Product" : "Mi3",
        "No" : 3,
        "IT" : "true"
}
{
        "_id" : ObjectId("53c63bd1b003603dfdcf8c53"),
        "Product" : "Smart Watch",
        "No" : 4,
        "Company" : "Sony",
        "IT" : "true"
}

Here we have 3 arguments for update() method. First one is intentionally left empty. In second argument, we specify the changes needed to the documents. In third argument we specify the option, multi: true – which tells mongo engine to match with all the documents in the collection.

Update with upsert: MongoDB

Lets learn to use upsert option with update() method.

upsert basically inserts document into the collection, if there is no matching document found in the collection. If there is a match found, then the document simply gets updated – in which case upsert option will not have any effect on the selected document.

update upsert mongodb Update with upsert: MongoDB

test database, names collection

1
2
3
4
5
6
7
8
9
10
11
12
13
> db.names.find().pretty()
{
        "_id" : ObjectId("53c6392a2eea8062e084cb57"),
        "Company" : "Google",
        "Product" : "Nexus",
        "No" : 1
}
{
        "_id" : ObjectId("53c639392eea8062e084cb58"),
        "Company" : "Apple",
        "Product" : "iPhone",
        "No" : 2
}

There are 2 documents inside “names” collection, with _id, Company, Product and No as its fields.

Update with upsert: MongoDB


You need to a flashplayer enabled browser to view this YouTube video

YouTube Link: http://www.youtube.com/watch?v=ZQjNHjVv_W8 [Watch the Video In Full Screen.]



update() method, with $set operator

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
> db.names.update({"Company": "Apple"}, {$set: {"Product": "Mac"}});
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
 
> db.names.find().pretty()
{
        "_id" : ObjectId("53c6392a2eea8062e084cb57"),
        "Company" : "Google",
        "Product" : "Nexus",
        "No" : 1
}
{
        "_id" : ObjectId("53c639392eea8062e084cb58"),
        "Company" : "Apple",
        "Product" : "Mac",
        "No" : 2
}

We select second document using “Company” – “Apple“. Here, the “Product” field value gets updated from iPhone to Mac.

Related Read: Update with SET Operator: MongoDB

No Match Found for Updation

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
> db.names.update({"Company": "Xiaomi"}, {$set: {"Product": "Mi3", "No": 3}});
WriteResult({ "nMatched" : 0, "nUpserted" : 0, "nModified" : 0 })
 
> db.names.find().pretty()
{
        "_id" : ObjectId("53c6392a2eea8062e084cb57"),
        "Company" : "Google",
        "Product" : "Nexus",
        "No" : 1
}
{
        "_id" : ObjectId("53c639392eea8062e084cb58"),
        "Company" : "Apple",
        "Product" : "Mac",
        "No" : 2
}

Since, there is no matching “Company” called Xiaomi in “names” collection, there will be no matching document, hence no modification of document.

updation with upsert option

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
> db.names.update({"Company": "Xiaomi"}, 
                  {$set: {"Product": "Mi3", "No": 3}},
                  {upsert: true});
WriteResult({
        "nMatched" : 0,
        "nUpserted" : 1,
        "nModified" : 0,
        "_id" : ObjectId("53c63b26b003603dfdcf8c52")
})
 
> db.names.find().pretty()
{
        "_id" : ObjectId("53c6392a2eea8062e084cb57"),
        "Company" : "Google",
        "Product" : "Nexus",
        "No" : 1
}
{
        "_id" : ObjectId("53c639392eea8062e084cb58"),
        "Company" : "Apple",
        "Product" : "Mac",
        "No" : 2
}
{
        "_id" : ObjectId("53c63b26b003603dfdcf8c52"),
        "Company" : "Xiaomi",
        "Product" : "Mi3",
        "No" : 3
}

Observe the 3rd argument of update() method – upsert: true. In this case, since there is no matching document found inside the “names” collection, for “Company” Xiaomi, it inserts the document with all the fields and values specified in the update command. i.e., Company: Xiaomi, Product: Mi3, No: 3

unintentional modification of data with upsert

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
> db.names.update({"No": {$gt: 3}}, 
                  {$set: {"Product": "Glass", "No": 4}}, 
                  {upsert: true});
WriteResult({
        "nMatched" : 0,
        "nUpserted" : 1,
        "nModified" : 0,
        "_id" : ObjectId("53c63bd1b003603dfdcf8c53")
})
 
> db.names.find().pretty()
{
        "_id" : ObjectId("53c6392a2eea8062e084cb57"),
        "Company" : "Google",
        "Product" : "Nexus",
        "No" : 1
}
{
        "_id" : ObjectId("53c639392eea8062e084cb58"),
        "Company" : "Apple",
        "Product" : "Mac",
        "No" : 2
}
{
        "_id" : ObjectId("53c63b26b003603dfdcf8c52"),
        "Company" : "Xiaomi",
        "Product" : "Mi3",
        "No" : 3
}
{
        "_id" : ObjectId("53c63bd1b003603dfdcf8c53"),
        "Product" : "Glass",
        "No" : 4
}

We have documents which has values 1, 2 and 3 for the field “No”. But in above command the condition is to select documents which has “No” value above 3 – which results in 0 documents being selected. But since we have “upsert: true” as the third argument of update() method, the “field: value” – “Product: Glass” and “No: 4″ gets inserted into the “names” collection. So we need to take proper care of our commands and conditions – so that unintentional insertion of data or document can be avoided.

upsert effect on existing data/document

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
> db.names.update({"No": 4}, 
                  {$set: {"Company": "Sony", "Product": "Smart Watch"}}, 
                  {upsert: true});
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
 
> db.names.find().pretty()
{
        "_id" : ObjectId("53c6392a2eea8062e084cb57"),
        "Company" : "Google",
        "Product" : "Nexus",
        "No" : 1
}
{
        "_id" : ObjectId("53c639392eea8062e084cb58"),
        "Company" : "Apple",
        "Product" : "Mac",
        "No" : 2
}
{
        "_id" : ObjectId("53c63b26b003603dfdcf8c52"),
        "Company" : "Xiaomi",
        "Product" : "Mi3",
        "No" : 3
}
{
        "_id" : ObjectId("53c63bd1b003603dfdcf8c53"),
        "Product" : "Smart Watch",
        "No" : 4,
        "Company" : "Sony"
}

In this case, there is a document inside the collection which has “No”: 4. So the upsert option will not have any effect on the selected data/document – it simply gets updated with the field values specified in the second argument of update() method. i.e., the document with “No”: 4 is selected and the “Company”: “Sony”, “Product”: “Smart Watch” gets added to the existing document.

Working With Arrays: MongoDB

Lets learn some of the methods and operators to work with arrays in MongoDB.

In this video tutorial, we’ll be looking at:
update()
$set
$push
$pop
$pushAll
$pull
$pullAll
$addToSet

update set push pop pushAll pull pullAll addToSet operators mongodb Working With Arrays: MongoDB

test database, names collection

1
2
3
4
5
6
7
8
9
MongoDB shell version: 2.6.1
connecting to: test
> db.names.find()
 
> db.names.insert({"_id": 1, "a": [1, 2, 3, 4]});
WriteResult({ "nInserted" : 1 })
 
> db.names.find()
{ "_id" : 1, "a" : [ 1, 2, 3, 4 ] }

We insert a document into “names” collection. We’ll be working on the array field present in the document.

Working With Arrays: MongoDB


You need to a flashplayer enabled browser to view this YouTube video

YouTube Link: http://www.youtube.com/watch?v=xc5TiSqvnPQ [Watch the Video In Full Screen.]



update() method

1
2
3
4
5
> db.names.update({"_id": 1}, {"a": [1, 2, 3, 4, 5]});
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
 
> db.names.find()
{ "_id" : 1, "a" : [ 1, 2, 3, 4, 5 ] }

We can update the array by simply using update() method. But here, we need to remember all the elements of the array as well as the new element to be inserted into the array. Thus, this method is somewhat tedious.

$set operator

1
2
3
4
5
> db.names.update({"_id": 1}, {$set: {"a.5": 6}});
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
 
> db.names.find()
{ "_id" : 1, "a" : [ 1, 2, 3, 4, 5, 6 ] }

We could insert an element into the array by making use of $set operator. Here we need to know the position where the new element needs to be inserted. In mongoDB, array index starts from zero.

$push operator

1
2
3
4
5
> db.names.update({"_id": 1}, {$push: {"a": 7}});
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
 
> db.names.find()
{ "_id" : 1, "a" : [ 1, 2, 3, 4, 5, 6, 7 ] }

using $push operator we can insert an element to the right hand side of the array. Here we simply specify the key and the value/element to be inserted.

$pop: {a: 1}

1
2
3
4
5
> db.names.update({"_id": 1}, {$pop: {"a": 1}});
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
 
> db.names.find()
{ "_id" : 1, "a" : [ 1, 2, 3, 4, 5, 6 ] }

$pop operator which has 1 as value to the key, removes the right most element from the array.

$pop: {a: -1}

1
2
3
4
5
> db.names.update({"_id": 1}, {$pop: {"a": -1}});
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
 
> db.names.find()
{ "_id" : 1, "a" : [ 2, 3, 4, 5, 6 ] }

$pop operator which has -1 as value to the key, removes the left most element from the array.

$pushAll operator

1
2
3
4
5
> db.names.update({"_id": 1}, {$pushAll: {"a": [7, 8]}});
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
 
> db.names.find()
{ "_id" : 1, "a" : [ 2, 3, 4, 5, 6, 7, 8 ] }

$pushAll operator inserts all the elements(array of elements) specified, to the right hand side of the existing array.

$pull operator

1
2
3
4
5
> db.names.update({"_id": 1}, {$pull: {"a": 3}});
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
 
> db.names.find()
{ "_id" : 1, "a" : [ 2, 4, 5, 6, 7, 8 ] }

$pull operator pulls or removes the specified element from the array, irrespective of its position.

$pullAll operator

1
2
3
4
5
> db.names.update({"_id": 1}, {$pullAll: {"a": [2, 7, 8]}});
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
 
> db.names.find()
{ "_id" : 1, "a" : [ 4, 5, 6 ] }

$pullAll operator pulls/removes all the elements(array of elements) specified from the array, irrespective of its position.

$addToSet operator

1
2
3
4
5
> db.names.update({"_id": 1}, {$addToSet: {"a": 3}});
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
 
> db.names.find()
{ "_id" : 1, "a" : [ 4, 5, 6, 3 ] }

$addToSet operator adds specified element to the array, if its not already present in the array. If the element is already present in the array, then it doesn’t add it once again.

1
2
3
4
> db.names.update({"_id": 1}, {$addToSet: {"a": 3}});
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 0 })
> db.names.find()
{ "_id" : 1, "a" : [ 4, 5, 6, 3 ] }

If the element is already present in the array, then $addToSet doesn’t add it once again.

Increment($inc) operator: MongoDB

Lets learn how to use $inc operator with update() method in MongoDB.

inc operator with update Increment($inc) operator: MongoDB

test database, names collection

1
2
3
4
5
6
7
8
9
10
11
12
> db.names.find().pretty()
{
        "_id" : ObjectId("53c3fc2005e0ce2719d91bd2"),
        "Company" : "Google",
        "No" : 1
}
{
        "_id" : ObjectId("53c3fd3bb9ae26fa217b1e12"),
        "Company" : "Apple",
        "Product" : "iPhone",
        "No" : 2
}

Here we have 2 documents. And we’ll be working on first document to illustrate the working of $inc operator.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
> db.names.update({"Company": "Google"}, {$inc: {"No": 2}});
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
 
> db.names.find().pretty()
{
        "_id" : ObjectId("53c3fc2005e0ce2719d91bd2"),
        "Company" : "Google",
        "No" : 3
}
{
        "_id" : ObjectId("53c3fd3bb9ae26fa217b1e12"),
        "Company" : "Apple",
        "Product" : "iPhone",
        "No" : 2
}

Here “No” field will be incremented by 2. So the final value of “No” field is 3 (1+2).

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
> db.names.update({"Company": "Google"}, {$inc: {"Sl_no": 1}});
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
 
> db.names.find().pretty()
{
        "_id" : ObjectId("53c3fc2005e0ce2719d91bd2"),
        "Company" : "Google",
        "No" : 3,
        "Sl_no" : 1
}
{
        "_id" : ObjectId("53c3fd3bb9ae26fa217b1e12"),
        "Company" : "Apple",
        "Product" : "iPhone",
        "No" : 2
}

If we apply $inc operator on an non-existing field, it will be created with the increment value itself.

Increment($inc) operator: MongoDB


You need to a flashplayer enabled browser to view this YouTube video

YouTube Link: http://www.youtube.com/watch?v=LjJJOY4DLMs [Watch the Video In Full Screen.]



1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
> db.names.update({"Company": "Google"}, {$inc: {"No": 1}});
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
> db.names.find().pretty()
{
        "_id" : ObjectId("53c3fc2005e0ce2719d91bd2"),
        "Company" : "Google",
        "No" : 4,
        "Sl_no" : 1
}
{
        "_id" : ObjectId("53c3fd3bb9ae26fa217b1e12"),
        "Company" : "Apple",
        "Product" : "iPhone",
        "No" : 2
}

Here we increment the value of “No” once again by 1. So the final value of “No” field is 4 (3+1).