Mongo Add account name indexes to query data faster
Analysts are often using Mongo aggregation queries with $lookup
, like this one:
db.account_object.aggregate([{
$match: {
}}, {
$lookup: {
"from": "comment_object",
let: {
authorr: "$name"
},
pipeline: [{$match: {$expr: {$and: [{"$eq": ["$author", "$$authorr"]}]}}}],
"as": "his_comments"
}
}])
Such queries are very slow. Let's run such query when we have ~590.000 comments.
Result:
15:32 - Started
15:38 - Ended
It is so slow because we using comment_object.author
field which is just string, and not indexed ObjectId field.
Let's add comment_object.author_id
field with index on it, and run slightly rewrited query after full re-sync:
db.account_object.aggregate([{
$match: {
}}, {
$lookup: {
"from": "comment_object",
let: {
authorr: "$_id"
},
pipeline: [{$match: {$expr: {$and: [{"$eq": ["$author_id", "$$authorr"]}]}}}],
"as": "his_comments"
}
}])
Result:
15:43:02 - Started
15:43:03 - Ended
Indexes could strongly improve performance of using & development of Mongo aggregation queries.
In this issue:
- Let's add them to all account field in all often-using collections.
- (Optionally) Improve code of adding of index. It is too long - 3 lines:
document owner_index;
owner_index << "owner_id" << 1;
doc.indexes_to_create.push_back(std::move(owner_index));
It is done for more flexibility, but usually we don't need to index on multiple fields.
Let's write helper function which will allow to create such index in 1 line of code (with 2 args - field name and index value, i.e. "owner_id" and 1), but if need we'll still could do it with an old way.