9. Data aggregation
The aggregation pipeline lets you transform and combine documents. It is like doing a GROUP BY in SQL, but much more powerful.
Pipeline structure
An aggregation is an array of stages. Each stage transforms the documents and passes them to the next one:
db.unicorns.aggregate([
{ $match: { ... } }, // Stage 1: filter
{ $group: { ... } }, // Stage 2: group
{ $sort: { ... } } // Stage 3: sort
])
The $group stage
The most important stage is $group. It groups documents and computes values:
// Count unicorns by gender
db.unicorns.aggregate([
{
$group: {
_id: '$gender',
total: { $sum: 1 }
}
}
])
Result:
[
{ _id: 'f', total: 4 },
{ _id: 'm', total: 4 }
]
The _id field indicates which field to group by. The $ symbol before a field name means "the value of this field".
Accumulation operators
Inside $group you can use these operators:
$sum: sums values
db.unicorns.aggregate([
{
$group: {
_id: '$gender',
totalWeight: { $sum: '$weight' }
}
}
])
$avg: calculates the average
db.unicorns.aggregate([
{
$group: {
_id: '$gender',
avgVampires: { $avg: '$vampires' }
}
}
])
$min: minimum value
db.unicorns.aggregate([
{
$group: {
_id: '$gender',
minWeight: { $min: '$weight' }
}
}
])
$max: maximum value
db.unicorns.aggregate([
{
$group: {
_id: '$gender',
maxVampires: { $max: '$vampires' }
}
}
])
$addToSet: array with unique values
db.unicorns.aggregate([
{
$group: {
_id: '$gender',
names: { $addToSet: '$name' }
}
}
])
The $match stage
Filters documents before grouping. It is like find:
// Average weight of unicorns under 600
db.unicorns.aggregate([
{ $match: { weight: { $lt: 600 } } },
{
$group: {
_id: '$gender',
avgWeight: { $avg: '$weight' }
}
}
])
The $sort stage
Sorts the results:
db.unicorns.aggregate([
{
$group: {
_id: '$gender',
total: { $sum: 1 }
}
},
{ $sort: { total: -1 } }
])
The $limit stage
Limits the number of results:
// Top 3 genders with the most unicorns
db.unicorns.aggregate([
{
$group: {
_id: '$gender',
total: { $sum: 1 }
}
},
{ $sort: { total: -1 } },
{ $limit: 3 }
])
The $unwind stage
Breaks down an array into separate documents. Very useful for analyzing arrays:
// Which food is the most popular?
db.unicorns.aggregate([
{ $unwind: '$loves' },
{
$group: {
_id: '$loves',
count: { $sum: 1 }
}
},
{ $sort: { count: -1 } },
{ $limit: 1 }
])
Complete example
Let's find the favorite food of the heavy unicorns (over 500):
db.unicorns.aggregate([
// 1. Filter heavy unicorns
{ $match: { weight: { $gt: 500 } } },
// 2. Break down the array of foods
{ $unwind: '$loves' },
// 3. Group by food and count
{
$group: {
_id: '$loves',
total: { $sum: 1 },
unicorns: { $addToSet: '$name' }
}
},
// 4. Sort by popularity
{ $sort: { total: -1 } },
// 5. Only the top 1
{ $limit: 1 }
])
Activity 1
- Calculate the total number of listings per neighborhood (
neighbourhood_group_cleansed) - Calculate the number of listings per room type (
room_type) - Find how many superhosts there are in total (count where
host_is_superhostis"t") - Count how many listings there are per property type (
property_type), showing only the 10 most common - Calculate how many listings have more than 50 reviews
Pro:
- Calculate the average of
accommodates(capacity) per neighborhood, sorted from highest to lowest - Find the top 5 neighborhoods with the most listings (group by
neighbourhood_group_cleansed, count and sort) - Calculate the maximum number of reviews (
number_of_reviews) for each room type (room_type)
This work is under a Attribution-NonCommercial-NoDerivatives 4.0 International license.
Building SPAs with Django and HTML Over the Wire: Learn to build real-time single page applications with Python
The HTML over WebSockets approach simplifies single-page application (SPA) development and lets you bypass learning a JavaScript rendering framework such as React, Vue, or Angular, moving the logic to Python. This web application development book provides you with all the Django tools you need to simplify your developments with real-time results.
Buy the bookHelp me keep writing
Every coffee gives me a push toward the next article.
Sure, it's on me!
Comments
There are no comments yet.