9. Data aggregation

The aggregation pipeline lets you transform and combine documents. It is like doing a GROUP BY in SQL, but much more powerful.

Pipeline structure

An aggregation is an array of stages. Each stage transforms the documents and passes them to the next one:

db.unicorns.aggregate([
    { $match: { ... } },    // Stage 1: filter
    { $group: { ... } },    // Stage 2: group
    { $sort: { ... } }      // Stage 3: sort
])

The $group stage

The most important stage is $group. It groups documents and computes values:

// Count unicorns by gender
db.unicorns.aggregate([
    {
        $group: {
            _id: '$gender',
            total: { $sum: 1 }
        }
    }
])

Result:

[
    { _id: 'f', total: 4 },
    { _id: 'm', total: 4 }
]

The _id field indicates which field to group by. The $ symbol before a field name means "the value of this field".

Accumulation operators

Inside $group you can use these operators:

$sum: sums values

db.unicorns.aggregate([
    {
        $group: {
            _id: '$gender',
            totalWeight: { $sum: '$weight' }
        }
    }
])

$avg: calculates the average

db.unicorns.aggregate([
    {
        $group: {
            _id: '$gender',
            avgVampires: { $avg: '$vampires' }
        }
    }
])

$min: minimum value

db.unicorns.aggregate([
    {
        $group: {
            _id: '$gender',
            minWeight: { $min: '$weight' }
        }
    }
])

$max: maximum value

db.unicorns.aggregate([
    {
        $group: {
            _id: '$gender',
            maxVampires: { $max: '$vampires' }
        }
    }
])

$addToSet: array with unique values

db.unicorns.aggregate([
    {
        $group: {
            _id: '$gender',
            names: { $addToSet: '$name' }
        }
    }
])

The $match stage

Filters documents before grouping. It is like find:

// Average weight of unicorns under 600
db.unicorns.aggregate([
    { $match: { weight: { $lt: 600 } } },
    {
        $group: {
            _id: '$gender',
            avgWeight: { $avg: '$weight' }
        }
    }
])

The $sort stage

Sorts the results:

db.unicorns.aggregate([
    {
        $group: {
            _id: '$gender',
            total: { $sum: 1 }
        }
    },
    { $sort: { total: -1 } }
])

The $limit stage

Limits the number of results:

// Top 3 genders with the most unicorns
db.unicorns.aggregate([
    {
        $group: {
            _id: '$gender',
            total: { $sum: 1 }
        }
    },
    { $sort: { total: -1 } },
    { $limit: 3 }
])

The $unwind stage

Breaks down an array into separate documents. Very useful for analyzing arrays:

// Which food is the most popular?
db.unicorns.aggregate([
    { $unwind: '$loves' },
    {
        $group: {
            _id: '$loves',
            count: { $sum: 1 }
        }
    },
    { $sort: { count: -1 } },
    { $limit: 1 }
])

Complete example

Let's find the favorite food of the heavy unicorns (over 500):

db.unicorns.aggregate([
    // 1. Filter heavy unicorns
    { $match: { weight: { $gt: 500 } } },

    // 2. Break down the array of foods
    { $unwind: '$loves' },

    // 3. Group by food and count
    {
        $group: {
            _id: '$loves',
            total: { $sum: 1 },
            unicorns: { $addToSet: '$name' }
        }
    },

    // 4. Sort by popularity
    { $sort: { total: -1 } },

    // 5. Only the top 1
    { $limit: 1 }
])
Activity 1
  1. Calculate the total number of listings per neighborhood (neighbourhood_group_cleansed)
  2. Calculate the number of listings per room type (room_type)
  3. Find how many superhosts there are in total (count where host_is_superhost is "t")
  4. Count how many listings there are per property type (property_type), showing only the 10 most common
  5. Calculate how many listings have more than 50 reviews

Pro:

  1. Calculate the average of accommodates (capacity) per neighborhood, sorted from highest to lowest
  2. Find the top 5 neighborhoods with the most listings (group by neighbourhood_group_cleansed, count and sort)
  3. Calculate the maximum number of reviews (number_of_reviews) for each room type (room_type)

Solutions

This work is under a Attribution-NonCommercial-NoDerivatives 4.0 International license.

Desafíos de programación atemporales y multiparadigmáticos

Desafíos de programación atemporales y multiparadigmáticos

Te encuentras ante un librillo de actividades, divididas en 2 niveles de dificultad. Te enfrentarás a los casos más comunes que te puedes encontrar en pruebas técnicas o aprender conceptos elementales de programación.

Buy the book

Will you buy me a coffee?

This is how I keep writing without ads or paywalls.

Comments

There are no comments yet.