Example: Predicting House Prices

Now it's time to put everything together! Let's build a complete Machine Learning model to predict house prices, using all the concepts we've covered: functions, matrices, statistics, probability, and gradient descent. This example will show you how these mathematical tools work together in a real machine learning application.

We'll walk through the entire process—from defining the problem to training the model—just like a data scientist would approach it in practice. This section is more mathematical, but you now have all the foundation needed to understand it.

Step 1: Defining Our Problem (Supervised Learning)

Remember from our discussion of machine learning types that this is a supervised learning problem—we have labeled training data (houses with known prices) and want to predict prices for new houses.

Specifically, it's a regression problem because we're predicting a continuous numerical value (price) rather than categories.

Our Dataset: Imagine you're working for a real estate company with data on:

• Size(square meters): Our primary feature.
• Location Score (1-10): Quality of neighborhood.
• Price (€): What we want to predict (our target variable).

We want to build a function (or model) that maps house features to prices:

In other words, we want to teach the model to capture the relationship between house features and price so it can make accurate predictions on unseen properties.

Step 2: Choosing Our Model (Linear Regression)

Based on our exploration of core machiine learning algorithms, we'll use linear regression because:

• House prices tend to increase with size (linear relationship).
• We want an interpretable model where we can understand each feature's impact.
• It's simple enough to solve step-by-step.

Single Feature Model: Starting with just house size, our function looks like:

Let's break it down:

• $y$ is the predicted price.
• $x$ is the size of the house.
• $w_1$ is the weight (how much price increases per square meter).
• $b$ is the bias (baseline price for a house with 0 square meters).

Multiple Features Model: Including location score, we extend to:

where $x_1$ is our first feature size and $x_2$ is location score.

We can extend a linear regression model depending on the amount of features we have!

Step 3: Representing Data with Matrices

Remember matrices from Chapter 2? Here's how we organize our data for efficient processing. Using matrix notation, we can write:

This is new to us, but remember how we can stack information to form matrices or vectors? Matrices after often represented using a captial letter ($\mathbf{X}$) while vectors are represented with lower case letters ($\mathbf{y}$).

What do these matrices and vectors look like do you think? Let's break them down.

$\mathbf{X}$ is the feature matrix, where we stack all features for each data point. The labels (or truth values) are stored in vector $\mathbf{y}$ and the weights are represented by $\mathbf{w}$.

Each row in $\mathbf{X}$ represents one house with all its features. Each column represents one feature across all houses. The $\mathbf{y}$ vector contains all the actual prices we're trying to predict.

This matrix representation lets us process all houses simultaneously instead of one at a time—much more efficient!

Step 3: Finding the Best Weights (Optimization)

Now comes the core challenge: finding the optimal weights. This is where statistics, derivatives, and loss functions from our previous discussions come together.

To build our model, we need to find the best weights that determine how much house size affects price. Intuitively, we want to find the line that best fits the data, minimizing the difference between our predictions and actual prices.

If we want to find $w_1$ by hand, let's start with two houses to illustrate the principle:

• House 1: Has a size of 50m² and costs €150.000.
• House 2: Has a size of 100m² and costs €300.000.

Let's use the slope formula, which is actually a derivative!

$\Delta$ (delta) just means "change." So when you see $\Delta x$, it means "change in x"—how much $x$ has increased or decreased. Same with $\Delta y$, which is the "change in $y$." It’s like saying: how far did we move horizontally ($x$), and how far did things shift vertically ($y$)?

This means each additional square meter adds €3.000 to the house price.

Now, to find the bias ($b$), we substitute one of our data points into the equation:

Thus, our final model for predicting price based on size is:

If we extend this to multiple features, we move from simple algebra to matrices—where the same principle applies, but we solve for all weights at once! This ensures we find the optimal weights that minimize the overall prediction error across all training data.

Step 5: Using Gradient Descent for Complex Cases

The manual approach works for simple cases, but real datasets have hundreds of houses and multiple features. This is where gradient descent (which we just learned about) becomes essential.

The process is as follows:

1. Start with random weights: $w_1 = 0.5, \quad w_2 = 0.3, \quad b = 1000$.
2. Make predictions: Use current weights to predict all house prices.
3. Calculate error: Use Mean Squared Error (our loss function from evaluation).
4. Compute gradients: Use derivatives to find which direction to adjust weights.
5. Update weights: Move in the direction that reduces error.
6. Repeat: Continue until error stops decreasing.

Why Derivatives Matter

Remember from Chapter 2 that derivatives tell us how functions change. In gradient descent:

• $\frac{\Delta MSE}{\Delta w_1}$ tells us how the error changes when we adjust the size weight.
• $\frac{\Delta MSE}{\Delta w_2}$ tells us how the error changes when we adjust the location weight.
• $\frac{\Delta MSE}{\Delta b}$ tells us how the error changes when we adjust the bias.

So, gradient descent uses these derivatives as a guide, steadily adjusting the weights until the model makes the most accurate predictions possible.

Step 6: Evaluating Our Model

Using our evaluation techniques, let's assess how well our model performs. After gradient descent, our model might learn:

• $w_1 = 2800$ (each m² adds €2.800).
• $w_2 = 15.000$ (each location point adds €15.000).
• $b = 50.000$ (base price regardless of size/location).

After plugging these values into our linear regression formula, our model becomes:

Let's interpret this model:

• Location has a bigger impact than size (weight of 15.000 vs 2.800).
• Even a tiny house in a great location has significant base value.
• This makes intuitive sense—location is often more important than size in real estate.

After using our train/validation/test split and testing our model we might get:

• Training MSE: 25.000² (average error of €25.000).
• Validation MSE: 28.000² (slightly higher, which is normal).
• Test MSE: 27.000² (final performance estimate).

The similar performance across all datasets suggests good generalization—our model learned patterns rather than memorizing training data.

Step 7: Making Predictions

Now we can use our trained model to predict prices for new houses!

🏠 New House Example: Let's say a new house comes on the market and we want to find out what price to ask for it, it has the following features:

1. Size: 120 m²
2. Location Score: 7

Using our model and the known features we get a prediction:

So this new house has a predicted price of €491.000.

Remember probability from Chapter 2? In practice, we'd also provide confidence intervals: "The predicted price is €491,000 ± €35,000 with 95% confidence." This acknowledges the inherent uncertainty in predictions.

What We've Accomplished

This example demonstrates how all our learned concepts work together:

• Functions: We created a mathematical function mapping house features to prices.
• Matrices: We organized data efficiently for batch processing.
• Statistics: We used mean squared error to measure and minimize prediction errors.
• Derivatives: We applied gradient descent to find optimal model parameters.
• Evaluation: We properly split data and assessed model performance.
• Supervised Learning: We learned from labeled examples to make predictions on new data.
• Linear Regression: We implemented one of the core machine learning algorithms.

Professional, real-world, models would also include:

• More features (bedrooms, age, garage, etc.).
• Polynomial regression (a higher order) for non-linear relationships.
• Regularization to prevent overfitting.
• Cross-validation for more robust evaluation.
• Feature engineering to create better input variables.

Together, these steps give you the foundation for building real-world machine learning models that are both accurate and reliable.

Final Takeaways

Building a machine learning model involves systematically applying mathematical and statistical principles to learn patterns from data. This house price predictor demonstrates how functions, matrices, derivatives, and optimization techniques combine to create systems that can make accurate predictions on new, unseen data.

The key insight is that machine learning isn't magic—it's the systematic application of mathematical tools to find the best possible function for mapping inputs to outputs. Understanding this foundation prepares you to tackle more complex machine learning problems and appreciate the sophisticated algorithms powering modern AI applications.