Statistics Behind Machine Learning: Understanding Linear Regression

Blog post by Anand: "Statistics Behind Machine Learning: Understanding Linear Regression" (published January 15, 2026; categories: Tech). Explore the mathematical foundations of machine learning through linear regression. Learn how statistics power AI predictions, from understanding dependent and independent variables to implementing regression in Python.

Without the math, the world is nothing. Even for AI, the situation has not changed. We are officially in the AI era. But has anyone known behind that artificial intelligence, the overall structure is built with math and statistics.

As a tech enthusiast, I had spent time to know 'What's actually happened behind the scenes?', 'How do they predict?', 'How do they respond?'

Through that process, I had learned something. It feels interesting. So, usually I am planning to share that as a blog.

So, without wasting time. Let's dive into it. But before that, a small disclaimer.

I learn this and share it through the medium and my website as a blog. If you find any points as wrong. Contact me through my mail at any time. I'm ready to agreed if any points are wrong.

What is Machine Learning?

Simply by using the given data, find the pattern and predict the future. Yes, they had many definitions, but simply it is the main core concept.

But how do they find the pattern?

Here, the influence of statistics came on. Today, we discussed the statistics behind linear regression and how it works in Python code. But a small other disclaimer. The upcoming all things are filled with statistical formulas and concepts. I try to explain without making it too sleepy. But if it's boring, sorry guys.

Before we continue, a few concepts behind the Linear Regression.

What is Regression?

One of the important terms behind ML, and one of the important types in Supervised Learning.

It's simply a method used to find the relationship between dependent & independent variables. The independent variables should be one or many, depending on the situation.

What is a Dependent & Independent variable?

The Independent variable means data that is used to predict the outcome for the dependent variable. Simply based on that, the dependent variable value changed accordingly. So, the dependent variable means it's an outcome for the independent variable.

They have a lot of other names for both dependent and independent variables, like Target & Feature, Attribute & Response.

Likewise, it goes on.

But what is the purpose of the dependent and independent variables, man?

One other question hit in mind, right? I will answer for it. It's simple. By using the independent variable, we will find the relation and then predict the dependent variable. But how?

Here, the statistics help us. We will see that with an example.

Linear Regression Formula:

In statistics, the equation is often written as

y = a + bx,

where a is the intercept and b is the slope.

In machine learning, we commonly write it as

y = mx + c.

Here for the blog, I'm going to use y = mx+c. Why, because these are all we do to know the ML behind Statistics right, so only.

Here, Y denotes the dependent variable, X denotes the independent variable, and m means slope.

The m helps to know how much y changes when x increases by 1.

Example:

If m=2:

x ↑ 1 → y ↑ 2

If m=−3:

x ↑ 1 → y ↓ 3

So, slope is the rate of change.

Then c denotes the intercept. It is a value of y when x = 0. A starting point of the line on the y-axis.

Example:

If c=5: x = 0 → y = 5

So, intercept = starting value of y.

🚨 Sleep Detected!

Wake up 😄

The above concepts feel boring right? Don't worry, the theory parts are over atmost. Now, enter into the example and apply the formulas in the example too.

Linear Regression Example

Let's first calculate the slope:

Formula to find the slope (m)

n = 4, ∑x = 10, ∑y = 170, Mean of x = 2.5, Mean of y = 42.5, ∑(x−x̄)(y−ȳ) = 25, ∑(x−x̄)² = 5

Finally, m = 5

Then calculate the intercept:

Formula to find the intercept

After applying the values, we got c = 30.

Then the equation forms for mx + c as 5x + 30.

Let's we check how it predicts?

If X = 2:

y = 5(2) + 30 = 40

Ya that's right. Even you see that in the above table.

I hope you now understand how the statistics work behind the linear regression. But we are not going to use these statistics to train and predict the model, right?

We are going to use Python. So, let's see how it works behind it.

Our Python code

The first five lines are simple; it's about importing the libraries and data. Then,

x = np.array(x).reshape(-1,1)

It's used to convert the list into a NumPy array.

But why do we need to change the list into an array?

While because the machine learning algorithms accept arrays. Also, not as a single array. It converts that into a 2D array.

lr = LinearRegression()

Here, we create an object by calling the class name LinearRegression().

lr.fit(x,y)

Here we will access the method name fit and complete the overall process.

Many of them were shocked. But ya it's true. By calling the object and using this method. We will do the entire process, which we see above as a statistical step, like finding m and c.

m = lr.coef_[0]

c = lr.intercept_

Here, m, c denotes our formula had m, c.

We will store that value by calling the attributes.

lr.coef_[0] means it is in the list. By using the [0], we extract the value and save it here in the m variable.

print(f"Regression Coefficient is {m:5.2f}")

print(f"Intercept is {c:5.2f}")

print(f"Regression equation is y = {m:5.2f}x + {c:5.2f}")

Then print the value in the form of an equation.

y1 = lr.predict([[1]])[0]

Here, by using the method predict, we will predict y if x = 1.

y = 5(1) + 30 = 35

Then we got 35 as output.

Finally, we reach the end of today's blog. But before ending the blog, I like to ask one question with an answer too?

Do you feel that AI grows faster than the tech boom?

But even before 2022, we were using AI with tech gadgets. But at that time, we called it pre-programmed in the censors and more. But now we called that as AI. It's not just a very fast-growing plant. Yes, I agreed, it's grown faster. But not very fast, we feel likewise due to the internet.

There is a days, we don't have internet. At that time, we had limited resources to know anything. But via the internet, when anything happens in the USA, we know here within 2 to 4 hours as max.

The best example the europe achieve industrialization. But russia achieve it at last. All these are due to one reason. We don't know.

While we talk about the touch screen magic, many of them already developed the neural network; it's a real feat and fate.

That's all for today, guys. As usual, when you feel this content is valuable, follow me for more upcoming Blogs.

Connect with Me:

LinkedIn: Anand Sundaramoorthy. Instagram: @anandsundaramoorthysa. Email: sanand03072005@gmail.com

Read it: https://www.anandsundaramoorthy.com/blog/statistics-behind-machine-learning-understanding-linear-regression