How to Apply the CLT (Central Limit Theorem) to Simulated Data

The Central Limit Theorem (CLT) is a fundamental concept in statistics that states that the distribution of the sample mean approaches a normal distribution as the sample size increases, regardless of the original distribution of the population, provided the data are independent and identically distributed (i.i.d.). To apply the CLT to simulated data, you’ll follow these key steps:

1. Understand the Central Limit Theorem

The CLT asserts that if you repeatedly take random samples from any population, calculate their sample means, and plot those means, the resulting distribution will be approximately normal, even if the original population is not normally distributed.

2. Simulate the Original Population

The first step is to simulate or generate an original population. This population can follow any distribution. Common choices for simulated populations include:

• Uniform distribution

• Exponential distribution

• Binomial distribution

• Poisson distribution

• Skewed or highly non-normal distributions

The idea is that you don’t need to start with a normally distributed population to demonstrate the CLT; the theorem holds for many types of distributions.

Example:

• You can simulate a population using a random number generator in a software tool like Python, R, or even Excel.

python
CopyEdit
import numpy as np
np.random.seed(0)
population = np.random.exponential(scale=1, size=100000)  # Exponential distribution

3. Draw Random Samples from the Population

Once you have your simulated population, draw a set of random samples from it. Each sample should have a fixed size $n$, and you’ll repeat this sampling process multiple times (e.g., 1,000 or more).

Example:

• For each sample, randomly select $n$ data points from the population.

python
CopyEdit
sample_size = 50
num_samples = 1000
sample_means = []

for _ in range(num_samples):
    sample = np.random.choice(population, size=sample_size)
    sample_means.append(np.mean(sample))

4. Calculate the Sample Means

For each of the samples drawn, calculate the mean of the sample. The sample mean will be the statistic that you track. As you increase the number of samples (e.g., 1,000), the distribution of these sample means will give you a good approximation of the normal distribution, regardless of the original population’s distribution.

Example:

• In the loop above, we calculated the sample mean for each randomly drawn sample.

5. Plot the Distribution of Sample Means

After calculating the means for all the samples, plot their distribution. This will show how the sample means are distributed. As per the CLT, the shape of this distribution should resemble a normal distribution, even if the original population is non-normal.

Example:

• You can use a histogram or density plot to visualize the sample means.

python
CopyEdit
import matplotlib.pyplot as plt
plt.hist(sample_means, bins=30, edgecolor='black', density=True)
plt.title('Distribution of Sample Means')
plt.xlabel('Sample Mean')
plt.ylabel('Density')
plt.show()

6. Verify the Normality of the Distribution

To verify that the sample means are approximately normally distributed, you can perform several tests:

• Visual Inspection: Check the histogram of sample means for a bell-shaped curve.

• Statistical Tests: Apply tests such as the Shapiro-Wilk test, Anderson-Darling test, or Kolmogorov-Smirnov test for normality.

Example (Shapiro-Wilk test):

python
CopyEdit
from scipy.stats import shapiro
stat, p_value = shapiro(sample_means)
print(f"Shapiro-Wilk Test Statistic: {stat}, P-value: {p_value}")

If the p-value is high (typically above 0.05), you can’t reject the null hypothesis, suggesting that the sample means are normally distributed.

7. Observe the Convergence to Normality

The CLT becomes more apparent as the sample size $n$ increases. If you repeat the above steps with larger sample sizes (e.g., $n = 100$ or $n = 500$), the sample means will more closely approximate a normal distribution.

Example:

• Try changing the sample size to observe how the distribution of sample means becomes more normal as $n$ increases.

python
CopyEdit
sample_size = 100  # Increase sample size
sample_means = []

for _ in range(num_samples):
    sample = np.random.choice(population, size=sample_size)
    sample_means.append(np.mean(sample))

plt.hist(sample_means, bins=30, edgecolor='black', density=True)
plt.title(f'Distribution of Sample Means (Sample size = {sample_size})')
plt.xlabel('Sample Mean')
plt.ylabel('Density')
plt.show()

8. Understanding the Impact of Sample Size

The CLT tells us that as the sample size increases, the distribution of sample means becomes more tightly centered around the population mean and becomes increasingly symmetric and bell-shaped. This behavior is more noticeable with larger populations and larger sample sizes.

• With small sample sizes, you might still observe skewness or kurtosis in the sample means’ distribution.

• As sample size increases, the standard deviation of the sample means (also known as the standard error) decreases, leading to a more normal distribution.

Conclusion

Applying the Central Limit Theorem to simulated data helps demonstrate its power and importance in statistics. Even if the original population is not normally distributed, the distribution of the sample means will approach normality as the sample size and the number of samples increase. This principle underlies much of classical statistical inference, allowing statisticians to use normal theory methods (like confidence intervals and hypothesis tests) even when dealing with non-normal data.

By simulating data and running repeated sampling experiments, you can visually and empirically see the CLT in action, which is a powerful tool for understanding the robustness of statistical methods.