Correlation & A/B Testing
5. Correlation vs Causation
Correlation = Two variables move together. Causation = One variable directly causes a change in the other.
Classic Example: Ice cream sales ↑ and drowning deaths ↑ — correlated, but ice cream doesn't cause drowning. Both increase because of a confounding variable (temperature/summer).
🧠 Interview mein ye line zaroor bolo: "Correlation suggests a relationship but doesn't prove causation. To establish causation, we need controlled experiments like A/B tests."
Correlation Coefficient (r)
| r value | Interpretation | Strength |
|---|---|---|
| +1.0 | Perfect positive — both increase together | 💪💪💪 |
| +0.7 to +0.9 | Strong positive | 💪💪 |
| +0.3 to +0.7 | Moderate positive | 💪 |
| -0.3 to +0.3 | Weak / no linear relationship | — |
| -0.7 to -0.9 | Strong negative — one increases, other decreases | 💪💪 |
| -1.0 | Perfect negative | 💪💪💪 |
Worked Problem: A dataset shows: Ad Spend vs Revenue has r = 0.85. Ad Spend vs Customer Complaints has r = -0.15.
Interpretation:
- Ad Spend ↔ Revenue: Strong positive relationship.
As ad spend increases, revenue tends to increase significantly.
- Ad Spend ↔ Complaints: Weak/no relationship.
There's no meaningful linear pattern.
Interview Response: "r = 0.85 suggests a strong positive correlation,
but I'd run a regression to confirm and check for confounding variables
before concluding that higher ad spend causes higher revenue."
6. A/B Testing
A/B testing is a controlled experiment used to compare two versions and determine which performs better.
Worked Problem — A/B Test Analysis:
An e-commerce company tested two checkout flows:
Control (Old): 5,000 visitors, 200 purchases → Conversion = 4.0%
Treatment (New): 5,000 visitors, 250 purchases → Conversion = 5.0%
Absolute lift: +1 percentage point
Relative lift: (5.0 - 4.0) / 4.0 = 25% improvement
p-value from proportions z-test: 0.018
Decision: p = 0.018 < 0.05 → Statistically significant
Recommendation: Roll out the new checkout flow ✅
Common A/B Testing Mistakes:
- Peeking too early — checking results before sufficient data leads to Type I errors
- Testing too many variants — increases false positives (multiple comparisons problem)
- Unequal sample sizes — reduces statistical power
- Not accounting for seasonality — run tests for complete business cycles
🧠 Sample size ka sawaal aayega: "I'd use a power analysis calculator. Typically for a 5% minimum detectable effect with 80% power and α = 0.05, you need roughly 1,500–2,000 users per group."
7. Confidence Intervals
A confidence interval provides a range within which we expect the true population parameter to fall.
Formula (for mean): CI = x̄ ± Z × (σ / √n)
Where Z = 1.96 for 95% confidence, 2.58 for 99% confidence.
Worked Problem:
Survey of 100 customers: Average monthly spend = ₹5,000, SD = ₹1,500.
95% CI = 5000 ± 1.96 × (1500 / √100)
= 5000 ± 1.96 × 150
= 5000 ± 294
= [₹4,706 , ₹5,294]
Interpretation: "We are 95% confident that the true average
monthly spend of all customers falls between ₹4,706 and ₹5,294."
Higher confidence level (95% → 99%) = wider interval. There's a trade-off between confidence and precision.
8. Central Limit Theorem (CLT)
The most important theorem in statistics.
Statement: Regardless of the population distribution, the distribution of sample means approaches a normal distribution as sample size increases (typically n ≥ 30).
Why it matters:
- It's why we can use z-tests and t-tests even when the original data isn't normal
- It's the foundation of confidence intervals and hypothesis testing
Example: Customer order values might be right-skewed (many small orders, few large ones). But if you take the average of random samples of 50 orders, those averages will form a normal distribution.
🧠 Interview mein bolo: "CLT is the reason we can apply normal-based tests to non-normal data, as long as our sample size is large enough — typically n ≥ 30."
9. Interview Questions (12 Questions)
Q1: "Mean vs Median — when would you use each?"
Answer: "Mean when data is symmetric with no outliers — like student heights. Median when data is skewed or has outliers — like income or house prices. For example, India's average income is misleading because a few billionaires pull the mean up, while median gives a more realistic picture. In business reporting, I'd always check the distribution first before choosing."
Q2: "Where do you see Normal Distribution in real life?"
Answer: "Heights of people, IQ scores, measurement errors, stock returns approximately. Many natural phenomena follow the bell curve because of the Central Limit Theorem — the combined effect of many random variables tends toward normal distribution. In analytics, I'd verify normality with a histogram or Q-Q plot before applying parametric tests."
Q3: "Explain p-value to a non-technical person."
Answer: "Imagine you flip a coin 100 times and get 90 heads. The p-value answers: 'If the coin is fair, what's the chance of getting 90+ heads?' That probability is extremely low. So we conclude the coin is probably NOT fair. In analytics, p-value tells us whether a pattern in data is real or just random luck. We typically use 0.05 as the threshold — if p is below 5%, we trust the result."
Q4: "What is the difference between Type I and Type II errors?"
Answer: "Type I is a false positive — concluding something has an effect when it doesn't. Like launching a campaign nationwide thinking it increased conversions, when the improvement was just random. Type II is a false negative — failing to detect a real effect. Like killing a campaign that actually was working. In practice, which error is worse depends on the business context — in medicine, Type II (missing a disease) is worse; in spam filters, Type I (blocking legitimate email) is worse."
Q5: "What is the Central Limit Theorem?"
Answer: "CLT states that regardless of the underlying population distribution, the sampling distribution of the mean approaches a normal distribution as sample size grows — typically n ≥ 30 is sufficient. It's the reason we can use z-tests, t-tests, and confidence intervals even when our raw data isn't normally distributed. It's the foundation of inferential statistics."
Q6: "Explain correlation vs causation with an example."
Answer: "Correlation means two variables move together; causation means one causes the other. For example, there's a strong correlation between shoe size and reading ability in children — but bigger feet don't cause better reading. The confounding variable is age: older kids have bigger feet AND read better. To establish causation, we need controlled experiments like randomized A/B tests."
Q7: "You ran an A/B test and got p = 0.06. What do you do?"
Answer: "At the traditional 0.05 threshold, this is not statistically significant, so I wouldn't claim a conclusive result. However, I wouldn't just dismiss it either. I'd consider: (1) Is the effect size practically meaningful? (2) Should we run the test longer to increase sample size? (3) In some business contexts, 90% confidence may be acceptable. I'd present the result as 'directionally positive but not conclusive,' and recommend extending the test."
Q8: "What is statistical power?"
Answer: "Power is the probability of correctly detecting an effect when it exists — mathematically, Power = 1 - β. Standard target is 80%. Highe