Intraclass Correlation (ICC) Calculator for Excel

Calculate ICC coefficients with confidence intervals for your Excel data

Data Format

Single Measures

Average Measures

Model Type

Number of Subjects (k)

Number of Ratings per Subject (n)

Mean Square Between (MSB)

Mean Square Within (MSW)

Mean Square for Raters (MSR) – Required for Two-Way Models

Confidence Level

Comprehensive Guide: How to Calculate Intraclass Correlation in Excel

Intraclass Correlation Coefficient (ICC) is a statistical measure used to assess the reliability of ratings or measurements by quantifying the degree of agreement between different raters or measurement methods. This guide provides a step-by-step explanation of how to calculate ICC in Excel, including the necessary formulas, data preparation techniques, and interpretation guidelines.

Understanding Intraclass Correlation (ICC)

ICC values range from 0 to 1, where:

0.00-0.50: Poor reliability
0.50-0.75: Moderate reliability
0.75-0.90: Good reliability
0.90-1.00: Excellent reliability

There are several ICC models, each appropriate for different research designs:

ICC(1,1): One-way random effects (single rater)
ICC(1,k): One-way random effects (average of k raters)
ICC(2,1): Two-way random effects (single rater)
ICC(2,k): Two-way random effects (average of k raters)
ICC(3,1): Two-way mixed effects (single fixed rater)
ICC(3,k): Two-way mixed effects (average of k fixed raters)

Preparing Your Data in Excel

Before calculating ICC, you need to organize your data properly in Excel:

Column Structure:
- First column: Subject IDs
- Subsequent columns: Measurements from each rater
Data Requirements:
- At least 2 raters per subject
- At least 5 subjects (more is better for reliable estimates)
- No missing values (use data imputation if necessary)

Example Data Structure:

Subject	Rater 1	Rater 2	Rater 3
1	8.2	7.9	8.5
2	6.7	6.5	6.8
3	9.1	8.9	9.0
4	7.5	7.3	7.6
5	8.8	8.7	8.9

Step-by-Step ICC Calculation in Excel

While Excel doesn’t have a built-in ICC function, you can calculate it using ANOVA tables. Here’s how:

Perform One-Way ANOVA:
- Go to Data → Data Analysis → Anova: Single Factor
- Input range: Select all your data (excluding subject IDs)
- Check “Labels in First Row” if applicable
- Set alpha to 0.05
- Click OK to generate the ANOVA table
Extract Key Values:
From the ANOVA output, note these values:
- Mean Square Between (MSB)
- Mean Square Within (MSW)
- For two-way models: Mean Square for Raters (MSR)

Calculate ICC Using Formulas:

Use these formulas based on your model:

ICC Model	Formula	When to Use
ICC(1,1)	(MSB – MSW) / (MSB + (k-1)*MSW)	Each subject rated by different raters, single measurement
ICC(1,k)	(MSB – MSW) / MSB	Each subject rated by different raters, average measurement
ICC(2,1)	(MSB – MSW) / (MSB + (k-1)MSW + k(MSR – MSW)/n)	Same raters rate all subjects, single measurement
ICC(2,k)	(MSB – MSW) / (MSB + (MSR – MSW)/n)	Same raters rate all subjects, average measurement

Where:

k = number of raters
n = number of subjects

Calculate Confidence Intervals:
For 95% confidence intervals, use these approximate formulas:
- Lower bound: ICC – (1.96 × SE)
- Upper bound: ICC + (1.96 × SE)
Where SE (Standard Error) can be estimated as:

SE = √[2*(1-ICC)²*(1+(k-1)*ICC)² / (k²*(n-1)*(k-1))]

Practical Example: Calculating ICC(2,1) in Excel

Let’s work through a complete example with sample data:

Sample Data (10 subjects, 3 raters):

Subject	Rater 1	Rater 2	Rater 3
1	78	76	80
2	65	67	64
3	82	80	83
4	70	72	69
5	90	88	91
6	75	74	76
7	68	70	67
8	85	84	86
9	72	73	71
10	88	87	89

Perform Two-Way ANOVA:
- Go to Data → Data Analysis → Anova: Two-Factor With Replication
- Input range: B1:D11 (including headers)
- Rows per sample: 10 (number of subjects)
- Columns per sample: 3 (number of raters)
- Click OK
ANOVA Output:
From the ANOVA table, extract:
- MSB (Mean Square for Subjects) = 420.67
- MSW (Mean Square for Error) = 4.22
- MSR (Mean Square for Raters) = 2.11
Calculate ICC(2,1):
Using the formula:

ICC = (MSB – MSW) / [MSB + (k-1)*MSW + k*(MSR – MSW)/n]

Plugging in the values:

ICC = (420.67 – 4.22) / [420.67 + (3-1)*4.22 + 3*(2.11 – 4.22)/10]

ICC = 416.45 / [420.67 + 8.44 + 3*(-0.211)/10]

ICC = 416.45 / 428.99 = 0.9708
Calculate 95% Confidence Interval:
First calculate SE:

SE = √[2*(1-0.9708)²*(1+(3-1)*0.9708)² / (3²*(10-1)*(3-1))]

SE = √[2*(0.0292)²*(2.9416)² / (9*9*2)] = 0.0089

Then calculate CI:

Lower bound: 0.9708 – (1.96 × 0.0089) = 0.9534

Upper bound: 0.9708 + (1.96 × 0.0089) = 0.9882

Interpreting ICC Results

The ICC value of 0.9708 (95% CI: 0.9534-0.9882) indicates excellent reliability. Here’s how to interpret different ICC ranges in research contexts:

ICC Range	Reliability	Research Implications	Example Applications
0.00-0.50	Poor	Unacceptable for most research purposes. Measurement tool needs significant revision.	Pilot studies, exploratory research
0.50-0.75	Moderate	May be acceptable for some exploratory research but not for definitive conclusions.	Social science surveys, preliminary clinical assessments
0.75-0.90	Good	Generally acceptable for most research purposes. Some room for improvement.	Clinical measurements, psychological assessments, educational testing
0.90-1.00	Excellent	Highly reliable. Suitable for critical decisions and high-stakes testing.	Diagnostic tests, standardized examinations, medical imaging interpretations

For our example with ICC = 0.9708:

The measurement system demonstrates excellent reliability
The 95% confidence interval (0.9534-0.9882) doesn’t include values below 0.90, confirming high reliability
This level of reliability would be suitable for clinical decision-making or high-stakes assessments
The narrow confidence interval suggests the estimate is precise

Common Mistakes to Avoid When Calculating ICC in Excel

Using the Wrong ICC Model:
Selecting an inappropriate ICC model can lead to incorrect interpretations. Always consider:
- Whether raters are fixed or random effects
- Whether you’re interested in single measurements or average measurements
- The structure of your study design
Incorrect Data Structure:
Common data organization errors include:
- Missing values (use data imputation or complete case analysis)
- Inconsistent number of ratings per subject
- Mislabeled columns or rows
Misinterpreting ANOVA Output:
Key mistakes when reading ANOVA tables:
- Confusing MSB with MSR
- Using the wrong mean square values in ICC formulas
- Ignoring the degrees of freedom when calculating confidence intervals
Inadequate Sample Size:
Small sample sizes can lead to:
- Unstable ICC estimates
- Wide confidence intervals
- Increased risk of Type II errors
General recommendations:
- Minimum 5 subjects (10+ preferred)
- Minimum 2 raters (3+ preferred)
- For publication-quality results, aim for 30+ subjects and 5+ raters
Ignoring Assumptions:
ICC calculations assume:
- Normal distribution of measurements
- Homogeneity of variance
- Independence of observations
Violations can be addressed by:
- Data transformation (for non-normal data)
- Using robust ICC estimators
- Checking for outliers

Advanced Techniques for ICC Analysis in Excel

For more sophisticated ICC analysis in Excel, consider these advanced approaches:

Automating ICC Calculations with VBA:

Create a custom VBA function to calculate ICC directly:

Function ICC_2_1(MSB As Double, MSW As Double, MSR As Double, k As Integer, n As Integer) As Double
    ICC_2_1 = (MSB - MSW) / (MSB + (k - 1) * MSW + k * (MSR - MSW) / n)
End Function

Usage: =ICC_2_1(B2, C2, D2, E2, F2) where cells contain the respective values

Creating ICC Calculation Templates:
Develop reusable Excel templates with:
- Pre-formatted data input areas
- Automatic ANOVA calculation setup
- ICC formula cells for different models
- Confidence interval calculations
- Interpretation guidance
Visualizing ICC Results:
Create informative charts to present ICC results:
- Bar charts showing ICC values with confidence intervals
- Bland-Altman plots for agreement analysis
- Scatter plots of rater agreements
Bootstrapping Confidence Intervals:
For more accurate CIs with small samples:
1. Resample your data with replacement (1,000+ times)
2. Calculate ICC for each resample
3. Use the 2.5th and 97.5th percentiles as CI bounds
Excel implementation:
- Use Data → Data Analysis → Sampling
- Create a macro to automate the resampling process
- Use PERCENTILE.EXC function for CI bounds

Comparing ICC with Other Reliability Measures

While ICC is the gold standard for reliability assessment, it’s helpful to understand how it compares to other common reliability metrics:

Metric	When to Use	Advantages	Limitations	Typical ICC Equivalent
Cronbach’s Alpha	Internal consistency for multi-item scales	Simple to calculate, widely understood	Assumes tau-equivalence, sensitive to number of items	ICC(3,k)
Kappa Statistic	Inter-rater agreement for categorical data	Accounts for chance agreement, good for nominal data	Only for categorical data, affected by prevalence	N/A (different scale)
Pearson Correlation	Relationship between two continuous variables	Familiar to most researchers, simple interpretation	Sensitive to systematic bias, doesn’t measure absolute agreement	Generally lower than ICC
Bland-Altman Limits	Assessing agreement between two measurement methods	Shows systematic bias, visual interpretation	Only for two raters/methods, no single summary statistic	Complementary to ICC
Standard Error of Measurement	Assessing measurement precision	Directly interpretable in original units, useful for individual assessments	Doesn’t provide relative reliability like ICC	Derived from ICC

Key differences between ICC and Cronbach’s Alpha:

ICC is appropriate when you have multiple raters measuring the same subjects
Cronbach’s Alpha is appropriate when you have multiple items measuring the same construct
ICC can handle different models (one-way, two-way, etc.) while Alpha assumes a single model
ICC provides information about both consistency and absolute agreement

Real-World Applications of ICC

ICC is used across various fields to assess reliability:

Medical Research:
- Assessing agreement between radiologists interpreting medical images
- Evaluating consistency of clinical ratings in psychiatric diagnoses
- Validating new diagnostic tools or measurement instruments
Example: A study of 5 radiologists evaluating 50 X-rays for pneumonia presence might find ICC(2,1) = 0.87, indicating good inter-rater reliability.
Psychology and Education:
- Evaluating consistency of essay grading between teachers
- Assessing reliability of psychological assessments
- Validating behavioral observation coding schemes
Example: For a new anxiety scale with 20 items rated by 4 clinicians on 100 patients, ICC(3,k) = 0.92 would indicate excellent internal consistency.
Sports Science:
- Assessing reliability of performance measurements
- Evaluating consistency of judging in subjective sports
- Validating new biomechanical measurement techniques
Example: In gymnastics judging with 7 judges scoring 20 routines, ICC(2,k) = 0.95 would show excellent agreement.
Market Research:
- Evaluating consistency of product quality ratings
- Assessing reliability of customer satisfaction surveys
- Validating sensory evaluation panels
Example: For a wine tasting panel with 5 experts rating 30 wines on a 100-point scale, ICC(1,k) = 0.89 would indicate good reliability.
Manufacturing and Quality Control:
- Assessing consistency of inspectors in quality control
- Evaluating reliability of measurement instruments
- Validating new inspection procedures
Example: For 10 inspectors measuring 50 components with calipers, ICC(2,1) = 0.97 would show excellent measurement system reliability.

Authoritative Resources on Intraclass Correlation

For more in-depth information about ICC and its calculation, consult these authoritative sources:

Frequently Asked Questions About ICC in Excel

Can I calculate ICC directly in Excel without ANOVA?
While ANOVA is the standard approach, you can use these alternative methods:
- Use the VAR.B and VAR.WS functions to calculate between-subject and within-subject variance
- Implement ICC formulas directly using these variance estimates
- For simple cases, use: = (VAR.B – VAR.WS) / (VAR.B + (k-1)*VAR.WS) for ICC(1,1)
However, the ANOVA method is generally more robust and recommended.

What’s the minimum sample size for reliable ICC estimates?

Sample size requirements depend on:

Expected ICC value (higher ICC requires smaller samples)
Number of raters (more raters allows smaller subject samples)
Desired confidence interval width

General guidelines:

Expected ICC	Number of Ratings per Subject	Minimum Subjects Recommended
0.50	2	50
0.50	3	30
0.75	2	20
0.75	3	15
0.90	2	10
0.90	3	8

How do I handle missing data when calculating ICC?
Options for dealing with missing data:
- Complete Case Analysis: Use only subjects with complete data (may introduce bias)
- Mean Imputation: Replace missing values with the mean of available ratings for that subject
- Multiple Imputation: Use Excel’s Data → Data Tools → Fill → Justify to create multiple imputed datasets
- Maximum Likelihood Estimation: Requires advanced statistical software
Recommendation: If <5% data is missing, complete case analysis is often acceptable. For 5-15% missing, use multiple imputation.
Can I calculate ICC for binary or categorical data in Excel?
ICC is typically used for continuous data. For categorical data:
- Use Cohen’s Kappa for nominal data (2 raters)
- Use Fleiss’ Kappa for nominal data (>2 raters)
- Use Weighted Kappa for ordinal data
- For binary data, consider Bland-Altman analysis or percentage agreement
Excel implementation:
- For Kappa: Use the formula = (Po – Pe) / (1 – Pe)
- Where Po = observed agreement, Pe = expected agreement by chance
How do I report ICC results in a research paper?
Follow these reporting guidelines:
- Specify the ICC model used (e.g., ICC(2,1))
- Report the ICC value with 2 decimal places
- Include 95% confidence intervals
- State the number of subjects and raters
- Describe the interpretation (e.g., “excellent reliability”)
- Mention any assumptions or limitations
Example reporting:

“Inter-rater reliability was assessed using two-way mixed-effects ICC(3,k) with absolute agreement. The ICC was 0.92 (95% CI: 0.88-0.95), indicating excellent reliability among the 5 raters evaluating 40 subjects.”

Conclusion

Calculating intraclass correlation in Excel provides researchers with a powerful tool to assess the reliability of their measurement systems. By following the step-by-step methods outlined in this guide, you can:

Properly structure your data for ICC analysis
Select the appropriate ICC model for your study design
Perform the necessary ANOVA calculations in Excel
Apply the correct ICC formulas to your data
Calculate and interpret confidence intervals
Avoid common pitfalls in ICC analysis
Present your results professionally

Remember that while Excel provides a accessible platform for ICC calculations, specialized statistical software may be preferable for complex designs or large datasets. The key to meaningful ICC analysis lies in careful study design, appropriate model selection, and proper interpretation of results within the context of your specific research questions.

For researchers seeking to establish the reliability of their measurement instruments, mastering ICC calculation in Excel represents a valuable skill that can enhance the quality and credibility of their work across diverse fields of study.

Calculate Intraclass Correlation In Excel