Overview

Significance Testing (Sig Test) helps determine whether differences between survey results are statistically meaningful or simply due to random variation. 

By enabling Sig Test, users can identify which segments, groups, or responses perform significantly higher or lower than others, making it easier to draw confident conclusions from survey data.

Sig Test is available in both Crosstabs and Survey Metrics, providing a consistent statistical analysis experience across reporting modules.

When to use it:

Use Sig Test when comparing survey results across segments, demographics, locations, or any other grouped data to determine whether observed differences are statistically significant.

Key Use Cases

• Compare survey segments statistically

• Identify significantly higher or lower performing groups

• Validate trends before making decisions

• Analyze differences between customer segments

• Compare results against overall totals

• Improve confidence in survey insights

Accessing Sig Test

Sig Test can be enabled directly from the reporting interface.

Crosstabs

1. Open a Crosstab report.

2. Switch to Table View.

3. Enable the Sig Test toggle from the toolbar.
   

Survey Metrics

1. Open the Survey Metrics report.

2. Enable the Sig Test toggle where supported.

Sig Test is supported only in table-based views. It is not available in unsupported visualizations.

Confidence Levels

Sig Test allows users to control the confidence level used for statistical calculations.

Available options include:

• 80%

• 90%

• 95% (Default)

• 99%
  

95% confidence is selected by default.

Users can select up to two confidence levels simultaneously.

When confidence levels are changed, significance calculations refresh automatically.

Use a 95% confidence level for most business decisions, as it provides a good balance between confidence and sensitivity.

Comparison Methods

Sig Test supports two comparison approaches.

Compare vs Total

Each column is compared against the Total column.

This helps determine whether a segment performs significantly differently from the overall dataset.

Compare Each Column

Each column is compared against every other column in the table.

This helps identify which segments significantly outperform or underperform one another.

Results update automatically whenever the comparison method changes.

Significance Direction

When using Compare Each Column, users can choose which type of significance to display.

Available options include:

• Higher

• Lower

Higher

Displays values that are significantly higher than comparison columns.

Lower

Displays values that are significantly lower than comparison columns.

Minimum Base

The Minimum Base setting controls the minimum sample size required for significance testing.

Any row or column that does not meet the minimum base requirement is excluded from significance calculations.

Results refresh automatically whenever the minimum base value is updated.

Excluded values continue to appear in the report but do not display significance indicators.

Letter Indicators

Letter indicators help users understand which columns differ significantly from one another.

When enabled:

• Each column is assigned a letter (A, B, C, D, etc.)

• Letters appear above column headers

• Letters beside values indicate which columns the result is significantly different from

Example:

• A-C = Significant at 95%

• a-c = Significant at 90%
  

This allows users to quickly identify significant differences without manually reviewing comparisons.

Uppercase letters represent higher confidence levels, while lowercase letters represent lower confidence levels when multiple confidence levels are selected.

Arrow Indicators & Cell Highlighting

Sig Test can visually highlight significant differences using arrows and cell colors.

Higher Values

• Green upward arrow

• Green cell highlighting

Lower Values

• Red downward arrow

• Red cell highlighting

These indicators improve readability and make significant results easier to identify.

Users can independently enable or disable:

• Arrow Indicators

• Letter Indicators

without affecting the underlying significance calculations.

Survey Metrics Support

Sig Test is available within Survey Metrics in addition to Crosstabs.

The same functionality is supported across both modules, including:

• Confidence Levels

• Comparison Methods

• Minimum Base

• Letter Indicators

• Arrow Indicators

• Significance Highlighting

Significance calculations automatically update whenever:

• Filters change

• Date ranges change

• Comparison settings change

• Confidence levels change

• Minimum base values change
  

This ensures consistency between Survey Metrics and Crosstab reporting.

Important Notes

• Sig Test is available only in supported table views.

• Transpose and Sig Test cannot be used simultaneously.

• Enabling Transpose automatically disables Sig Test.

• Previous Sig Test settings are restored when Transpose is turned off.

• Sig Test never changes actual report values; it only adds statistical indicators.

• Multiple confidence levels can be selected, with a maximum of two.

Best Practices

• Use 95% confidence for standard reporting.

• Set an appropriate minimum base before interpreting results.

• Compare against Total when validating segment performance.

• Use Compare Each Column for detailed segment-to-segment analysis.

• Review tooltip explanations when interpreting significant differences.

• Combine arrows, highlights, and letters for the clearest analysis experience.

Statistical significance helps validate whether differences are meaningful, reducing the risk of making decisions based on random fluctuations in survey data.