Design Atlas
한국어
⟳ Design process

A/B test

A/B 테스트

Show two versions of the same screen to randomly split real users and compare the metric difference statistically.

Also known as: Split test분할 테스트
···
html
<div class="stage">
  <div class="tag">example data</div>
  <div class="cols">
    <div class="v">
      <div class="hd">A</div>
      <button class="mock a">Buy now</button>
      <div class="metric"><span id="ca">0</span>%</div>
      <div class="meter"><span id="ma"></span></div>
    </div>
    <div class="v">
      <div class="hd">B</div>
      <button class="mock b">Get it today</button>
      <div class="metric win"><span id="cb">0</span>%</div>
      <div class="meter"><span id="mb" class="win"></span></div>
    </div>
  </div>
</div>
css
.stage{width:94%;height:88%;display:flex;flex-direction:column;gap:8px}
.tag{align-self:flex-end;font:700 9px/1 monospace;color:var(--muted);letter-spacing:.04em}
.cols{flex:1;display:flex;gap:10px}
.v{flex:1;border:1px solid var(--line);border-radius:10px;background:var(--surface);display:flex;flex-direction:column;
  align-items:center;justify-content:center;gap:8px;padding:6%}
.hd{font:700 11px/1 monospace;color:var(--muted)}
.mock{border:none;border-radius:8px;padding:8px 14px;font:700 11px/1 sans-serif;color:#fff}
.mock.a{background:var(--muted)}
.mock.b{background:var(--accent)}
.metric{font:800 20px/1 monospace;color:var(--fg)}
.metric.win{color:var(--accent-3)}
.meter{width:80%;height:6px;border-radius:3px;background:var(--line);overflow:hidden}
.meter span{display:block;height:100%;width:0;background:var(--muted)}
.meter span.win{background:var(--accent-3)}
js
const ca = document.getElementById('ca');
const cb = document.getElementById('cb');
const ma = document.getElementById('ma');
const mb = document.getElementById('mb');
function run() {
  let a = 0, b = 0;
  ma.style.width = '0%'; mb.style.width = '0%';
  const id = setInterval(() => {
    a = Math.min(3.2, a + Math.random() * 0.35);
    b = Math.min(5.8, b + Math.random() * 0.55);
    ca.textContent = a.toFixed(1);
    cb.textContent = b.toFixed(1);
    ma.style.width = (a / 6 * 100) + '%';
    mb.style.width = (b / 6 * 100) + '%';
    if (a >= 3.2 && b >= 5.8) clearInterval(id);
  }, 120);
}
run();
setInterval(run, 3600);

An A/B test answers with an experiment instead of an opinion. Version A (control) and version B (variant) are shown to randomly split traffic at the same time, and a metric like click-through or conversion rate is compared. "At the same time" is the crucial part — run A for a month and then B for a month and seasonality or trend gets tangled in with whatever the variant actually changed.

A slightly higher number doesn't automatically mean a winner. With a small sample, random noise easily looks like a real effect, so a result only counts once it clears statistical significance — the odds this gap happened by chance are low enough to trust.

Where usability testing is qualitative and tells you why people struggle, A/B testing is quantitative and tells you which version performs better. Teams typically narrow candidates with usability testing first, then confirm the winner against real traffic with an A/B test.

The counter below is example data to illustrate the concept — not a real experiment result.

When to use

Use it for a final call once you're down to two candidates and have enough traffic. With low traffic it takes a long time to reach significance, so consider a qualitative method first.