Home / RBT Study Guide / Data Collection and Graphing / Data reliability, procedural fidelity and summarizing data

Domain A · Data Collection and Graphing · Tasks A.3, A.6, A.8

Data reliability, procedural fidelity and summarizing data

Reliable data means the numbers match what actually happened. Procedural fidelity means the plan was run as written. When either one slips, the supervisor ends up making treatment decisions from a graph that is not true.

Last reviewed November 10, 2026 against the BACB RBT Test Content Outline (3rd ed.)

Why unreliable data is a risk

Task A.8 of the RBT Test Content Outline asks you to describe the risks of unreliable data collection and poor procedural fidelity. The core risk is simple: your supervisor makes treatment decisions from the graph. If the numbers are wrong, the decisions can be wrong.

Data usually become unreliable for ordinary reasons: recording from memory after the session, a vague definition, guessing at interval timing, or being too busy to tally.

Procedural fidelity

Procedural fidelity (also called treatment integrity) is how closely the plan is implemented as written. Giving a prompt the plan does not call for, skipping reinforcement, or giving attention when the plan says to withhold it are all fidelity errors.

Poor fidelity has its own risks. If the plan was not run as written, no one can say whether the plan works. Inconsistent implementation can also slow learning or accidentally reinforce the behavior the plan targets.

In session

Leo's plan says to deliver a token after every correct response. His RBT, juggling materials, delivers tokens about half the time. Leo's accuracy stalls. If the supervisor does not know about the missed tokens, she may decide the token system does not work for Leo, when it was never delivered as designed. The RBT's job is to say so.

Interobserver agreement, drift and reactivity

The usual check on data is interobserver agreement (IOA): two people record the same behavior, in the same session, independently, and then compare.

A common convention treats 80% or higher as acceptable, but your supervisor sets the standard for each program. Low agreement does not show who is wrong. It shows the definition or the recording needs attention.

Two threats come up often:

What you do about it:

Common mix-up

Drift is a slow change in the observer's scoring. Reactivity is a change in behavior caused by being observed. If the scenario says "gradually" or "over several weeks," think drift. If it says "when the supervisor was in the room," think reactivity.

Permanent product recording

Task A.3 covers permanent product recording: measuring a behavior by the lasting result it leaves, after the behavior is over. You count the product, not the action. Examples include worksheet problems completed, toys in the bin, beads strung, or words typed.

The advantage is that you do not have to watch the behavior, so you can teach during the work period and the product can be checked later by a second person. The limits are just as important. The behavior has to leave a consistent product, and you need to be confident the client produced it. A finished worksheet does not tell you whether a peer helped or how long it took.

Calculating and summarizing data

Task A.6 names three ways to summarize raw data. Each turns a session's tallies or times into one number that can go on the graph.

SummaryFormulaWorked example
RateCount divided by observation time12 hits in 30 minutes: 12 / 30 = 0.4 per minute, or 24 per hour
Mean durationTotal duration divided by number of occurrencesTantrums of 3, 5 and 10 minutes: 18 / 3 = 6 minutes per tantrum
PercentagePart divided by whole, times 10014 correct of 20 trials: 14 / 20 = 0.70, or 70%

Use rate when session lengths differ, because counts from a 30-minute and a 60-minute session cannot be compared directly. Use mean duration when the length of each episode matters. Use percentage for trial-based teaching and for interval data, where the result is the percentage of intervals scored. The measures behind these numbers are explained under continuous measurement.

On the exam

For calculation items, write the formula before touching the numbers, and check the units. Rate answers carry a time unit (per minute, per hour). A mean duration divides by the number of episodes, not by session length. A percentage can never be above 100, so an IOA answer over 100 means the division was upside down.

When data or fidelity problems happen, how and when you tell your supervisor falls under documentation and reporting.

Practice questions

Original questions in the style of the exam. Pick an answer before you open the explanation.

1. Hitting is defined as forceful contact with an open or closed hand. Over several months, Priya gradually starts counting light taps as hits too. What is this an example of?

  1. Observer reactivity
  2. Observer drift
  3. Treatment drift
  4. Observer expectancy
Show answer

B. Observer drift

Observer drift is a gradual, unintended shift in how an observer applies a definition. Reactivity is a change in behavior caused by being observed, and treatment drift is about how the intervention is delivered, not how the behavior is scored.

2. Two observers independently count Sam's requests during the same session. One records 16 and the other records 20. What is the total count interobserver agreement?

  1. 64 percent
  2. 90 percent
  3. 125 percent
  4. 80 percent
Show answer

D. 80 percent

Total count agreement is the smaller count divided by the larger count, times 100: 16 / 20 = 0.80, or 80 percent. Dividing the larger by the smaller gives 125 percent, which cannot be right because agreement never exceeds 100 percent.

3. After a session, an RBT counts how many math problems Mia completed on her worksheet without having watched her work. Which procedure is this?

  1. Permanent product recording
  2. Momentary time sampling
  3. Whole interval recording
  4. Continuous event recording
Show answer

A. Permanent product recording

Permanent product recording measures the lasting result of a behavior after it has occurred. Continuous event recording would mean tallying each response as it happened, which requires watching the behavior live.

Common questions

What is procedural fidelity in ABA?

Procedural fidelity, also called treatment integrity, is the extent to which a plan is carried out exactly as written. When fidelity is poor, the team cannot tell whether the plan itself works, and the client may make less progress or have problem behavior reinforced by accident.

How do you calculate interobserver agreement (IOA)?

For a simple total count check, divide the smaller count by the larger count and multiply by 100. If one observer counts 18 and the other counts 20, IOA is 18 / 20 x 100 = 90 percent. For interval data, divide the number of intervals where both observers agreed by the total number of intervals and multiply by 100.

What is permanent product recording?

It is measuring a behavior by the lasting result it leaves, such as completed worksheet problems, dishes put away or items assembled. The observer does not need to see the behavior happen. It only fits behaviors that reliably leave a product that can be tied to the client.