The Family Relations Institute, Inc. (FRI) offers a data management service for coding and classifying DMM assessments of attachment for researchers.
FRI employs reliable coders who are entirely blind to the purposes of the study and identity of the participants. Using these coders, FRI delivers to the researcher a set of data with measures of both coders’ reliability (to an external standard) and inter-coder agreement (on the data within the study).
FRI codes and classifies the following assessments:
- Infant CARE-Index (ICI) for infants aged 0-15 months.
- Toddlers CARE-Index (TCI) for children aged 15-60 months.
- Strange Situation Procedure (SSP) for children aged 11-15(17) months.
- Preschool Assessment of Attachment (PAA) for children aged 15(17)-60 months.
- School-Age Assessment of Attachment (SAA) for children aged 6-13 years.
- Transition to Adulthood Attachment Interview (TAAI) between 16 and 25 years of age.
- Adult Attachment Interview (DMM-AAI) for adults.
Who needs FRI to manage the coding of their data?
If you have used DMM attachment assessments in your research, FRI’s data management service offers the best way to obtain methodologically sound data. FRI selects skilled coders (Level I and II reliability) who are competent in the culture of your data and manage the coding to ensure the coders maintain agreement throughout the coding process.
Some researchers assume that they must train as a coder and code their own research assessments. This is not the case because coding one’s own data can render it unpublishable for methodological violations. When a single person (usually the principal investigator) defines the hypothesis, sets up the study, collects cases, and also codes the data, the coding cannot be blind and findings are compromised and made invalid. We recommend that researchers take the assessment course purely to understand the data that they will receive. We do not recommend that the researcher attempt to achieve reliability and strongly advise against them of the coding of their own data.
Training to reach reliability in the DMM assessments is a process that depends on many factors, including the trainees’ availability of time to do all the required practice and homework. For many researchers, the focus is on managing their projects in specified time frames rather than on gradually developing observational skills; achieving reliability under these conditions becomes a stressful and often unsuccessful process. Engaging FRI to manage the coding yields sound data that can both answer the research question and also meet publication standards.
FRI’s data management service is subject to the following specifications:
- FRI manages the coding of attachment assessments only for publishable research. To ensure that our limited resources are assigned to important and publishable studies, proposals for the coding/classification of video or transcribed assessments will be evaluated by FRI’s Director (currently Dr Andrea Landini). If the study meets research and publication standards, it will be evaluated by the Director of research to see if FRI can meet the time constraints of the study. This is a particular problem for doctoral students whose tight timelines may not be achievable by FRI. If FRI accepts the responsibility to produce the data, the process will be managed by FRI’s Director of Research (currently Dr. Patricia Crittenden).The Director evaluates the study’s goals and methodology to determine whether or not the question being asked can be answered by the study and the design meets publishable standards. Typical requirements include a sufficient sample size for the question, appropriate comparison groups, and expectation of a range of codings that will permit a statistically significant finding. For example, the ICI and TCI have 15-point scales (0-14), but if the sample is entirely child protection cases, that range may effectively be limited to 7 or fewer points; finding statistically significant differences in such a narrow range can be very difficult.If the study is accepted for data management and coding, further communication is with the Director of Research and does not include any information about the study (thus keeping the coding procedure completely blinded.)
- FRI uses coders who have certified levels of reliability, recently obtained and culturally suitable for the populations included in the study. FRI uses coders who have currently valid reliability certificates at Level I (90% or better reliability) or II (85% or greater reliability). Reliability is based on a reliability test that includes cases using the full range of variation of the assessment (from exemplary through normal to risk and high risk). If the data to be coded uses a restricted range (for example the data reflect only a risk sample), then agreement will not be as high, and, depending upon the degree of restriction, may not be adequate. Researchers need to consider this before contracting for coding and in writing up their results for publication.
All FRI certificates of reliability have a closing date.
Each certificate defines which cultures the coder is reliable to code. Therefore, for each study, the coders selected will be reliable on cultures that are compatible with the sample to be coded.FRI offers 4 levels of reliability, of which Levels I and II are suitable for research
The levels are:
Level I: Forensic reliability. Person-specific record-keeping or reporting
Level II: Research reliability. Coding data
Level III: Screening for risk. Not suitable for research or publication
Level IV: Pre-screening when accompanied by a reliable coder. Not suitable for research or publication
- FRI establishes inter-coder agreement before the selected coders begin coding a sample.
When FRI is responsible for coding a sample, coders are engaged with a current Level I or II reliability certificate, but are not paid until they demonstrate agreement on the sample to be coded. Ten agreement videos or transcripts are selected by the researcher to reflect the range of cases in the sample, and are coded before independent coding is undertaken. FRI provides an agreement coding for the 10 cases as the standard for the coders. The agreement coding reflects the best understanding of the video/transcript after FRI and each coder has provided an initial coding; the agreement coding is not a compromised coding, but rather one that reflects at least three inputs and is the final best coding. This process ensures that the coders are both up to speed and also have a range of coded exemplars from within the sample being coded. It is crucial that the primary agreement coder (FRI) be totally blind to the nature of the study (see #1 above) so that agreement is based solely on the study cases and not preconceptions about the sample.
- FRI uses periodic random agreement checks. When FRI manages the coding of a set of data, the coders know that some of their cases will be coded by someone else, but they do not know how many or which cases. FRI’s Director of Research keeps track of their agreement and, when the differences are too great, coders are asked to resolve the difference and, if they cannot, the Director of Research will. In asking coders to resolve a disagreement, coders will send their coding and explanations to the Director of Research, who will distribute their comments to all the coders – without identifying the coders. That way, personality issues do not affect the process of reaching agreement.
- For small samples (for example 100 or fewer assessments), FRI uses 20% for agreement codings. This rate for agreement codings is chosen because it fits criteria usually required by reviewers for publication. For larger samples, FRI and the research will decide what size agreement sample is best.
- FRI asks that all cases to be coded be ready before coding begins and that cases from different groups in the study design (for example, pre- and post-intervention) be mixed in a way that coders cannot discern. Researchers often want their material coded as the data comes in; this can lead to coder drift and, if the sample is a pre/post intervention study, it can distort the findings. Coding in a concentrated period of time after establishing coder agreement minimizes coder drift and maximizes the uniformity of coding across the sample.
For enquiries about FRI coding for research, contact us.