A Few Technical Notes: Standards for
Reliability & Validity
Criterion-related validity
* Each scale correlates .20 or higher with at least one relevant criterion variable (i.e., scores on each scale can predict other measures).
Construct validity
* Given conceptual descriptions of the constructs measured by the scales, subject matter experts are able to sort every item into its correct scale (i.e., experts in this field of research agree on the items that belong in a given scale).
* Each scale correlates .70 or higher with other measures accepted as being valid measures of the same construct (i.e., scale scores match the scores obtained from other similar scales).
* Each scale correlates .50 or lower with any scale that does not measure the same construct (i.e., scale scores do not match scores obtained from other different scales).
* Each item in a scale correlates more highly with that scale than with any other scale (i.e., items in a scale predict that scale's score better than items not in that scale).
* Each item in a scale correlates more highly with that scale than does any other item not in that scale (i.e., items not in a scale do not predict that scale's score as well as items in that scale).
Content validity
* Each scale represents a set of beliefs, attitudes, or behaviors important to the study (i.e., each scale measures a meaningful area germane to the research).
* Each scale is seen by experts as within the scope of interest (i.e., experts in this field of research agree that each scale is relevant to the study).
Across-item consistency
* Each scale has an alpha coefficient of .70 or higher (i.e., scores on items in a scale are statistically consistent with each other).
|
|
organizations, institutions, businesses, and policy and opinion makers can develop strategies informed by a profound understanding of current American values and aspirations.
Survey Sampling, Methods, and Standards
The In Our Own Words 2000 research team completed over 1,600 interviews with American householders over 18 years of age. The sample of respondents was closely controlled by age, gender, and geography for each of the 12 standard U.S. Census regions. This is a degree of representativeness seldom achieved in survey research. The overall margin of error is ±1.5% to ± 2.4% at a 95% confidence level. Each interview took approximately 43 minutes and covered over 210 questions on beliefs, values, and attitudes concerning society, work, spirituality, media, health, and lifestyle.
A team of social scientists and other research professionals was responsible for the design, implementation, analysis, review, and reporting of this major research program. A professional research company employing highly trained telephone interviewers gathered the data using a computer-aided interviewing system. The proper use of well-constructed and validated assessment instruments is governed by the standards established by The American Psychological Association, The National Council on Measurement in Education, and The American Educational Research Association. These standards arise from traditionally accepted psychometric concepts regarding relevant forms of reliability and validity. The usefulness and credibility of IOOW 2000 research data and analyses are supported by the project's rigorous adherence to these standards, which are outlined in the opposite table.
Comparative Research
The IOOW 2000 Research Program also includes comparative summaries and reviews of survey data and other research across many related studies of American values and beliefs concerning ethics, spirituality, and the Internet, which have been conducted by other research foundations, pollsters, academic institutions, and independent researchers. This additional body of work is summarized in this executive briefing and covered more extensively in the IOOW 2000 full report.
|
