Define reliability, including the different types and how they are assessed.A complete and adequate assessment of validity must include both theoretical and empirical approaches. While it may be a reliable instrument, it is not a valid instrument to determine someone’s weight in pounds. In the reliability section, we discussed a scale that consistently reported a weight of 15 pounds for someone. Validity is specific to the appropriateness of the interpretations we wish to make with the scores.
Table 3.Reliability refers to whether an assessment instrument gives the same results each time it is used in the same setting with the same type of subjects. Therefore, validity coefficients, unlike reliability coefficients, rarely exceed r. However, a single test can never fully predict job performance because success on the job depends on so many varied factors. The larger the validity coefficient, the more confidence you can have in predictions made from the test scores.
The statistical choice often depends on the design and purpose of the questionnaire. There are different statistical ways to measure the reliability and validity of your questionnaire. Define validity, including the different types and how they are assessed.Reliability and validity are two very important qualities of a questionnaire. Reliability is a part of the assessment of validity.
Your clothes seem to be fitting more loosely, and several friends have asked if you have lost weight. If their research does not demonstrate that a measure works, they stop using it.As an informal example, imagine that you have been dieting for a month. Instead, they collect data to demonstrate that they work.
Psychologists consider three types of consistency: over time (test-retest reliability), across items (internal consistency), and across different researchers (inter-rater reliability). ReliabilityReliability refers to the consistency of a measure. In evaluating a measurement method, psychologists consider two general dimensions: reliability and validity. But if it indicated that you had gained 10 pounds, you would rightly conclude that it was broken and either fix it or get rid of it.
This is typically done by graphing the data in a scatterplot and computing Pearson’s r. Clearly, a measure that produces highly inconsistent scores over time cannot be a very good measure of a construct that is supposed to be consistent.Assessing test-retest reliability requires using the measure on a group of people at one time, using it again on the same group of people at a later time, and then looking at test-retest correlation between the two sets of scores. This means that any good measure of intelligence should produce roughly the same scores for this individual next week as it does today. A person who is highly intelligent today will be highly intelligent next week. For example, intelligence is generally thought to be consistent across time. Test-retest reliability is the extent to which this is actually the case.
Figure 5.2 Test-Retest Correlation Between Two Sets of Scores of Several College Students on the Rosenberg Self-Esteem Scale, Given Two Times a Week ApartAgain, high test-retest correlations make sense when the construct being measured is assumed to be consistent over time, which is the case for intelligence, self-esteem, and the Big Five personality dimensions. In general, a test-retest correlation of +.80 or greater is considered to indicate good reliability. Pearson’s r for these data is +.95.
In general, all the items on such measures are supposed to reflect the same underlying construct, so people’s scores on those items should be correlated with each other. Internal ConsistencyA second kind of reliability is internal consistency, which is the consistency of people’s responses across the items on a multiple-item measure. So a measure of mood that produced a low test-retest correlation over a period of a month would not be a cause for concern. The very nature of mood, for example, is that it changes.
Validity And Reliability Of Instruments Series Of Bets
One approach is to look at a split-half correlation. This measure would be internally consistent to the extent that individual participants’ bets were consistently high or low across trials.Like test-retest reliability, internal consistency can only be assessed by collecting and analyzing data. For example, people might make a series of bets in a simulated game of roulette as a measure of their level of risk seeking. This is as true for behavioural and physiological measures as for self-report measures. If people’s responses to the different items are not correlated with each other, then it would no longer make sense to claim that they are all measuring the same underlying construct.
Pearson’s r for these data is +.88. For example, Figure 5.3 shows the split-half correlation between several university students’ scores on the even-numbered items and their scores on the odd-numbered items of the Rosenberg Self-Esteem Scale. Then a score is computed for each set of items, and the relationship between the two sets of scores is examined.
Cronbach’s α would be the mean of the 252 split-half correlations. For example, there are 252 ways to split a set of 10 items into two sets of five. Conceptually, α is the mean of all possible split-half correlations for a set of items. Figure 5.3 Split-Half Correlation Between Several College Students’ Scores on the Even-Numbered Items and Their Scores on the Odd-Numbered Items of the Rosenberg Self-Esteem ScalePerhaps the most common measure of internal consistency used by researchers in psychology is a statistic called Cronbach’s α (the Greek letter alpha).
To the extent that each participant does in fact have some level of social skills that can be detected by an attentive observer, different observers’ ratings should be highly correlated with each other. Then you could have two or more observers watch the videos and rate each student’s level of social skills. For example, if you were interested in measuring university students’ social skills, you could make video recordings of them as they interacted with another student whom they are meeting for the first time. Inter-rater reliability is the extent to which different observers are consistent in their judgments. Interrater ReliabilityMany behavioural measures involve significant judgment on the part of an observer or a rater. Again, a value of +.80 or greater is generally taken to indicate good internal consistency.
There has to be more to it, however, because a measure can be extremely reliable but have no validity whatsoever. When a measure has good test-retest reliability and internal consistency, researchers should be more confident that the scores represent what they are supposed to. But how do researchers make this judgment? We have already considered one factor that they take into account—reliability. ValidityValidity is the extent to which the scores from a measure represent the variable they are intended to. Interrater reliability is often assessed using Cronbach’s α when the judgments are quantitative or an analogous statistic called Cohen’s κ (the Greek letter kappa) when they are categorical. In this case, the observers’ ratings of how many acts of aggression a particular child committed while playing with the Bobo doll should have been highly positively correlated.
Here we consider three basic kinds: face validity, content validity, and criterion validity. The fact that one person’s index finger is a centimetre longer than another’s would indicate nothing about which one had higher self-esteem.Discussions of validity usually divide it into several distinct “types.” But a good way to interpret these types is that they are other kinds of evidence—in addition to reliability—that should be taken into account when judging the validity of a measure. Although this measure would have extremely good test-retest reliability, it would have absolutely no validity.
0 kommentar(er)
