Communication Research, June 1997 v24 n3 p197(37)
The role of political sophistication in learning from news;
measuring schema development.
June Woong Rhee; Joseph N. Cappella.
Full Text: COPYRIGHT 1997 Sage Publications Inc. Measuring Schema Development
Accounting for
learning from news has focused on media attention, media gratification,
content-specific exposure,
and prior knowledge.
One measure of prior knowledge is political sophistication, measuring components
of both
knowledge and
ideology. Two studies evaluated the reliability, concurrent validity, and
comparative predictive
validity of
political sophistication in comparison to those of exposure, attention,
and civics knowledge. Study 1
focused on the
1993-1994 health care reform debate. Those higher in political sophistication
learned more from
broadcast and
print news, and had more differentiated constructs and higher quality arguments
about health policy
issues in open-ended
essays. These findings were replicated in Study 2, which was conducted
within a simulated
mayoral election
campaign. The results suggest that political sophisticates do not simply
consume more news but
process it differently
than their less-sophisticated counterparts.
Political knowledge
is for the most part obtained vicariously. National political campaigns
and policy debates are
conducted through
the media with the public's direct experience of the events rather limited
(but see Kinder &
Sears, 1981;
Stoker, 1992). The news media are the primary carriers of political information
to the polity,
sometimes in
less-filtered forms such as live telecast on C-SPAN, but more likely in
highly interpreted forms such
as talk radio
shows or a New York Times cover story. An informed public is one capable
of casting its votes
based on knowledge
of candidates' positions and performance and registering its opinion on
matters of public
policy from
a position of knowledgeable evaluation. The processes by which the public
becomes informed are
central to our
understanding of both the successes and failures of massive information
campaigns aimed at raising
the public's
level of knowledge about political affairs. In the research reported here,
a measure of prior political
knowledge called
political sophistication is proposed and evaluated as a predictor of learning
from news. Previous
research has
shown various versions of prior knowledge to be consistently related to
learning in both experimental
and survey contexts.
We pit our measure against other predictors of learning to evaluate its
effectiveness, asking in
the end what
distinguishes the way political sophisticates process news.
Learning From the News Media
Popular perception
suggests that the media should have large effects on learning and attitude
change (e.g.,
Davison, 1983).
When topics are high in personal relevance and when prior knowledge on
a topic is limited,
learning from
news should be substantial. The health care reform debate of 1993-1994
is just such a policy issue.
But there are
many reasons why mass*e media effects do not materialize even in the face
of high relevance and
little prior
knowledge (McGuire, 1986).
Simple Media Exposure and Learning
Robinson and
Davis (1990) reviewed large-scale, national surveys of public knowledge
about candidates and
public affairs
issues completed in the period 1967 to 1990. Their review concluded that
the most important
predictor of
accurate recall is education, followed by political interest. Newspaper
reading was a less powerful but
consistent predictor
whereas viewing television news had a very small to negligible effect on
learning. These
conclusions
are not surprising. Exposure to news may be a necessary condition for learning
political information,
but it is certainly
not sufficient.
Some researchers
have argued that simple exposure measures are blunt instruments for understanding
the effects
of media on
learning. J. M. McLeod and McDonald (1985) distinguished between general
media exposure and
specific exposure
to public affairs content in both broadcast and print media. They also
assessed attention to the
media. In addition
to these refined measures of media exposure, the authors also argued that
how people use the
media may have
important mediating effects on learning. If surveillance and utility are
two of the typical
gratifications
sought from the media, then perhaps learning would be elevated when these
needs are activated.
J. M. McLeod
and McDonald found that simple, generic measures of exposure are not effective
predictors of
learning of
specific content. Rather, when measures of exposure to specific content
were obtained, specific types
of learning
could be predicted. For example, exposure to news about foreign affairs
during a presidential campaign
was more likely
to predict knowledge about candidates' positions on foreign affairs than
was general exposure to
news. Also,
attention to the media and gratifications sought in processing information
were important predictors in
achieving even
modest amounts of explained variance. These conclusions suggest that much
more refined
measures of
exposure to news, including at least specific forms of exposure and specific
gratifications, are
necessary to
explain learning effects. Other modifications to the measurement of simple
exposure have also been
proposed and
tested with some success. Chaffee and Schleuder (1986) suggested the need
for general and
specific measures
of attention to various media in addition to exposure measures. They found
that media attention
added significantly
to variance explained in their learning scores.
The claim that
learning from television news is minimal has often been challenged. Chaffee,
Zhao, and Leshner
(1994) found
that newspaper reading and television news consumption are related to a
voter's knowledge of
issues and to
knowledge about the candidate's personal characteristics. The strongest
relationship was to
knowledge about
the candidate's character where variance explained was 10.5%. Candidate
personal knowledge
consisted of
information such as who disliked broccoli and who was a Rhodes scholar.
The authors concluded
that "this study
adds to the documentation of television's emergence as a principal medium
of campaign
communication"
(p. 317). Contrary to detractors, television news consumption "stands out
particularly in relation
to knowledge
about issue differences between the candidates" (p. 318). Chaffee et al.'s
data indicated that
learning was
related to simple exposure and attention measures. On the negative side,
the strongest relationships to
straight exposure
were for a 10-item candidate personal knowledge. Television news consumption
was related to
a 10-item measure
of candidate differentiation on issues, but all the exposure measures accounted
for 1.8% of the
variance. Both
were also related to a 20-item measure of party issue information, although
again the variance
accounted for
was only 3.7%.
The consensus
across a range of research is that simple measures of exposure to news
on television, in print, and
on radio do
not account for much variance in the public's knowledge about campaigns
or public policy
debatesunter,
1991; Robinson, 1986; Robinson & Davis, 1986; Robinson & Levy,
1986). This consensus cannot
be interpreted
to mean that the public does not learn from news at all. Rather, the what
and when of learning from
news depends
on a complex of factors related to education, prior knowledge, attention
to issues and to the media
themselves,
audience motivations, and specificity and reliability of measurement. The
research presented here
focuses on a
measure of political knowledge that we call political sophistication. We
explore the reliability and
predictive and
concurrent validity of the measure and pit it against alternative ways
of predicting learning from
controlled exposure
to news.
Background Knowledge as an Indicator of Schematic Structure
Media exposure,
attention, and gratifications are at best surrogates for the measure that
would be a direct measure
of reception
of information (Zaller, 1992). Reception of information assumes that a
person has been exposed to
the information,
attended to it, comprehended it, and can use it effectively in making political
judgments. Consider
an analogy.
Teachers evaluate students in terms of their ability to comprehend, recall,
and use the information
supplied in
lectures and readings. They do not evaluate students' performance solely
in terms of attendance,
attentiveness,
and attitude toward the subject matter. Although these factors are undoubtedly
correlated with more
direct measures
of performance, the direct measures of mastery of course materials are
certainly more valid
indicators than
the indirect measures.
This is precisely
what Zaller has proposed at both the theoretical and measurement levels.
Reception is measured
by asking fairly
general questions about the positions of various political actors and groups
on different issues. For
example, one
question might ask: "On the issue of abortion some people are primarily
pro-choice and some are
primarily pro-life.
Where does Ross Perot stand on this issue?" To correctly answer this question,
a person must
have received
information about Perot and his position on the choice-life issue; that
is, there had to be exposure,
attention, comprehension,
and recall. What Zaller (1987, 1992) has proposed is that the family of
measures of
domain-general
political knowledge will be better predictors of specific forms of political
learning and attitude
change than
will measures of media exposure to political information (and its surrogates).
Other researchers
have suggested similar measures, calling them political sophistication
(Crone, 1993; Luskin,
1987), political
expertise (Fiske, Lau, & Smith, 1990), public affairs knowledge (D.
M. McLeod & Perse, 1994;
Price &
Zaller, 1993), awareness (Zaller, 1987), and political schema (Kuklinski,
Luskin, & Bolland, 1991). We
will use the
phrase political sophistication. Public affairs knowledge is often not
background knowledge but factual
knowledge specific
to various research topics. Expertise suggests a special class of individuals
whose
sophistication
may be very high indeed. Political schema carries other structural baggage
that is not directly
measured by
simple knowledge scores.
Crone (1993)
has carried out a careful analysis of the concept of political sophistication
as used in political science
and political
psychology. She isolated two approaches: sophistication as "overarching
ideological viewpoints" (p.
1) and as the
construction of political opinion, knowledge, and information. The former
is fundamentally due to the
work of Converse
(1964). Crone maintained that Converse's view implicitly mixes ideology
and knowledge.
Although he
does not make the connection explicitly himself, the operational definitions
employed seem to require
knowledge about
the ideological positions of parties, groups, and candidates.
The second view
is based on the cognitive perspective from psychology in which information
and its organization
is central to
the definition of those who are sophisticated. In a special issue of the
journal Social Cognition,
Krosnick (1990)
brought together a number of researchers actively studying the structure
of political knowledge.
Their conceptual
and operational approaches to political sophistication varied, but all
focused on a central
theme--those
higher in political sophistication had more complete and more accurate
knowledge about politics,
government,
and civics than those who were lower.
Krosnick (1990)
included in his measure of sophistication interest in polities, attention
to the mass media, and
political participation
as well as political knowledge. Fiske et al. (1990) included political
activity, mass media
exposure, and
"political self-schemata" (related to political interest) in addition to
political knowledge. Judd and
Downing (1990)
combined political involvement and knowledge into a single scale, as did
McGraw and Pinney
(1990) with
behavior, knowledge, interest, and media use.
Only Zaller (1990)
and Kinder and Sanders (1990) used pure knowledge measures. There are a
number of
reasons for
adopting the position of these authors according to Crone. First, knowledge
is the only measure that is
common to all
those who have conceptualized this variable. Even Converse's ideological
tilt can be seen as
involving knowledge
in a central way. Second, knowledge is conceptually distinct from ideology,
because one can
be ideologically
driven and still possess low levels of knowledge in terms of breadth and
organization. Measures
like interest
in politics, mass media exposure to political information, and involvement
in the political process are
conceptually
distinct. Although they may be related to political sophistication, they
should remain conceptually
distinct. Third,
political knowledge is a good predictor of the outcome that it should predict,
including learning
(Price &
Zaller, 1993), reaction times to answer political questions (Fiske et al.,
1990), and attitude consistency
(Judd &
Downing, 1990; Wyckoff, 1987). Crone concluded her theoretical and empirical
assessment of political
sophistication
with the notion that the two worlds of political sophistication--ideology
and information--are not
independent
but related. Conceptual and operational practice then should incorporate
into political sophistication
both ideological
and informational components.
Part of the theoretical
value of the political sophistication construct is found in its explanatory
role in models of
change in public
opinion. Zaller (1987) reformulated the models of McGuire (1968) and Converse
(1962) in a
way that explicitly
showcases the role of political sophistication in political belief change.
The McGuire-Converse
model holds
that the probability of change is a function of the product of two other
probabilities: the probability of
exposure to
a persuasive message and the probability of acceptance of the message given
exposure. In turn, the
probability
of exposure is a monotonically increasing function of political sophistication
as the more knowledgeable
audiences are
more likely to be exposed than the less knowledgeable audiences. The probability
of accepting a
message is a
monotonically decreasing function of political sophistication because more
knowledgeable audiences
are presumably
more difficult to persuade (Petty & Cacioppo, 1986). The result is
a family of (often) nonlinear
change curves
as a function of political sophistication. Zaller (1992) tested this theory
in a wide variety of political
contexts. He
found that a relatively simple measure both operationally and theoretically
is the driving engine of a
theory of political
belief formation and change. Important, the measure is a surrogate for
the media's effects and a
surrogate for
the audience's structuring of general political knowledge.
Price and Zaller
(1993) specifically tested the relative efficacy of political sophistication
versus newspaper and
television exposure,
and interpersonal discussion in predicting people's knowledge of specific
news stories
appearing in
the national news agenda. Across 16 news items, political sophistication
was a strong, significant
predictor of
learning even in the presence of education, whereas newspaper exposure
was not significant and
television news
exposure had small effects, significant in about half the cases. The authors
argued that even very
simple measures
of political knowledge are better predictors of learning from the media
than are most any
combination
of exposure measures. The authors avoided concluding that television, print
news exposure, and
education have
no predictive power, but they did conclude that "their incremental predictive
power is relatively
weak, and preexisting
levels of general political knowledge clearly offers us the most reliable
and parsimonious
way of predicting
individual differences in likelihood of news reception" (p. 153).
Although the
authors did not imply media exposure measures are useless in studies of
learning and opinion change,
they did invite
students of political communication to "abandon their normal reliance on
self-reported levels of
news media exposure
and look instead to prior political knowledge [political sophistication]
as a preferred general
indicator of
news reception" (p. 160). It is our view that such advice is premature,
not because Price and Zaller's
findings are
flawed in any obvious way, but because replication and extension of their
analyses are necessary
before the long
and venerable traditions of using media exposure measures are replaced.
Also, political
sophistication
is presumed to be related to exposure to political information and to its
processing. Exploration of
this very assumption
is the primary purpose of the present research.
Research Questions and Hypotheses
Our research
was conducted in the service of three goals: (a) to replicate the findings
on the relationship between
political sophistication
and learning in other contexts, (b) to explore the conceptual underpinnings
of the political
sophistication
construct empirically and theoretically, and (c) to begin explaining how
political sophistication comes
about.
The Relationship Between Prior Knowledge and Learning
If political
sophistication is to be a good measure of reception, then its relationship
to learning should be robust.
Numerous studies
and a variety of researchers have noted that those with higher levels of
general knowledge also
learn more from
exposure to information from the media. When education is treated as a
surrogate measure of
prior knowledge,
it predicts learning in a variety of contexts, often more strongly than
any other factor (Chaffee et
al., 1994; Robinson
& Davis, 1990).
Education, when
coupled with levels of political interest, may be an indicator of prior
knowledge as Robinson
(1986) suggested,
but it is not a direct measure of prior knowledge. When prior knowledge
is measured directly,
subsequent learning
is predicted strongly. Robinson and Davis (1986) showed in two studies
of learning in the
United States
and England that those with high levels of prior knowledge comprehend and
recall more from a
single evening's
newscast than those with low prior levels of knowledge. Learning from news
is "not a simple
by-product of
the educational process but is influenced by some personal investment of
interest and ability as well"
(p. 129).
Neuman, Just,
and Crigler (1992) conducted a series of experiments on learning from various
media. Part of their
concern was
with the audience's level of ability to process information. They wanted
to differentiate participants'
ability level
but not confound general cognitive abilities with simple recall. Rather,
they tried to keep cognitive
ability distinct
from recall by using vocabulary and inference skills as their measure of
ability. In one experiment,
participants
were presented with a story from network TV news, a news magazine, or a
newspaper. They found
that knowledge
about news stories that the audience brought into the experiment was correlated
with cognitive
skill. And,
after exposure to the news stories, the percentage correct on tests of
knowledge was linearly related to
education, cognitive
skill, and previous levels of attention to the story.
Those with low
cognitive skill and low attention to issues were lowest in knowledge prior
to exposure to news and
had lowest recall
scores after exposure to news; those highest in each of these factors were
highest in preexposure
levels of knowledge
and highest in postexposure recall. Those high in verbal and cognitive
skills not only have
substantial
levels of knowledge to carry with them as they approach news but also retain
more either as a result of
previous knowledge,
skill at retention and retrieval, or both. The authors argued that those
who are highly skilled
are simply more
efficient than the less skilled. They can scan much more material than
those who are low in skill
and retrieve
what is useful and relevant.
Price and Zaller
(1993) used a measure of factual political knowledge to predict learning
from news. Factual
political knowledge
was based on a person's ability to identify jobs held by various figures
(such as George
Schultz, Margaret
Thatcher, and Ted Kennedy) and questions about which party is more conservative,
in power
in the House
and Senate, and so on. The authors argued that even very simple measures
of political knowledge
are better predictors
of learning from the media than are most any combination of exposure measures.
They did
not argue that
political knowledge is expertise but only that previous knowledge is important
for subsequent
knowledge perhaps
because of its relation to exposure and perhaps because of its relation
to reception. Price and
Zaller (1993)
found that their measure of political knowledge predicted learning from
news better than any
measure of exposure
to print or broadcast news.
In sum, both
experimental and survey evidence indicated that previous knowledge of topics
facilitates recall of
information
from the news even when the previous knowledge is not topically related
to the news items. Also,
prior knowledge
predicts learning of news better than does simple exposure to print or
broadcast news. What is
not clear is
why political sophistication should function so effectively in predicting
reception of information from
news.
Explanations of the Sophistication-Learning Relationship
One answer to
this question has been that political sophistication is a surrogate for
receptivity and is therefore an
indicator of
successful comprehension and recall of political information. Actually,
the measure of sophistication is
itself a measure
of accurate recall of factual information, but it is assumed to represent
intervening subprocesses
such as attention
and comprehension. This explanation is unsatisfying theoretically because
political sophistication
may simply be
a more abstract indicator of learning and related to specific learning
in much the same way that
general intelligence
is related to more specific forms of intelligence.
Some researchers
have argued that political sophistication is an indicator of the degree
of schema development
(Fiske et al.,
1990; Hsu & Price, 1993) and all that such development entails (Markus
& Zajonc, 1985). If it is,
then those high
in political sophistication should have more developed political schemes
with positive
consequences
for political learning (Fiske et al., 1990) such as a greater number of
thoughts in response to
persuasive messages
(Hsu & Price, 1993), faster response times to political questions,
and more accurate
inferences from
political messages.
However, the
measure of political sophistication is only a measure of general factual
knowledge and not anything
like the organized
structures of knowledge and procedure presumed to be the core of the concept
of schema. As
Kuklinski et
al. (1991) have pointed out, the schema concept is much broader and deeper
than its measurement
has been. If
we are to even entertain the idea that political sophistication measured
through general questions
about political
issues is an indicator of the level of development of political schemes,
then additional evidence of
schematicity
is necessary. One goal of the research reported here is to ask whether
the construct of political
sophistication
is an indicator of the level of development of political schemes.
There are many
ways to index a cognitive schema, but, as a practical matter, none is definitive.
Schemas are
organized knowledge
structures in which the content can be social, personal, role, self, analogic,
temporal,
technical, political,
and so on (Fiske & Taylor, 1991) and the organization hierarchical,
associational, or a
combination.
Schemas are basically unobserved constructs that, if they are to be accepted
as viable accounts of
human cognition
and behavior, have to be useful shortcuts for explaining observed outcomes.
As with any
unobserved construct,
researchers use schemes conceptually to make sense of observed outcomes
while trying to
gather increasingly
direct evidence of the existence of the schema itself. In our studies,
two indicators of schematic
organization
are employed: construct differentiation in open-ended narratives and argumentative
depth in
open-ended questions
about social issues.
Response times
can be used to assess schematic organization (Luce, 1986). To do so requires
knowing the
schematic structure
of knowledge in advance or establishing it experimentally. In either case,
one must have an
operational
definition of schema in advance of employing the response time measure
that is independent of the
response time
measure itself. In short, measuring schemes is a boot-strapping process
in which softer measures
such as cognitive
differentiation and integration must serve as direct measures in some studies
and validators of
harder measures
in other studies. Our research design did not allow the use of response
time measure of
schematic operation.
Construct differentiation
has been widely used as a measure of the complexity of a person's knowledge
about a
particular topic.
In its various guises, construct differentiation has been employed in studies
of interpersonal
communication
(Burleson & Waltman, 1988), information processing (Schroder, Driver,
& Streufert, 1967), and
political reasoning
(Tetlock, 1985). If political sophistication is an indicator of the level
of schematic development,
then those higher
in sophistication should also have higher differentiation scores. Presumably,
the greater the
construct differentiation,
the more different ways a person has to categorize and differentiate issues
on a topic.
Construct differentiation
indexes the dimensionality of a person's schema and is, therefore, a necessary
condition
for showing
that political sophistication is an indicator of schema development. But
differentiation shows neither
the organization
of knowledge in the schema nor the person's ability to use the knowledge
in that most important
of political
activities--reasoned argument. If political sophistication is an indicator
of schema development, then
those higher
in sophistication should exhibit more highly developed reasoning in their
political arguments.
Our measure of
political argument depth is derived from the work of Kuhn (1991). Kuhn
studied arguments that
people made
in interviews about such topics as recidivism and failure in school. She
coded the number of different
reasons that
people cited for a particular claim, their use of evidence, their ability
to conjure relevant
counterarguments
to their stated positions (e.g., from someone who might disagree), and
their ability to refute the
hypothetical
counters. People who show mastery of all of these components of an argumentative
exchange might
be said to have
a well-organized knowledge structure (on the topic) and to be capable of
using that knowledge in
dialogic arguments.
Based on Kuhn's research and theorizing about argument, a measure of argumentative
depth
was created
(Woodard, 1995). We assumed that those high in political sophistication
would also show more
argumentative
depth and that this depth is indicative of more integrated schemes and
greater ability to use the
knowledge stored
there.
When media exposure
on a topic is controlled, does political sophistication still explain learning
on the topic over
and above the
learning due to exposure? It is assumed that political sophistication is
a measure of receptivity and
that receptivity
is exposure plus attention plus comprehension plus recall. So when exposure
is controlled, and if
political sophistication
still results in learning, then the results must be attributed to attention,
comprehension, and
recall and not
to simple exposure. This would mean that those who differ in political
sophistication differ not just in
exposure but
how, cognitively, they are processing the information to which they are
exposed (Graber, 1988;
Woodall, 1986).
Antecedents of Political Sophistication
The final research
question raised in this article concerns the antecedents of political sophistication
and of learning.
Although political
sophistication may be a predictor of learning, it is almost certainly not
independent of media
exposure to
political news. On the contrary, news exposure is a necessary condition
for learning and activates the
knowledge structures
of sophisticates. What is key is a precise understanding of the relationship
between
measures of
exposure and measures of political sophistication. Because those high in
political sophistication are
purported to
have high exposure and to process the information to which they are exposed
more completely
(through attention,
comprehension, and recall), we hypothesize that two broad classes of predictors
must be
considered when
accounting for sophistication scores: exposure levels (obviously) and the
uses to which the
information
is likely to be put.
The extent to
which people pay attention to political events and issues and the way they
use the news should differ
for those who
differ in political sophistication. Specifically, we expect the effects
of media news exposure on
political sophistication
to be mediated by the extent to which people pay attention to news and
how politically
involved they
are. As suggested by J. M. McLeod and McDonald (1985) and Chaffee and Schleuder
(1986), the
level of attention
represents an important way that media news can matter to people who are
more and less
politically
sophisticated.
In sum, four
questions are raised in the research reported here: Does political sophistication,
measured as
knowledge and
ideology, predict learning from controlled sources of news? Does political
sophistication predict
learning better
than, or worse than, other predictors including education, media exposure,
and other measures of
knowledge? Do
people high in political sophistication exhibit more developed cognitive
schemes as evidenced in
more elaborated,
differentiated, and deeper reasoning about issues? Is more elevated political
sophistication the
result of demographic,
attentional, media use, and involvement differences?
Study 1
From late summer
1993 to early fall 1994, the Clinton administration pushed vigorously for
change in the health
delivery system
in the United States. Media attention to the need for and prospects of
health care reform was
intense. In
the period from January 15, 1994, to July 25 of the same year, we catalogued
2,066 newspaper items
about health
care reform from 10 national and local newspapers and 678 items from 9
sources of broadcast news.
During the same
period, national public opinion polls showed that the percentage of persons
who knew what a
"health alliance
or consumer purchasing cooperative" was changed from 22% in early October
1993 to 36% in
late June 1994.(2)
Despite massive exposure, the public's knowledge and confidence in their
knowledge about the
issues surrounding
health care remained at low levels.
Method
Data to evaluate
our hypotheses came from a field experiment on the effects of print news
formats on learning and
public cynicism
during the health care reform debate. The experiment was conducted during
March 1994 and
some of its
results are discussed in Cappella and Jamieson (1994).
SUBJECTS
Approximately
350 people from six media markets (New York; Philadelphia; Washington,
D.C.; Dallas; Los
Angeles; and
Chicago) were recruited to participate in the field experiment through
notices posted in fraternal,
social, and
religious groups, work, and other settings. The sample was not random and
participants were recruited
for pay. Space
limitations prohibit a detailed discussion of the demographic characteristics
of the sample. Our
participants
were more highly educated (58% college degree or higher), more female (60%),
and more
Democratic (47%)
than national samples. With regard to age and race, they were roughly comparable
to national
norms.
STIMULI
The types of
news coverage tested were chosen to reflect the kinds of coverage that
journalists had been giving to
health care
reform from September 1993 through March 1994. These are discussed in detail
elsewhere (Cappella
& Jamieson,
1994). Suffice it to say that four unique types of media coverage were
used. Six combinations of
print news stories
were given to the six experimental groups.
DESIGN AND PROCEDURES
A research assistant
was on-site for a full week in each of the media markets. He or she administered
a pretest
questionnaire
to each of the participants before any news articles were read. Everyone
received 15 news articles,
3 per day, to
read. One factual article on the basic issues in health care reform was
common to all groups
including the
control. The control received 14 other news articles on current affairs
but not on the health care
debate.
The design was
a 1 x 7 factorial with posttest-only measures for all variables except
those for attitudes that were
obtained both
in the pretests and posttests. Random assignment to condition was successful
for various
demographic
variables, as well as media consumption, attitude, political sophistication,
and cynicism variables.
Participants
reported following instructions about reading news articles (or were dropped
from analysis). They
also found the
news articles realistic, like the ones they normally find in newspapers,
but they reported reading
them somewhat
more closely than they normally would have. At the end of the week, all
participants met together
to watch an
edited, 20-minute debate on health care that had previously appeared on
C-SPAN. They filled out
the final questionnaire
on attitudes, learning, cynicism, and other items, and were debriefed.
Measures
POLITICAL SOPHISTICATION
Political sophistication
was measured with a series of questions on general political knowledge.
Specifically,
participants
were asked to place Bill Clinton, most conservatives, and most liberals
on a continuum that most
closely described
their positions on various issues. For example,
Some people say that abortion should not be permitted because it is
taking the life of an unborn child. Others say that abortion should be
permitted in the first three months of pregnancy as a matter of personal
choice.
What does Bill Clinton say about life and choice issues?
Pro-life:--:--:--:--:--:--:--:Pro-choice
Other issues
included preference in hiring for minorities, NAFTA and jobs, environment
and job loss, health costs
and price controls,
and health reform and government involvement.
Scoring of the
answers was done by comparing a person's positioning of Clinton, most liberals,
and most
conservatives
relative to one another. For example, if a person indicated that the position
of most liberals on
NAFTA was more
toward the "lose jobs" end of the continuum whereas that of most conservatives
was toward
the "gain jobs"
end, they would be given 1 point correct but not if the order was reversed.
Thus, the positioning of
Clinton relative
to conservatives, Clinton to liberals, and liberals to conservatives was
compared for correct
positioning
relative to one another. Being on the "correct" side of the scale was not
evaluated; only the relative
positions were
evaluated. In those cases in which the relative positions did not yield
a clear answer in terms of
acknowledged
public positions, the comparison was ignored. For example, Clinton is not
clearly more liberal nor
clearly more
conservative than most liberals on the issue of pro-choice versus pro-life.
So this comparison does
not enter the
index of political sophistication.
This measure
provides a number of advantages as an indicator of political sophistication.
First, it allows people to
use the scales
in their own way and only makes comparisons within persons rather than
evaluating the person's
score to some
external standard. Second, the measure allows both knowledge and ideology
to be factored
simultaneously.
As Crone (1993) has argued, political sophistication should reflect both
ideological knowledge
and knowledge
not readily derived from ideological heuristics. By comparing ideologically
affiliated groups to one
another on a
continuum defined by an issue (and not an ideology), the measure has features
of ideological and
nonideological,
substantive knowledge as components. Finally, the measure does not sound
like a test of
knowledge, so
respondents may be less anxious about it than recall or recognition tests.
The scale consisted
of 11 comparisons with a range of 0 to 11 (M = 8.50, SD = 2.4), and an
internal reliability of
.79.(3)
CIVICS KNOWLEDGE
In contrast to
political sophistication, which tries to measure current political knowledge,
civics knowledge
assesses background
information about the functioning of government. Five questions were used
to measure civics
knowledge, including
the size of the majority necessary to override a presidential veto, the
number of full terms of
office that
can be served by the president, the term of office of a United States senator,
the party currently in
power in the
House, and the names of the two senators from the person's state. The range
on this variable was 0
to 1.25 (M =
1.06, SD = .26). Cronbach's alpha for this measure was .51.
POLITICAL INVOLVEMENT
Political involvement
was assessed by a 7-item yes or no scale asking people if they had ever
written a letter to a
newspaper, joined
a political organization, written a letter to a politician, signed a petition
on a political issue,
participated
in a march or demonstration, voted in the most recent presidential election,
and voted in the most
recent senate
election. This measure has a range of 0 to 7 (M = 3.8, SD = 1.9) and an
internal reliability of .72.
ATTENTION TO NATIONAL ISSUES
Attention to
national issues was measured by four questions asking how closely (scored
1 to 4) the person had
been following
the Whitewater allegations, NAFTA, health care reform, and discussions
about the national debt.
The range on
this variable was 1 to 4 (M = 2.5, SD = .69). This measure was found to
be reliable with
Cronbach's alpha
of .80.
MEDIA NEWS EXPOSURE
Nine questions
were asked about exposure to media for news. People were queried about
their use of national
TV news programs,
local TV news, TV interview news, newspapers, news magazines (like Newsweek),
radio
news, radio
talk and call-in, TV talk shows, and conversations with friends and coworkers.
All questions were of
the form "How
often do you listen to radio call-in and radio talk shows for your news?"
with weekly or daily
measures used
as appropriate. Four indexes were generated through factor analysis: TV
news (3 items with an
internal reliability
of .52), newspapers (2 items with a correlation of .31), radio news (2
items with a correlation of
.47), and a
single item for conversation with friends.(4)
LEARNING
At the final
meeting of the participants in each research site, people answered questions
to test their recall of
information
from the C-SPAN debate and from a factual background newspaper article
about health care.
Everyone in
the study had exposure to both sources. Three types of questions were used
to measure recall: factual
recall from
the print article, recall of strategic information from the videotaped
debate, and issue-based recall from
the debate.
One cluster of
items consisted of seven issue-oriented questions from the C-SPAN debate.
For example, these
included questions
such as:
Who argued that providing Americans with free health services would
not work because people would not "show up" to get these services?
Who complained that health care costs only slow down when there is
a threat of government price controls?
Although about
27% of the participants got fewer than half the issue-based answers correct,
there was a sharp
skew toward
correct answers with one quarter of the participants getting more than
85% of the answers correct.
A second group
from the C-SPAN debate consisted of five strategy-oriented questions. Strategy-based
questions
included:
Who leaned back in his chair and looked overconfident?
Who referred to the other people on the panel by their first names?
Sixty-one percent
of the participants got 40% to 60% of the strategy answers correct. The
distribution of correct
answers was
approximately normal.
The third cluster
from the newspaper article consisted of five questions. Questions testing
knowledge gleaned from
the factual
newspaper article included:
Do the health care plans being debated now agree on anything?
Which of the plans set government limits on national health spending?
Each of the questions
had an appropriate set of alternatives from which to choose. Sixty percent
got 40% to 60%
of the answers
correct. The distribution of correct answers was approximately normal.
It should be
emphasized that the correct answers from the C-SPAN debate required watching
and attending to
the debate.
They could not be answered solely from one's previous exposure to information
about health care
reform (although
it was our hypothesis that exposure to certain kinds of coverage would
enhance or depress
people's ability
to handle strategic and issue-based information). Correct answers based
on questions from the
newspaper article
could be enhanced or depressed as a result of reception of outside information
about the health
care reform
debate.
ARGUMENTATIVE DEPTH
To assess argumentative depth, participants were asked the following series of open-ended questions:
In the debate about health care, almost everyone agrees that between
35 and 45 million Americans have no health coverage.
Why do you think these people do not have health care coverage?
What other reasons come to mind as to why people don't have health
care coverage?
If someone were to disagree with your reasons about why people don't
have coverage what might that person say to try to convince you that
your reasons were wrong?
What reasons could you give to convince your friend that you were right
in the first place?
Based on Kuhn's
(1991) work with naturally occurring arguments, a hierarchical coding system
was developed
that reflected
whether (relevant) reasons were provided at all; the number of (relevant)
reasons provided; the
number of claims
with coherent reasoning; whether a relevant counter was provided; or whether
a relevant rebuttal
to the counter
was provided.(5) A single score representing argumentative depth summed
the five components.
The subcomponents
of argumentative depth together produce an internal reliability of .72
for the entire scale,
suggesting they
are each measuring a component of the same process. Reliability of coding
was assessed at each
of the five
levels of the coding system and overall. The range across levels was .56
to 1.00 on Krippendorff's
alpha (Krippendoff,
1980) and 67% percent to 100% on agreement.
CONSTRUCT DIFFERENTIATION
To assess the
level of complexity of people's knowledge about the health care debate,
participants were asked to
answer the following
open-ended question at the conclusion of the field experiment:
Pretend that a good friend writes you a letter asking what you know
about the health care debate in the United States. Your friend has been
out of the country and knows nothing except that a health care policy
is currently being debated. Write a letter to your friend telling him or
her as much as you can about the alternative health proposals, the
persons sponsoring them, and their likely effect on you if they are
passed. In responding to your friend draw on everything you know
about the debate including information gotten from other sources and
from what you read and watched in the past week as a part of this study.
To evaluate the
complexity of these essays, a coding system was developed based on the
methodologies of
Tetlock (fetlock
& Hannuum, 1984) and of Burleson (Burleson & Waltman, 1988) that
evaluated the essays for
the number of
unique constructs related to the health care debate.(6) Fifty-seven of
340 essays were coded for
construct differentiation.
The reliability of the coding was high, being in the range of .90. However,
the correlation
between construct
differentiation and simple word counts was so high (r = .95) that coding
was abandoned in
favor of simple
word counts. The two measures share 90% of their variance. Word counts
ranged from 0 to 686
(M = 161, SD
= 109).
Results
CORRELATIONAL ANALYSIS
Zero-order correlations
among political sophistication, civics knowledge, involvement, attention,
media variables,
and demographics
are reported at the bottom diagonal of Table 1. Political sophistication
was modestly and
significantly
associated with civics knowledge, political involvement, and attention
to national issues. In relation to
media news exposure
variables, political sophistication showed a weak positive association
with newspaper
exposure (r
= .11, p < .05), whereas it was negatively linked with television news
exposure (r = -.21,p < .001).
Civics knowledge
and political involvement were significantly associated with newspaper
exposure but not with
television exposure.
Attention to national issues was closely related to all the media exposure
variables. In addition
to the modest
relationship with education, political sophistication is significantly
associated with other political
orientation
variables. This suggests that, for the most part, the political sophistication
construct is empirically sound
in the sense
of concurrent validity
Table 1
Pearson Correlations
among Variables (Studies 1 and 2)
Political Civics
Sophistication Knowledge
Variable
1
2
Political sophistication
--
.37(***)
Civics knowledge
.30(***) --
Involvement
.40(***) .40(***)
Attention
.26(***) .31(***)
Print
.11(*) .30(***)
TV
-.21(***) .03
Radio
-.08
.08
Talk
-.02
.02
Education
.35(***) .32(***)
Age
.01
.18(**)
Female
-.03
-.12(*)
Involvement Attention Print
Variable
3
4 5
Political sophistication
.25(***) .24(***)
.12
Civics knowledge
.39(***) .31(***)
.27(***)
Involvement
--
.32(***) .15(*)
Attention
.42(***) --
.33(***)
Print
.31(***) .36(***)
--
TV
-.01
.29(***) .21(***)
Radio
.01 .21(***)
.06
Talk
.09 .25(***)
.25(***)
Education
.42(***) .16(**)
.16(**)
Age
.18(**) .25(***)
.25(***)
Female
-.06 -.16(**)
-.16(**)
TV Radio
Talk
Variable
6
7 8
Political sophistication
-.02 .13(*)
--
Civics knowledge
.13(*) .17(*)
--
Involvement
.09 .05
--
Attention
.42(***) .10
--
Print
.33(***) .18(**)
--
TV
-- .08
--
Radio
.22(***) --
--
Talk
.17(**) .14(*)
--
Education
.22(***) -.06
.06
Age
.17(**) .13(*)
-.15(**)
Female
-.23(***) -.07
-.03
Education Age
Female
Variable
9 10
11
Political sophistication
.29(***) -.01
-.03
Civics knowledge
.22(***) .24(***)
-.07
Involvement
.31(***) .28(***)
.13
Attention
.14(*) .19(**)
-.05
Print
.10 .26(***)
-.04
TV
-.11 .40(***)
.01
Radio
-.01 .05
-.07
Talk
-- --
--
Education
-- .01
.08
Age
.11(*) --
.02
Female
-.09 .02
--
Note. The bottom
diagonal is for Study 1 (N = 325). The upper diagonal is for Study 2 (N
= 243). (*) p <.05.
(**) p <.01.
(***) p <.001.
PREDICTING OUTCOME MEASURES
WITH POLITICAL SOPHISTICATION
In an effort
to examine the predictive power of political sophistication, three conceptually
distinct outcome
measures were
assessed: (a) learning measured by three different types of recall items
(factual, issue, and strategy
recall); (b)
use of knowledge in argumentation measured by argumentative quality; and
(c) construct differentiation
measured by
the number of words in the essay question. Multiple regressions were run
to evaluate the relative
importance of
political sophistication in predicting these outcome measures as compared
to other predictors,
including media
exposure variables, attention, involvement, and civics knowledge. Education
and experimental
condition (1
= experimental condition, 0 = control one) were also entered in the regression
equation to control for
the effects
of added exposure.
The first three
regression equations in Table 2 indicate that political sophistication
is a significant predictor of each
of the three
types of learning (two from broadcast and one from print). Even after controlling
for the effects of
education, experimental
condition, and news exposure, people with greater political sophistication
were more
accurate in
their recall of information. The effect size for factual recall questions
about the newspaper articles (p =
.24, p <
.001) was about the same with that of issue questions about the videotaped
debate (p = .26, p < .001).
But it was relatively
small for strategy-type questions about the videotaped debate (p = .17,
p < .01). Except for
the negative
effect of television exposure on issue recall (and the borderline effect
of radio exposure on strategy
recall), media
exposure variables did not approach significance in predicting learning
measures. The results suggest
that the presence
of an effect of political sophistication on learning must be due to differences
in the way the high
sophisticates
process information in comparison to the lows.
Table 2
Predicting Recall,
Argument Quality, and
Construct Differentiation
(Study 1)
Outcome Measures
Factual Issue
Strategy
Predictor
Recall Recall
Recall
Education
.14(*) .06
.02
Experimental
condition .05
.01 -.00
Political sophistication
.24(***) .26(***)
.17(**)
Civics knowledge
-.01
.08 .02
Involvement
-.01
.13(*) .00
Attention
.12(+) .06
.02
Print
-.02
-.07 -.04
Television
-.03
-.15(**) -.04
Radio
-.04
.07 .10(+)
Talk
.01
.03 .07
[R.sup.2]
.15
.20 .05
F
5.86(***) 7.88(***)
1.82(+)
N
332
332
332
Argument Construct
Predictor
Quality Differentiation
Education
.10(*) .24(***)
Experimental
condition .06
.00
Political sophistication
.19(**) .24(***)
Civics knowledge
.12(+) .02
Involvement
-.06
.10
Attention
.10
.04
Print
.00
-.09
Television
-.03
-.14(*)
Radio
-.04
-.06
Talk
.08
.05
[R.sup.2]
.14
.24
F
5.28(**) 10.31(***)
N
328
332
Note. Cell entries are beta weights from ordinary least squares regression.
(+) p < 10.
(*) p < .05.
(**) p < .01.
(***) p < .001.
Columns 4 and
5 in Table 2 present the results on the two indicators of schematicity:
argumentative depth and
construct differentiation.
In both cases, political sophistication was strongly related to these indicators
of
schematicity
over and above the effects due simply to education. People high in political
sophistication wrote
essays about
the health care reform debate that had more constructs than those who were
low ([Beta] = .19,p <
.01). Similarly,
sophisticates crafted arguments about one of the fundamental problems in
the health care system
that were of
greater depth than those less sophisticated ([Beta] = .24, p < .001).
Together, these two findings
suggest that
sophisticates may have more developed schemes reflected in the complexity
of constructs employed,
their integration,
and ready availability for use in crafting arguments.
In all five regression
analyses, the tolerance indexes for all the predictors were above .65,
which suggested that
there was no
serious problem of multicollinearity. Therefore, the beta weights for the
predictors can be interpreted
as the relative
importance in predicting the outcome measures. The beta weights for political
sophistication ranged
from .17 to
.26. They were larger than those of all the other predictors. This indicates
that political sophistication is
a better predictor
of learning, argument quality, and cognitive differentiation than civics
knowledge, involvement,
attention, and
news exposure variables.
STRUCTURE OF POLITICAL SOPHISTICATION
Political sophistication
shows good predictive validity in terms of learning and schematicity. In
this section, we ask
what accounts
for variation in political sophistication. If it is a measure of reception,
then it should be related to
exposure measures
and to other measures of how the news media are used. Table 3 shows the
relative amount of
explained variance
in sophistication by three classes of predictors: demographics, political
orientations (knowledge
of government,
involvement, attention to current issues), and media exposure variables.
Four regressions are
presented: demographic
predictors alone, political orientations alone, news exposure measures
alone, and all three
groups at once.
Table 3
Predicting Political
Sophistication (Study 1)
Equations
Predictor 1 2 3
Education
.36***
Age
-.02
Female
-.03
Civics knowledge
.14(*)
Involvement
.32(***)
Attention
.07
Print
.13(*)
Television
-.22(***)
Radio
-.03
Talk
.06
Adj. [R.sup.2]
.12 .18
.04
F
15.7(***) 24.9(***) 4.8***
N
327 327
329
Equations
Predictor 4
Education
.19(***)
Age
-.07
Female
.05
Civics knowledge
.11(*)
Involvement
.22(***)
Attention
.23(***)
Print
-.03
Television
-.22(***)
Radio
-.03
Talk
-.07
Adj. [R.sup.2]
.27
F
12.7(***)
N
325
Note. Cell entries are beta weights from ordinary least squares regression.
(*) p < .05.
(**) p < .001.
In Equation 1,
demographics accounted for about 12% of the total variance in political
sophistication with one
significant
predictor--education ([Beta] = .36, p < .001). Originally, party affiliation
was included in Equation 1.
However, partisanship
did not predict political sophistication at all. Equation 2 reveals that
civics knowledge,
involvement,
and attention explained 18% of the variance in political sophistication.
Among these, civics
knowledge ([Beta]
= .14, p < .05) and involvement ([Beta] = .32, p < .001) were significant
predictors. As
shown in Equation
3, a mere 4.4% of the variance in sophistication was accounted for by the
media news
exposure variables.
Newspaper exposure was significant with a beta of .13 (p < .05), and
television exposure was
negatively related
to political sophistication with a beta of -.22 (p < .001). Equation
4 indicates that all the
explanatory
variables accounted for about 27% of the variance in political sophistication.
No evident problem of
multicollinearity
was found in Equation 4. That is, the level of tolerance for the predictors
was at least above .63
and the variance
inflation factors (VIF) were all below 2.0. This allows us to interpret
the beta weights in Equation
4 as the relative
importance in predicting political sophistication. Education ([Beta] =
.19, p < .001), political
involvement
([Beta] = .22,p < .001), attention to national issues ([Beta] = .23,
p < .001), and (lack of) exposure
to television
([Beta] = -.22, p < .001) were the most important predictors. That is,
the politically sophisticated are
more likely
to be educated, be engaged in political activities, and closely follow
national issues, although less likely
to watch television
news.
To summarize,
the low explained variance due to media exposure, coupled with the substantial
effect sizes due to
political orientations,
supports the claim that political sophistication is less the result of
exposure to news and more
the result of
how news is processed. Those with higher attention to national issues and
greater political involvement
are more likely
to use the news media productively than are those comparably lower. The
political news has
greater utility
for those following national issues or those involved in affecting those
issues.
Study 2
Although the
results of Study 1 speak to a number of important issues concerning the
predictive validity of political
sophistication
within the context of the 1994 health care reform debate, one might raise
a question as to whether
the findings
in Study 1 will hold in other political contexts. Study 2 aimed to replicate
the findings of the previous
study using
the same research variables within a different political context--an election
campaign. In particular, to
keep consistency
in the framework of hypothesis testing, key research variables--such as
political sophistication,
media exposure,
and civics knowledge--were constructed in much the same way as Study 1.
One exception was
the measure
of attention. In Study 1, attention was measured by how closely people
followed major national
issues; this
time, in Study 2, attention was specifically measured by the extent to
which people paid attention to the
1992 presidential
election campaign and to the news coverage of it. In addition, to retest
the validity of political
sophistication
as a measure of schematicity, a new outcome measure, called the issue-elaboration
index, was
employed. This
measure concerned the extent to which people attributed campaign issues
to political agents (i.e.,
major candidates)
and particular claims made by them. Because this measure presumably captures
the cognitive
ability to sort
out different campaign issues and to make connections among social agents,
sources, and claims, we
believe that
it taps into advanced uses of political knowledge.
Method
The data came
from a research project on news in a simulated political campaign (Jamieson
& Cappella, 1993)
examining the
effects of news reporting frames on learning, cynicism, and voting intention.
This research project
was designed
as a field experiment with a sample of 276 adults recruited in seven states.
The sample
characteristics
and the nature of study design were similar to those of Study 1. The sample
was representative in
terms of gender,
age, and race as compared to the estimates of U.S. voter population in
the National Election
Survey. The
sample overrepresented the most highly educated. The experimental materials
were news stories
about the 1991
Philadelphia mayoral election campaign. In practice, two different formats
of news
coverage--issue
versus strategy-oriented reporting--were created for print and broadcast
news. The results of the
stimulus-evaluation
tests and the manipulation check were reported in the experimenters' report
(Cappella &
Jamieson, 1994).
It was found that the news stories were generally consistent with what
the experimenters had
assumed about
the articles. Before the experimental treatment, participants answered
the questions about
demographics,
political orientations, and political sophistication. Participants were
randomly assigned to the
experimental
setting and were asked to read or watch the news materials. After 5 days
of exposure to the
materials, the
items measuring learning and schematicity were administrated as part of
posttests.
Measures
POLITICAL SOPHISTICATION
In constructing
the measure of political sophistication, the same method used in Study
1 was employed. Five issue
items (hiring
for minorities, defense spending, abortion, environment, and Medicare)
about three political agents'
ideological
stances (Bill Clinton, most conservative, and most liberals) rendered 11
discernible comparisons. This
scale ranged
from O to 11 (M = 8.26, SD = 2.89) and had an internal reliability of .84.
CIVICS KNOWLEDGE
The same 5-item
scale used in Study 1 was used. The number of correct answers to the five
questions was
counted. Because
one of the questions had two correct answers, this measure ranged from
O to 6 (M = 4.60, SD
= 1.46). An
internal reliability of this measure was .57.
POLITICAL INVOLVEMENT
Among the seven
items used in Study 1, six of them were employed here except for the one
about participation in
a march or demonstration.
This measure ranged from 0 to 6 (M = 3.15, SD = 1.53) with an internal
reliability of
.64.
ATTENTION TO ELECTION CAMPAIGN
Media attention
was measured by summing three 5-point scales measuring the extent of attention
that participants
paid to the
1992 presidential election campaign during the last 2 months of the campaign.
The measure was
constructed
based on the average value of three items, including the extent of paying
attention to newspapers, the
extent of paying
attention to television programs, and the extent of paying attention to
the political campaigns. This
measure had
a reliability coefficient of .81 with a mean of 3.55 (SD = .90).
MEDIA NEWS EXPOSURE
Television news
exposure consisted of three items about network news, local news, and television
interviews.
Because the
items were scaled slightly differently, they were standardized and summed.
This measure was found
to be internally
reliable with Cronbach's alpha of .78. Print news exposure was measured
by two standardized
interval scales
about newspaper and news magazine consumption (r = .38). Radio exposure
consisted of two
items about
radio news and radio talk show (r = .59).
LEARNING
Two sets of recall
items were employed to measure recall: issue recall and strategy recall.
Participants (except for
those in the
control condition) received either issue or strategy news stories about
the simulated mayoral election
campaign. Fifteen
issue recall items and eight strategy recall items were asked. For example,
one issue recall
question was
"Which candidate for mayor of Philadelphia favored privatization of park
maintenance and trash
collection?"
One strategy recall was "Which candidate had the most to lose in the debates?"
The number of
correct answers
to each type of question was counted and divided by the number of valid
answers. Issue recall
had a mean of
.57 (SD = .21) whereas strategy recall had a mean of .69 (SD = .22).
ISSUE ELABORATION INDEX
An index of issue
elaboration was based on answers to an open-ended question. During the
posttest, participants
were asked to
write "the most important things that come to mind" from the news materials
that they had watched
and read about
the Philadelphia mayoral candidates, the issues, or the campaign. The answers
to this question
were coded by
two primary coders following a set of predetermined rules. The coding rules
were constructed to
capture a conceptual
unit called an event. An event conveys a thought that has information about
an agent, an
&