Lying in Ponds: Methods

Each column is downloaded from the web. The column is searched, and names, words or phrases which can be identified with either the Democratic Party or Republican Party are tagged. For a person's name to be a political reference, it usually must include a last name ("George Bush", "George W. Bush", "Bush"), although some frequently used first names and nicknames have been added to the database ("Hillary", "Dubya"). Neither personal pronouns nor job titles are tagged, except for some common executive branch names such as "President", "Vice President", and "Commander in Chief". When group names or organizations are used as partisan references, they are tagged ("Bush administration", "administration"). When "White House" is used to refer to a particular administration, it is tagged, but not when it refers to the building. Variations on names of the parties are tagged ("Democrat", "Democratic", "GOP", etc.). References to political figures before World War II are ignored, based on the assumption that they are rarely or never used for current partisan arguments, so references to them are irrelevant to the question of partisanship in the punditry.
Each tagged partisan reference is examined. In some cases, a tagged reference turns out to be a normal word instead of a political reference ("dole" instead of "Dole", "gore" instead of "Gore"), so it is must be untagged.. Some names occur in both parties and must be clarified ("Ford" could be a reference to Gerald Ford or Harold Ford).
Each correctly-tagged Democratic or Republican reference is evaluated as positive, negative or neutral. Both the immediate context of the reference and the overall tone of the column is considered. If a column is a partisan screed, nearly every reference to the despised party is usually evaluated as negative. If a column is more nuanced, close calls are usually evaluated to be neutral. When something positive or negative is expressed, it is usually evaluated to be a positive or negative reference, even if it is a simple fact such as that a politician or party lost an election. When the columnist quotes someone else saying something positive or negative about a political party or figure, it can be either a positive or negative reference depending on the context of the quotation (respectful, mocking). When political figures are themselves quoted, it usually is evaluated as a positive reference unless the statement is clearly being ridiculed. An attempt is made to detect sarcasm and satire, but not to read the columnist's mind or use knowledge of previous writings to guess at ambiguous references.
Since evaluating partisan references is completely subjective, the results depend on who is doing the evaluating, and there's plenty of room for disagreement on how to do it. Lying in Ponds tries to be consistent, but some types of references are more difficult to judge. The most difficult columns are often those which focus on the political process, rather than issues and ideology. It helps that those type of columns (e.g. David Broder, E. J. Dionne) are usually not very partisan.
After all the tagged references are evaluated, the Partisanship Index (PI) for the column is calculated. It is given as:
```
                      (D+ + R-) - (R+ + D-) 
        PI = 100 * ---------------------------                       (1)
                   D+ + D- + D= + R+ + R- + R=
```
where D+, D- and D= are the number of positive, negative and neutral Democratic references, and R+, R- and R= are positive, negative and neutral Republican references. The numerator is the difference between Democratic- and Republican-leaning references, and the denominator is the total number of all kinds of references, including neutral ones. When presenting the numbers, the absolute value of the PI is usually shown in color, with blue indicating a Democratic-leaning PI (positive PI, first term in the numerator larger than the second) and red indicating Republican. The absolute value of PI ranges from 0 to 100, with 0 meaning that the references are exactly balanced between the parties, and 100 meaning that all the references favor only one party. When there are no references to either party, the PI is numerically undefined, but is assigned a value of zero.
The PI of an individual column can be misleading. A pundit could write a non-political column, but a single offhand negative or positive reference to a political figure would make the PI for that column equal to 100.
Beginning in 2004, the PI calculation has been modified by normalizing for party occurrence. Because of political circumstances (control of the White House, Congress, etc.), references to one party may dominate for extended periods. This generally leads to lower scores for pundits of the dominant party and higher scores for pundits of the opposition party, because columnists tend to be more strongly unfavorable toward the other party than they are favorable toward their own. To illustrate this, the PI can be rearranged:
```
         PI = 100 * ( DF*DI - RF*RI )                                (2)
```
where the Democratic and Republican Fractions are:
```
                       D+ + D- + D=
         DF = -----------------------------
              (D+ + D- + D= + R+ + R- + R=)

                       R+ + R- + R=
         RF = -----------------------------
              (D+ + D- + D= + R+ + R- + R=)
```
and the Democratic and Republican Indices are:
```
                       D+ - D-
         DI = 100 * --------------                                   (3)
                    (D+ + D- + D=)
```
```
                       R+ - R-
         RI = 100 * --------------                                   (4)
                    (R+ + R- + R=)
```
From equation (2), it can be seen that a pundit who treats both parties exactly the same (DI equal to RI) would nevertheless have a nonzero PI if they refer to one party more than the other (DF not equal to RF), even if the disparity is fully explained by the fact that one party is dominating the political news. To remedy this, normalization factors are calculated based on the frequency of party occurrence for the entire set of pundits over the entire current year. If there are more total Democratic references (DTOT) than total Republican references (RTOT) for the year, then the Democratic and Republican normalization factors are:
```
                RTOT
           DN = ---- ,  RN = 1                                       (5)
                DTOT
```
or if the reverse is true:
```
                         DTOT
           DN = 1,  RN = ----                                        (6)
                         RTOT
```
The normalization doesn't affect the Democratic and Republican Indices, but the Democratic and Republican Fractions are replaced by Normalized Democratic and Republican Fractions:
```
                         DN*(D+ + D- + D=)
         NDF = -------------------------------------
               DN*(D+ + D- + D=) + RN*(R+ + R- + R=)

                         RN*(R+ + R- + R=)
         NRF = -------------------------------------
               DN*(D+ + D- + D=) + RN*(R+ + R- + R=)
```
Then the Normalized Partisanship Index (NPI) becomes:
```
           NPI = 100 * ( NDF*DI - NRF*RI )                           (7)
```
The effect is to "devalue" references to the dominant party in all calculations. With this modification, a columnist would properly receive a zero value of NPI if they treat both parties equally and have the same ratio of party references as the Lying in Ponds roster of pundits.
The premise of Lying in Ponds is that the measurement of partisanship becomes meaningful when evaluated over many columns. Two different methods are used to aggregate the partisanship index over a set of columns. The simplest way is to calculate a Normalized Total Partisanship Index by applying equation (7) to the total numbers of the various kinds of references. A high value of Total PI generally indicates intense partisanship, but in some cases a columnist who writes mostly non-political columns will have a misleadingly high Total PI because it is based on only a few references.
To measure the consistency of partisanship over multiple columns, the Normalized Median Partisanship Index is calculated. The Normalized Median PI is the median value of the set of individual normalized column PI values, where the sign of each column PI is retained. For instance, Paul Krugman's columns in January 2002 had PI values of 0, 28, 87, 100, 73, 100, 64, 100 and 67 (all the nonzero values were positive, or Democratic). If those values are ordered, then the median (or central) value is 73. Peggy Noonan's January 2002 columns were -72, 86, -3, -17, and -75 (the negative values indicate Republican-leaning columns), giving a Median PI of -17. For comparison of pundits, the absolute value is again used and color is used to indicate the sign. Pundits who write mostly non-political columns generally have a Normalized Median PI of zero.
Since both of these measures of partisanship are considered to be important, they are finally averaged together to form the Normalized Combined Partisanship Index. The Normalized Combined PI is the variable chiefly used to compare pundits at Lying in Ponds; it is used for the Top Ten list of the most partisan pundits on the main page.

Some additional variables may be computed. The Positive Index is:

                              D+ + R+ 
        POS = 100 * ---------------------------                         (6)
                    D+ + D- + D= + R+ + R- + R=

The Negative Index is:

                              D- + R-
        NEG = 100 * ---------------------------                         (7)
                    D+ + D- + D= + R+ + R- + R=

The Neutral Index is:

                              D= + R= 
        NEU = 100 * ---------------------------                         (8)
                    D+ + D- + D= + R+ + R- + R=