FanNation - The Republic of Sport

This is rather long, so pack a lunch and a beverage or two …

A common theme in the NCAA football part of FanNation is the topic of “strength of schedule” (SOS).  Given that there’s no play-off to decide the national championship (NC) in the FBS (formerly Division 1-A), it all comes down to what many fans call a “beauty contest”.  There’s a great deal of truth in that pejorative label, and a big part of a team’s 'beauty' is based on the perceived strength of the teams on its schedule.  A team’s schedule in any given year for all but the scattered few remaining independent teams (including Notre Dame) is composed of two parts:  it’s conference schedule and its out-of-conference (OOC) schedule.  

Generally, schedules are determined years in advance, although they can be changed by mutual agreement between teams when conflicts arise and circumstances require a change.  Forecasting your SOS isn’t so easy when teams change from year to year.  Even perennial winners can have a down season or two.  Perennial doormats can become powerhouses for a season or two, often without having shown that potential in the year preceding.  When a schedule is set, it’s not obvious going into the season just what the SOS really is.  Obviously, if USC schedules an OOC game with a Division II opponent, even one that has a tradition of success within that lower division, that would be perceived as a creampuff opponent that would not contribute positively to the SOS for USC that year.  But it that team went on after being trounced by USC to win the Division II championship, that win for USC would necessarily mean more than if that opponent had a mediocre or poor season within that division.  I suspect, without proof, that trying to manipulate your OOC schedule one way or another would be a frustrating exercise, because of the unpredictable nature of your opponents’ team quality when looking at them as opponents years in advance.

When it comes down to the within-conference part of a team’s schedule of games for that year, the perceived strength of the conference is a big issue regarding your SOS.  In some conferences, teams can play every other team in their conference within 8-9 games.  In others, teams play only a part of their conference opponents and a conference championship game at the end of the season decides the conference championship – thereby adding another game to the schedule of those two teams that is presumably against a “quality opponent” at a more or less neutral site.

Digression – Game siting

That brings up a digression.  It’s generally conceded that when a team plays on its home field, it has something of an advantage, whereas a road game is considered a tougher hurdle than a game at home.  In some games (including many bowl games), the site is considered to be neutral – not on the home field of either team.  The true neutrality of a game on a neutral site is open to debate.  The Red River Shootout every year between Oklahoma and Texas is probably as close to a truly neutral site as it’s possible to be.  The fans of each team are on opposite sides of the 50-yard line and the seats are filled in both halves of the stadium (the Cotton Bowl in Dallas).  Yes, the site is in Texas (very close to halfway between Austin, Texas and Norman, Oklahoma), but each team’s fans are very close to equally represented.

In some “neutral site” games, especially bowl games, one team has a decided advantage.  If either Kansas or Kansas State or Missouri play in a conference championship in Kansas City, their opponent is likely to be rather underrepresented among the fan attendees at the game.  When USC plays a bowl game in the Rose Bowl, or LSU plays in the Sugar Bowl, or any of the Florida teams plays in the Orange Bowl, this is not a truly neutral site.   

Any game not played at the home field of one of the opponents can be considered “neutral” but the degree of neutrality can vary considerably.  And can be a bone of contention between advocates for one team versus those of another.
============= end digression ===========

Anyway, every  conference has its top teams and its doormats every year.  A schedule that includes more of the top teams would naturally be considered more difficult than a schedule that had a majority of the creampuff teams in that conference.  The issue of who are the good teams in that conference can’t be settled by the rankings until the season has played out.  The rankings are often badly out of line with reality at the beginning of a college football (CFB) season, and although they gradually improve in this regard as the season wears on, it’s not until the final games of the season (including bowl games) are played that the rankings can be said to be reasonably accurate.

The determination of the strength of an entire conference relative to other conferences also can’t be done based on the rankings until the end of the season.  And the results may be rather ambiguous – for example, this year the Pac-10 began the season with a dismal record of OOC performance, but wound up the year undefeated in its five bowl games.  How might this be interpreted?  When its five best teams played in bowls, they obviously performed well.  But that says nothing about the rest of the teams in the Pac-10.

Digression – weak teams in a conference

That brings up yet another digression.  Just what impact can be assigned to victories against the weakest teams in a conference?  Since every conference winds up with a set of teams having losing records (at the end of the season), games won against those teams apparently don’t mean as much as games won against in-conference teams with winning records.  If the OOC schedules of the teams in a conference that do poorly within the conference turn out to be mostly wins, that seems to say that even the poor teams within that conference are still quality opponents.  But what if their OOC wins are against the poor teams in other conferences (or against lower division teams in the NCAA)?  Those victories don’t seem so important.  In bowl games, it’s only the teams in each conference with non-losing records that are “bowl eligible”.  Those conference teams with poor records don’t get invited to bowl games at the end of the season, so how they might do against comparable teams in other conferences can’t be known with certainty.  Fans can speculate, but without playing those games, the outcome remains conjecture, not objective fact.

============= end digression ===========

Digression  - head-to-head outcomes

This also brings up yet another digression:  the “best” team doesn’t necessarily win in every game – that is, a head-to-head outcome.  Lots of factors go into determining which team wins a specific game:

1. Officiating
2. Pure dumb luck in one or more “big plays”
3. The weather
4. The site of the game (road/home/”neutral”)
5. Injuries, ejections, suspensions, academic ineligibility
6. Individual match-ups
7. Psychology of the team
8. etc.

Presumably, when a team wins a game against a quality opponent, the outcome frequently hinges on one or more of these factors that could be considered to be outside of the issue of which team is “best” – however we might choose to define specifically what we mean by “best”.  This provides the losing team’s fans with a host of excuses they can offer to explain why the team they favored lost the game.  And the fans of the winning team usually respond by quoting the final score and claiming that the other fans are just making excuses.  There are merits to both sides in this sort of argument.  Given that the W-L record for a team is going to be a major factor in determining the SOS for its opponents, having the outcome of individual games hinge so strongly on these quirks in a particular game is a troubling part of any SOS calculation.

The only definitive way to determine the “best” team would be for every team in the FBS to play every other team in the FBS multiple times each year, with random site selection, weather conditions, player line-ups, etc.  If each one of the teams played a large number (say, 100) games with every other FBS team, then the average of the outcomes would be used in determining a final ranking that would represent a truly definitive outcome.  That means each team would play roughly 100 games against roughly 120 opponents – 12,000 games.  In all, roughly 1,440,000 games.  It’s pretty evident that this is not a practical way to determine which team is best, but it would be definitive!

============= end digression ===========

To return to the main thread of this essay, SOS depends on the teams it plays (in and out of the conference).  What sort of standard would be used to define a "quality opponent"?  A team that wins all of its games, or only loses one or two games is likely to be considered a quality opponent in any case.  Although it's arbitrary, a quality opponent could be a team with more wins than losses, for example  “Bad losses” against relatively weak opponents (teams with fewer wins than losses, for instance) should be counted against the SOS evaluation.  But SOS clearly plays a role in deciding the ultimate ranking for that team.  Since it’s a practical impossibility to have every team’s SOS be the same as every other teams, this leaves open the issue of how SOS is calculated.  Any system for doing so should include a clear explanation of what factors are considered in calculating the SOS and the precise formula used for that calculation.

One factor to consider is rankings within a conference.  Presumably, the W-L record of those games within that conference would accomplish some of what would be needed to rank the teams in a conference at the end of the season.  Unfortunately tie-breakers would be needed.  A list of tie-breakers might include:  head-to-head outcomes, comparative scores with common opponents, OOC wins.  And of course, the SOS rankings of the opponents.  Unfortunately, if you’re going to use SOS of opponents to break ties, you need to know the SOS first.  That is, if we’re going to use conference rankings to determine SOS, you also need to know the SOS in order to break any ties.  This is something of a dilemma!  Since the FBS conferences don’t all have the same number of teams, and there are still a few independents in the FBS, how should the conference ranking figure in?  And for conferences with a head-to-head conference championship game, how should that figure in?

Part of the issue in any formula is that it shouldn’t reward teams (or penalize teams) for unique circumstances.  For example, independents have no conference rankings to help them (or hurt them).  Teams in conferences that play conference championship games shouldn’t be given undue credit for that additional game, which would give them an advantage in the final analysis over teams that don’t play in a conference with a conference championship game.

Any number of approaches could be used to calculate SOS, but it should be evident that no simple measure of SOS is going to be adequate, no matter what that measure is.  And given that it’s impossible to know the SOS before the season is over, any estimate of SOS before the end of season amounts to unproven speculation.  Once the season is over, the major ranking polls only include, say, the top 25 teams.  If you’re going to include the final rankings in a formula for SOS, a method has to be developed for ranking teams that aren’t included in the top 25.  Let me propose some variables to consider using in a multivariate SOS analysis done at the end of the year.

The first step is to determine how much to weight each of the variables.  For winning percentage,  I propose to use that percentage value – say 85% as a number.  A team that wins all of its games gets 100 points.  A 12-2 record gets 85.71 points.  And so on.  This method ensures that since all teams don’t play the same number of games, the contribution from this term doesn’t depend on the number of games.

A team ranked in the top 25 (say, the AP poll) gets an extra 10 points
A team ranked in the top 10 gets another 10 points, for a total of 20 points
A team ranked in the top 5 gets another 10 points, for a total of 30 points

Any road game win gets an extra 5 points
Any home game loss earns a 5 point deduction

Any neutral site win earns an extra 2 points

Any game won against a “quality opponent” – say a team with a winning percentage greater than 50% - gets an extra 5 points

A loss against a “weak opponent” – say a team with a winning percentage less than 50% - means a 15 point deduction

A win against an opponent ranked in the top 25 at the end of the season earns the team an extra 10 points
A win against an opponent ranked in the top 10 at the end of the season earns the team an extra 10 points, for a total of 20 points
A win against an opponent ranked in the top 5 at the end of the season earns the team an extra 10 points for a total of 30 points

A win against a non-FBS team earns a 5 point deduction
A loss to a non-FBS team earns an extra 10 point deduction for “weak opponents”

As an example of this, consider applying it to some selected teams:

Oklahoma
    Winning percentage:  12-2, or 85.71 points
    Final AP ranking:  #5, or 30 points
    Road game wins:  (five – Washington, Baylor, KSU, TAMU, OkSU), or 25 points
    Home game losses: none
    Neutral site wins:  (one – Mizzou – Big 12 Championship game), or 2 points
    Quality opponent wins (seven - Cincinnati, TCU, KU, NU, TTU, OkSU, Mizzou), or 35 points
    Weak opponent losses: none
    Wins against top 25 ranked teams:  (five – Cincinnati, TCU, TTU, OkSU, Mizzou), or 50 points
    Wins against top 10 ranked teams:  (one – TCU), or 10 points
    Wins against non-FBS teams:  (Chattanooga), or – 5 points
    Losses to non-FBS teams:  none
Total points:  223.71

Texas
    Winning percentage:  13-1, or 92.86 points
    Final AP ranking:  #4, or 30 points
    Road game wins:  (three – UTEP, CU, KU), or 15 points
    Home game losses:  none
    Neutral site wins:  (two – OU, OhSU), or 4 points
    Quality opponent wins (eight – FL Atl, Rice, OU, Mizzou, OkSU, TTU, KU, OhSU), or 40 points
    Weak opponent losses: none
    Wins against top 25 ranked teams (four – OU, Mizzou, OkSU, OhSU), or 40 points
    Wins against top 10 ranked teams (two – OU, OhSU), or 20 points
    Wins against top 5 ranked teams (one – OU), or 10 points
    Wins against non-FBS teams:  none
    Losses to non-FBS teams:  none
Total Points:  251.86

Mississippi
    Winning percentage:  9-4, or 69.23 points
    Final AP ranking:  #14, or 10 points
    Road game wins:  (three – UFl, Arkansas, LSU), or 15 points
    Home game losses:  (two – Vanderbilt, South Carolina), or –10 points
    Neutral site wins:  (one – TTU), or 2 points
    Quality opponent wins:  (three - UFl, LSU, TTU), or 15 points
    Weak opponent losses:  none
    Wins against top 25 ranked teams:  (two – UFl, TTU), or 20 points
    Wins against top 10 ranked teams:  (one – UFl), or 10 points
    Wins against top 5 ranked teams:  (one – UFl), or 10 points
    Wins against non-FBS teams:  (two – Samford, LA-Monroe), or –10 points
    Losses to non-FBS teams:  none
Total points:  131.23

Ohio State
    Winning percentage:  10-3, or 76.92 points
    Final AP ranking:  #9, or 20 points
    Road game wins:  (four – WI, MSU, NwU, IL), or 20 points
    Home game losses:  none
    Neutral site wins:  none
    Quality opponent wins:  (five – Troy, MN, WI, MSU, NwU), or 25 points
    Weak opponent losses:  none
    Wins against top 25 ranked teams (one – MSU), or 10 points
    Wins against top 10 ranked teams:  none
    Wins against top 5 ranked teams:  none
    Wins against non-FBS teams: (one – Youngstown State), or –5 points
    Losses to non-FBS teams:  none
Total points:  146.92

Utah
    Winning percentage:  13-0, or 100 points
    Final AP ranking:  #2, or 30 points
    Road game wins:  (six – MI, USU, AF, WY, NM, SDSU), or 30 points
    Home game losses:  none
    Neutral site wins:  (one:  AL), or 2 points
    Quality opponent wins:  (seven – AF, Weber State, OrSU, CSU, TCU, BYU, AL), or 35 points
    Weak opponent losses:  none
    Wins against top 25 ranked teams (four – OrSU, TCU, BYU, AL), or 40 points
    Wins against top 10 ranked teams:  (two – TCU, AL), or 20 points
    Wins against top 5 ranked teams:  none
    Wins against non-FBS teams: (one – Weber State), or –5 points
    Losses to non-FBS teams:  none
Total points:  252.00 points


I could go on but this is a time-consuming process.  Note that as we approach the bottom of the ladder, the team totals could become negative.  If the absolute worst team total is a value of -TOTmin, then if we add the value (+TOTmin) to the totals for all the teams, then the worst team winds up having a total of zero.  And the team with the highest point total, say TOTmax, gets its value boosted to TOTmax+TOTmin.

Once point values have been calculated for every FBS team, then for each team we can calculate the sum of the point totals for all their opponents - call it TOTsum. The higher TOTsum is for a particular team, the greater the SOS for that team.   Let each team's SOS be represented by its TOTsum value.  If we sort all the FBS teams by their SOS values in descending order, we’ll arrive at an SOS ranking order for all the FBS teams.

For the team with the highest SOS value - call it SOSmax - that team's point total remains the same.  For every other team, its final point total could be multiplied by the ratio (SOS/SOSmax) where SOS is the TOTsum for that team, where this ratio is always less than one except for the team with the highest SOS value (whose ratio = 1).  At this point, I can only speculate about what those numbers might actually be, unfortunately.

Once the SOS is determined for each team and the team totals adjusted by the SOS factor can be calculated, it would be a simple matter to use the adjusted team totals to produce an average SOS-adjusted total for all the teams in the conference.  This would be an objective method for ranking the conferences.  

Let the fun begin!!  Comments about  any perceived errors, weaknesses, or omissions in the foregoing will be of interest.  I welcome suggestions for improvements or modifications within this general framework.  Please, only serious comments.  No homerisms are welcome.

Perhaps during the offseason I can do this for all the FBS teams. 

Note added:  I thought of another factor to include:  the W-L percentage for teams beaten, and the W-L percentage for teams lost to.   Take the W-L percentage for teams beaten, and multiply that by 100.  That gives a point value - call it 'WLPts'. Then take the W-L percentage for teams lost to and multiply that times WLPts.  If the team is unbeaten for the year, then its points are simply WLPts.

For Oklahoma:  W-L percentage x 100 for teams beaten (83-70) = 54.24 WLPts.  W-L percentage for teams lost to (25-2) = 0.9259.  Final product = 54.24*0.9259 = 50.23 points

For Texas: W-L percentage x 100 for teams beaten (89-64) = 58.17 WLPts.  W-L percentage for teams lost to (11-2) = 0.8462.  Final product = 58.17*0.8462 = 49.22 points

For Mississippi: W-L percentage x 100 for teams beaten (62-50) = 55.36 WLPts.  W-L percentage for teams lost to (34-19) = 0.6415.  Final product = 55.36*0.6415 = 35.51 points

For Utah:  W-L percentage x 100 for teams beaten (88-77) = 53.33 WLPts.  W-L percentage for teams lost to (no losses) = XX.  Final product = 53.33*1 = 53.33 points

For Ohio State: W-L percentage x 100 for teams beaten (71-76) = 48.30 WLPts.  W-L percentage for teams lost to (24-2) = 0.9231.  Final product = 48.30*0.9231 = 44.59 points

Simply add these to the previous point totals to get the revised point totals for these five teams. 

This system penalizes teams for playing, and especially for losing to, mediocre teams (e.g., Mississippi).  Losing to high-quality teams gives only a slight penalty - going undefeated is clearly helpful, but most of the teams you beat need to have good W-L records.  Wins against teams with poor W-L records don't help (beating 0-12 Washington did OU no service).