One of the ways that OFCCP may have failed to apply the rule of law is by "a failure to achieve rules at all, so that every issue must be decided on an ad hoc basis." Lon L. Fuller, The Morality of Law, at 39. OFCCP's adventurous concept of "Pay Analysis Groups" presents an important example of this fundamental problem.
In 2013, OFCCP rescinded the Agency's 2006 compensation standards (Interpreting Nondiscrimination Requirements of Executive Order 11246 With Respect to Systemic Compensation Discrimination, 71 Fed. Reg. 35,124-141 (June 16, 2006)) and replaced them with Directive 307. One of the profound changes that OFCCP adopted with Directive 307 was a turn to highly aggregated regression analyses organized by so-called "Pay Analysis Groups." Plaintiffs' lawyers and their experts have long pursued highly aggregated regression analyses because those analyses dramatically increase sample sizes, thereby amplifying small average differences in pay and making them appear to be statistically significant. See William T. Bielby and Pamela Coukos, "Statistical Dueling with Unconventional Weapons: What Courts Should Know about Experts in Employment Discrimination Class Actions, 56 Emory L. J. 1563 (2007).
Directive 307 defines "Pay Analysis Groups" as:
A group of employees (potentially from multiple job titles, units, categories and/or job groups) who are comparable for purposes of the contractor's pay practices. Regression analysis may be performed on different types of pay analysis groups. A pay analysis group may be limited to a single job or title, or may include multiple distinct units or categories of workers. A pay analysis group may combine employees in different jobs or groups, with statistical controls to ensure that workers are similarly situated.
Directive 307, at § 7. OFCCP outlined the process by which its auditors were instructed to develop "Pay Analysis Groups" in the Notice of Rescission:
OFCCP will develop pay analysis groups by considering the following, at a minimum: The particular industry, the types of jobs and compensation at issue, the contractor's actual compensation practices and available data. Compensation practices may differ by role (e.g., executives, managers, supervisors, and individual contributors), by level (with higher-level employees tending to receive additional or alternate forms of compensation), by function (such as sales employees who are paid on commission), by unit (department, division, location, etc.) and/or by job classification (exempt or non-exempt, part time or full time, bargaining unit, etc.). This information may be found through a review of the contractor's policy or training documents, description of its compensation system or practices, compensation data, records and coding, job descriptions, and other facts relevant to determining groups, such as the ability of workers to rotate or transfer among different positions within a business unit, a common hiring or selection process, a common performance review practice or other common identifiable employment practice relevant to compensation.
- 78 Fed. Reg. at 13,520.
- 78 Fed. Reg. at 13,520.
When implementing Directive 307 in audits over the past five years, OFCCP has requested data from contractors on a wide variety of possible classification structures, including department, business unit, grade, job family, job sub-family, job function, job sub-function, career level, management level, etc. Upon receipt of the data, OFCCP appears to run regression models by any and all of these various structures, as well as combinations of the structures. It is widely perceived by the contractor community that the Agency determines what it will deem to be the appropriate structure based on the results of testing these many models, i.e., the appropriate structure is the one that identifies statistically significant differences in pay. OFCCP contributes to these perceptions by typically refusing to explain how it determined whether a particular classification among others was the appropriate "Pay Analysis Group."
However, the problem is even deeper than the perception of a purely results-oriented review process. In particular, the regulatory descriptions of "Pay Analysis Groups" do not provide any standard for determining the appropriate regression structure. Unfortunately, OFCCP did not afford contractors an opportunity to provide input as to these issues in the rescission process. The Notice of Proposed Rescission never mentioned a concern about "pooled regressions" or the concept of a "Pay Analysis Group." See 76 Fed. Reg. at 62-64. These issues were first discussed in the Notice of Final Rescission, to which contractors had no opportunity to comment. See 78 Fed. Reg. at 13,513, 13,519-520. In fact, OFCCP expressly rejected requests from the contractor community for an opportunity to comment on the alternative approach that the Agency would adopt after rescission of the 2006 Standards. See 78 Fed. Reg. at 13,517-518 ("An employer association noted that if OFCCP rescinds the Standards and Guidelines, ‘new guidelines should be established through a formal public rulemaking process that mirrors the EEOC's enforcement of Title VII.'").
Regulated entities have a due process right to "fair notice" under "the principle that agencies should provide regulated parties fair warning... 
While OFCCP's so-called "Pay Analysis Groups" may be too ill-defined to provide fair notice to contractors, there are even more significant problems lurking: Pay Analysis Groups are inconsistent with the Supreme Court's discussion of statistical aggregation in Wal-Mart Stores, Inc. v. Dukes et al., 564 U.S. 338, 356-57 (2011). In Dukes, the Supreme Court explained that statistical aggregation in regressions designed to demonstrate pay discrimination must be structured (1) to evaluate pay decisions of a common decision-maker, 564 U.S. at 350, or (2) to assess whether a common pay decision-making criteria caused a disparate impact in pay, 564 U.S. at 353. OFCCP's definition of Pay Analysis Groups makes no mention of these factors nor does it bear any relationship to a recognized theory of discrimination.
Remarkably, OFCCP did not even mention, much less discuss, Dukes when adopting "Pay Analysis Groups" in the Notice of Final Rescission or in Directive 307 in 2013. Instead, OFCCP relied on cases decided before Dukes. For example, OFCCP cited cases where the courts declined to even evaluate the statistical evidence based on a concern with making a premature merits determination at the class certification stage. See Satchell v. FedEx Corp., No. C 03-02659 SI, 2005 WL 2397522, at *6 (N.D. Cal. Sept. 28, 2005) ("it is not appropriate at the class certification phase for the Court to decide which party's statistical model is correct"); Warren v. Xerox Corp., No. 01-CV-2909 (JG), 2004 WL 1562884, at *9 (E.D.N.Y. Jan. 26, 2004) ("‘statistical dueling' is proscribed at the class certification stage"); Hnot v. Willis Grp. Holdings Ltd., 228 F.R.D. 476, 481 (S.D.N.Y. 2005) ("‘statistical dueling' is not relevant to the certification determination"). The Supreme Court expressly rejected this "hands off" approach in Dukes. 564 U.S. at 350-52 ("‘[T]he class determination generally involves considerations that are enmeshed in the factual and legal issues comprising the plaintiff's cause of action.'" (quoting General Telephone Co. of Southwest v. Falcon, 457 U.S. 147, 160 (1982))) and at 354 ("proof of commonality necessarily overlaps with respondents' merits contention that Wal-Mart engages in a pattern or practice of discrimination."). 1
Because the cases relied on by OFCCP in the Notice of Final Rescission were decided before Dukes, none of those lower courts had the benefit of the Supreme Court's discussion of statistical proof in the context of an aggregate pay discrimination claim. Further, many of the lower courts in those cases indicated a concern with adequate sample size. OFCCP's typical approach to Pay Analysis Groups often could find no justification based on sample size concerns, because the Agency has typically aggregated across very large populations numbering in the several hundreds or even thousands. Further, the 2006 Standards addressed sample size concerns by allowing "pooled regressions" with safeguards that required inclusion of "interaction terms" if called for by the "Chow test." 71 Fed. Reg. at 35,140. In the Notice of Final Rescission, OFCCP rejected application of the Chow test, again relying on cases where the lower court's reasoning was inconsistent with Dukes. See Taylor v. District of Columbia Water & Sewer Auth., 241 F.R.D. 33, (D.D.C. 2007) ("More significantly, such ‘statistical dueling' is irrelevant to the certification determination."). And OFCCP overlooked other courts in Title VII systemic discrimination cases that have recognized the need to consider interaction terms in regression models. See, e.g., Coates v. Johnson & Johnson, 756 F.2d 524, 541-42 (7th Cir. 1985) ("The more important distinction between this case and Capaci is that here the defendants' expert testified that he had performed a Chow test that indicated that it was statistically inappropriate to pool the data. Nothing in the Capaci opinion suggests that the company's expert offered any statistical reasons why pooling would lead to inaccurate statistical results."); Eastland v. Tennessee Valley Authority, 704 F.2d 613, 624-25 and n. 16 (11th Cir. 1983) (discussing need for interaction terms when aggregating across different job categories).
Because the so-called "Pay Analysis Group" concept is too vague to provide meaningful guidance to federal contractors, it deprives regulated entities of fair notice and due process. Equally significant for incoming leadership intent on adhering to the rule of law, the "Pay Analysis Group" finds no support in judicial interpretations of Title VII, including the decisions of the Supreme Court of the United States.
1. See also In re Titanium Dioxide Antitrust Litig., 284 F.R.D. 328, 336–37 (D. Md. 2012), amended, 962 F. Supp. 2d 840 (D. Md. 2013) (“The rigorous analysis that must be undertaken in the class certification context extends to disputes between experts: Resolving expert disputes in order to determine whether a class certification requirement has been met is always a task for the court – no matter whether a dispute might appear to implicate the ‘credibility’ of one or more experts....”) (internal citations and quotation marks omitted); In re Air Cargo Shipping Servs. Antitrust Litig., No. 06-MD-1175 JG VVP, 2014 WL 7882100, at *41–43 (E.D.N.Y. Oct. 15, 2014), report and recommendation adopted, No. 06-MD-1775 JG VVP, 2015 WL 5093503 (E.D.N.Y. July 10, 2015) (stating that “courts may not categorically defer expert disputes to the merits, and must instead take a ‘hard look’ at the probative value of expert testimony before relying on it to certify a class”). ↵