Today's guest blogger is Patricia Gillette, a partner at Orrick, Herrington & Sutcliffe.

Making a list and checking it twice. That's what Santa does. And then he brings gifts. For a long time now, law firms have been helping the legal industry’s equivalent of Santa (read "law firm publications") make lists to bring the ultimate gift: the appearance of success on any number of metrics in the eyes of our clients, peers, and colleagues. We have lists that measure diversity, women's progress, profits per partner, number of associates hired from top schools, mentoring programs, part-time policies, parenting policies, and top clients. You name it, law firms are ready to gather the information necessary to get on "the list."         

This list mania relies on two axioms: Lawyers love data, and they are competitive. What better way to inspire participation, than to provide an exercise that plays to both of these drives? Firms gather data, massaging it to make certain that the firm looks at least as good as its competitors. The lists keep expanding, leading one diversity coordinator for a large firm to estimate that she responds to more than 100 inquiries per year for data—each demanding different formatting and data calculations. Now, it seems that every law/business publication has a list, every diversity organization has a list, and every law school has a list.

The initial inspiration for this exercise was a good one: to bring some transparency to the profession and to pressure firms to improve gender and racial diversity, associate satisfaction, and client service. But what has the proliferation of these lists done for lawyers and law firms? Has it improved diversity or increased the number of women in leadership or equity partnership ranks? Has life improved for associates?

Yes, the lists have made firms evaluate and confront some important issues. But they have also inspired some law firms to engage in creative data reporting. It is not that firms are lying outright—but if you give lawyers the opportunity to present data in a way to make their point, they will do that. So when firms are asked about profits per partner, for example, there are ways to "count" people and to present compensation that skews the results favorably.

If we truly wanted to measure firms' diversity or financial success or any other quality, why not have an organization with no constituency and no profit motive—perhaps a law school—take responsibility for creating one survey for all law firms? This group would gather the data and report it without awards or rankings—no luncheon or dinners (paid for by firms) to honor the winners. 

Another alternative is to gather data, either regionally or nationally, without attribution to any firm, so that we can track the bigger picture of how the legal industry is doing. The National Association of Women Lawyer’s Foundation has done this with statistics about women in law firms with great success, giving law firms and the legal industry a report that appears to be more realistic than many other surveys.

While neither of these ideas is perfect, either would be better than the current process. Right now, promotion of women or diversity or other goals aren't being advanced in any meaningful way. In fact, lists can be harmful to firms' efforts, as those that receive the kudos—based on what may or may not be actual data—pat themselves on the back and think the problem is solved. 

In the short term, the solution may be to do what some law firms have already done: Refuse to participate in the list game. Because I think it is time to admit that, just as Santa isn't real, neither are the lists.

Click here to vote for the Careerist as Top ABA Blog in category of Careers/Law Schools. (Voting ends December 20, 2013)

E-mail  Vivia Chen: [email protected]     Follow her on Twitter: https://twitter.com/lawcareerist