📄 http:^^www.cs.washington.edu^homes^lazowska^myd.html
字号:
Date: Tue, 10 Dec 1996 15:07:43 GMTServer: NCSA/1.4.2Content-type: text/htmlLast-modified: Thu, 18 Jul 1996 05:43:23 GMTContent-length: 11700<html><head><title>"Median Years to Ph.D." is not what you think!</title></head><body bgcolor="#ffffff" text="#000070"><h1>"Median Years to Ph.D." in new Conference Boardstudy of doctorate programs is not what you think!</h1><h3><!WA0><!WA0><a href="http://www.cs.washington.edu/homes/lazowska/lazowska.html">EdLazowska</a><br><!WA1><!WA1><a href="http://www.cs.washington.edu/index.html">Departmentof Computer Science & Engineering</a><br><!WA2><!WA2><a href="http://www.washington.edu/index.html">Universityof Washington</a></h3><h4>September 1995</h4><p><hr><p>The just-released Conference Board study of research-doctorateprograms in the United Statesincludes a measure for each program labeled <b>"MedianYears to Degree,"</b> which is widely interpreted to be "themedian number of years that students spend in this graduateprogram." <b>Don't be fooled!</b><p>In fact, what the study reports is bestdescribed as <b>"the median number of years that elapsefrom when thestudent first enters any educational program in any fieldat any institution after receiving his/her Bachelors degree,until the student receives his/her Ph.D."</b><p>Suppose, for example, that a student enters a Mastersprogram immediately after receiving his/her Bachelorsdegree, and graduates from this Masters program in 2years. Then the student enters the workforce for 5years. Wanting to make the transition to research(perhaps in an entirely different field than theMasters degree!),the student then enrolls in a Ph.D. program, fromwhich s/he graduates 4 years later. The Ph.D.-grantinginstitution probably feels prettygood -- cranked this student out in 4 years! But inthe Conference Board study, this student will weighin at 2 + 5 + 4 = 11 years!<p>This semantic confusion is one issue: we all attribute aparticular semantics to the "MYD" measure, which is not atall what it actually represents. A careful definition of"MYD" would reduce confusion somewhat. But many wouldstill be confused by the misleading title.And even if the semantic confusion could be cleared up,my opinion is thatthe "MYD" measure does not convey information thatcharacterizes in a meaningful way the graduate programto which it is attached: it is not relevant to astudent trying to choosebetween graduate programs, nor to an administratorlooking for bloated programs.I might even argue that it does not represent somethingworth tabulating and reporting at all, and that it's confusingto do so. (Looking more broadly than individual graduateprograms, I question whether "MYD" is even germane to a field asa whole, since so many factors contribute to the measure.)<h3>Background</h3>In the Department of Computer Science & Engineeringat the University of Washington, we routinely calculatethe median time that students spend in our doctoralprogram. This number has been stable at between 5 and 6years for more than a decade. (We do not require a Mastersdegree en route to a Ph.D., so this number represents"total time in graduate school" for a student who entersdirectly from a bachelors program. There are no trickysemantics here -- it's exactly what you'd expect.)<p>We were, therefore, surprised when the new Conference Boardstudy reported 8.19 years as the "MYD" for our program. Spurredon by MIT, which noticed a similar phenomenon, we exploredfurther.<p>Our first step was to calculate the median time spentin our program for graduates in the specific yearsconsidered by the Conference Board study. We didthis, using our own database, and confirmed a valuein the 5 to 6 year range.<p>Next, withthe help of our Graduate School, we obtained datadirectly from the NRC Survey of Earned Doctorates -- theactual data that had been used as input to the Conference Boardstudy.(Graduating students fill out an SED form which is sentto NRC.)The SED form asks the student for a wide range ofdata: year of high school graduation, years ofattendance at every college (including 2-year) andgraduate institution where the student has spent time,full time equivalent years as a student since receiptof first Bachelors degree, etc.While there were of course a few glitches among our 60-oddgraduates over the multi-year reporting interval, overallthe return rate was very high and the quality of the datawas very good.We calculated a variety of measures from this data, andformed a variety of hypotheses.<p>Finally, NRC staff provided essential assistance by re-workingtheir calculations for our program and reviewing them with us.Without this assistance -- way beyond the call of duty -- wewould still be speculating. (Data gathering and analysisfor the studyare the responsibility of NRC's Office of Scientific andEngineering Personnel, which looks at human resource issuesacross all science and engineering fields.)<h3>What "MYD" Really Means</h3>As noted in the preamble, <b>the "MYD" measure in theConference Board study</b>, while widely interpreted to be"the median number of years that students spend in thisgraduate program," is in fact <b>"the median number ofyears that elapse from when the student first enters anyeducational program in any field at any institution afterreceiving his/her Bachelors degree, until the student receiveshis/her Ph.D."</b> Indisciplines or instances where significant employmentoccurs between receipt of a Masters degree and entryinto a Ph.D. program, the difference can be huge.<p>This is not a measure that we've ever calculated forour graduate program, nor is it a measure that weconsider particularly germane. Surely, time spentfully employed as part of a career plan, betweenreceiving a Masters degree from some other institutionand enrolling in our graduate program, is notcharacteristic of our graduate program. (Pushing abit harder, it's not even obvious that the time spentin that Masters program elsewhere is germane, since wedon't require a Masters degree en route to thePh.D., and all students, regardless of background,enter our program on an even footing in terms ofthe "checkpoints" of the program.)<p>This is by far the greatest source of the discrepancybetween the "MYD" figure reported by the ConferenceBoard study and our own intuition about our graduateprogram.It's worth noting, though, that even when we usethe Conference Board's "MYD" definition and calculatethis measure from our own database, we obtainsomewhat different results than the Conference Boardstudy. There are several secondary contributing factorswhich may be of interest.<p>First, the Conference Board study calculates an integernumber of years for each student, by subtracting thecalendar year of entry from the calendar year of exit.A student who enters in September of Year X andgraduates in January of Year X+5 actually spent 4.33years in the program, but will be reported as 5 -- asmall but consistent effect, since most students firstenroll in the fall.<p>(It's worth noting, in this context, how the studyarrives at an "MYD" that is reported to two decimaldigits. Those students who fall in the median yearare considered to have graduated uniformly acrossthat year, and based upon this, an offset within thatyear is calculated to two digits and reported.)<p>Second, students occasionally mis-code themselves. Inthe case of our own program, four students coded themselvesas "Computer Engineering" rather than "Computer Science"and were attributed to our Electrical Engineering departmentin the study ... offset by four students we'd never heard ofwho coded themselves "Computer Science!"<p>Third, students who omit essential fields from theSED form must of course be omitted from the calculation.This affected a non-negligible number of our graduates.<p>For simplicity, this explanation has been presented in the contextof the University of Washington Department of ComputerScience & Engineering, but it applies to all programssurveyed in the Conference Board study.<h3>Lessons</h3><i>State definitions precisely.</i> From theConference Board study document, one would beunlikely to discern this definition of "MYD" andits implications.<p><i>Avoid using titles that will be assumed by manyto mean something other than what is really being reported.</i> Itis better to choose a title with no obvious semantics than one withthe wrong obvious semantics.<p><i>Be mindful of correct definitions when makingstatements.</i> Statements in the Conference Boardstudy document such as "It took graduates in the1980s longer to earn a degree on average thangraduates of these programs took 10 years earlier"would seem to contribute to misinterpretation.<p><i>Consider the appropriateness of measures.</i> Understandingthe definition of "MYD" will allow the community toconsider if this is the most appropriate measure. TheSED form includes a wide variety of data; "MYD" isthe measure that the Conference Board studyhas <i>chosen</i> to calculate and use.<p><i>Don't confuse accuracy and precision.</i> The ConferenceBoard study reports to two decimal digits a widely-misunderstoodmeasure with lots of fuzz in it.<p><i>Handling survey instruments is difficult.</i> Codingerrors are inevitable. If the community wantsreliable analyses, we are going tohave to take the time to verify that we are providingreliable data.<h3>Acknowledgements</h3>Jeff Dean, a graduate student in our department,noticed the anomalous figure reported for us immediatelyafter the <!WA3><!WA3><a href="http://cra.org">Computing ResearchAssociation</a> placedthe <!WA4><!WA4><a href="http://cra.org/cgi-bin/RankCS">Conference Boardstudy's Computer Science information on the Web</a>.(Juan Osuna at CRA was responsible for thiseffort, and also provided much assistance in trackingthings down.) John Guttag of MIT contacted me after notingthe same anomaly for his program, and furnished considerableguidance.<p>At the University of Washington, contributionscame from Frankye Jones (our staff graduate programadvisor), Carl Ebeling (our faculty graduate programadvisor), Dale Johnson (Dean of the Graduate School),and John Drew (Manager of Computer Services at theGraduate School).<p>At the National Academy of Sciences, CharlotteKuh and Jim Voytuk of the Office of Scientificand Engineering Personnel (the organizationresponsible for the Conference Board study)expended a large amount of time and patiencehelping us understand what was going on.It's important to note the magnitude andcomplexity of the Conference Board study: 41fields, 274 universities, 3,634 research-doctorateprograms, 78,000 faculty members, and, by 1993,nearly 40,000 Ph.D.s awarded per year. MarjoryBlumenthal ofthe Computer Science and TelecommunicationsBoard also provided guidance.<p><hr><p>Related material:<ul><li><!WA5><!WA5><a href="http://cra.org/cgi-bin/RankCS">Computer Sciencedata from the Conference Board study</a>, provided bythe Computing Research Association<p><li><!WA6><!WA6><a href="http://cra.org">Computing ResearchAssociation</a> home page<p><li><!WA7><!WA7><a href=http://www.cs.washington.edu/homes/lazowska/production.html>Massy-Goldman report alleging50% CSE Ph.D. over-production to be re-issued dueto flawed data</a><p><li><!WA8><!WA8><a href="http://www.cs.washington.edu/homes/lazowska/cra">Computing Research: Driving InformationTechnology and the Information Industry Forward</a><p><li><!WA9><!WA9><a href="http://www.nas.edu/nap/bookstore/0309050944.html"><i>Research-DoctoratePrograms in the United States: Continuityand Change</i></a>, National Academy Press, 1995.</ul></body><address><hr><!WA10><!WA10><a href="mailto:lazowska@cs.washington.edu">lazowska@cs.washington.edu</a></address></html>⌨️ 快捷键说明
复制代码
Ctrl + C
搜索代码
Ctrl + F
全屏模式
F11
切换主题
Ctrl + Shift + D
显示快捷键
?
增大字号
Ctrl + =
减小字号
Ctrl + -