This report uses a unique database from the labor market information company Burning Glass and other sources to analyze the skill requirements and the advertisement duration time for millions of job openings. It reaches the following conclusions:

These job openings data provide new evidence that, post-recession, STEM skills, particularly those associated with high levels of educational attainment, are in high demand among employers. Meanwhile, job seekers possessing neither STEM knowledge nor higher education face extraordinary levels of competition for a scarce number of jobs. Governments at all levels, educators, training organizations, and civic leaders can utilize job vacancy data to better understand the opportunities available to workers and the specific skills required of them. Improving educational and training opportunities to acquire STEM knowledge should be part of any strategy to help unemployed or low-wage workers improve their earnings and employability.

Workers with skills in science, technology, engineering, and mathematics (STEM) play a hugely important role in driving innovation and economic growth.¹ As economies have developed in wealth and complexity since the industrial revolution, scientists, engineers, and other STEM workers have grown in relative importance as a share of the labor market, and STEM skills are widely needed across a wide variety of blue-collar, craft, and professional occupations.² Likewise, the rewards for both education and STEM skills have grown relative to other positions, and workers in STEM today are among the highest paid.³

This report addresses a set of issues of increasing interest to policy makers, educators, businesses, and workers of all ages: What skills are most in demand and most likely to lead to a rewarding career? Is there a shortage of workers possessing the skills used in STEM occupations (i.e., STEM skills)? What skills and bodies of knowledge should schools and post-secondary institutions offer to teach students in order to provide them the best chances of economic success?

The results show that skills common to STEM occupations are in short supply relative to demand and are valued more by employers. Moreover, companies located in regions with low unemployment rates for STEM workers have greater difficulty filling their openings, all else being equal. It follows that increased training in STEM fields like computer science and medicine will ease hiring for employers and lead to high-paying career paths for workers.

While this is certainly not the first report to highlight the economic advantages that accrue to those with STEM skills, the data and analysis here are novel and useful in a few ways. First, unlike most reports on the STEM workforce or skill-shortage issues more broadly, this report draws on a database (developed by the labor market information company Burning Glass) that allows one to measure the duration of job vacancy advertisements—a measure of hiring difficulty. There has never been a nationwide analysis of vacancy duration for the United States by occupation. The closest such study came from a 1964 survey conducted by the United States Employment Service of unfilled openings posted at local government employment centers in 78 metropolitan areas.⁴ Among the few recent studies that have looked at vacancy duration, none have examined differences by occupation, let alone skill level, geography, company, educational requirements, or status as a STEM worker, as this report does. This report also draws upon a uniquely large sample size: 1.1 million advertisements were used to calculate summary statistics for the first quarter of 2013, and 3.3 million were used in the formal statistical analysis, across 52,000 companies. These make up the full universe of job advertisements listed on company websites, not a survey, which would be subject to sampling and response bias. Finally, each vacancy is listed on a company website, not a job board, and examining only these listings insures that duplication is not an issue and there are no differences across companies in advertisements costs.

In short, the technological breakthroughs in “big data,” such as those provided by Burning Glass and used here, mean that no previous job vacancy database has ever been assembled that can so thoroughly test labor market theories related to job search, skill mismatch, and shortage.

Even with these advantages, no database is perfect. Job openings posted online tend to require higher levels of skill and education than openings that are not advertised online, and the interpretation of advertisement duration is complicated by the fact that low-skilled, low-wage, high-turnover positions are often advertised continuously (see the appendix for details).

Understanding Job Vacancies in the Context of Long-Run Versus Short-Run Skill Shortage

Over the last three decades, the earnings of college graduates have soared far above the earnings of those with just a high school diploma. Leading labor economists have argued that this trend represents a long-standing shortage (or inadequate growth in supply) of college-educated workers in the United States relative to demand.⁵ As it happens, the earnings for STEM knowledge have also increased at any given level of education and experience. In 1970, workers earned 12 percent higher earnings if they worked in an occupation that was significantly more STEM oriented (scoring one standard deviation higher). By 2012, that premium was up to 21 percent.⁶

The Great Recession obviated long-run shortage problems but only temporarily. As the economy recovered, signs of hiring difficulty for STEM workers returned. For young graduates with STEM degrees, census data show rising relative salaries from 2009 to 2012, the only years with field-of-degree data.⁷ Meanwhile, job vacancy data for the United States in early 2014 reveal five job openings for every unemployed computer worker, 3.3 for every unemployed health care practitioner, and 1.7 and 1.1 for those in architectural/engineering and science occupations, respectively. By contrast, there were just 0.7 openings for those in legal occupations, 0.2 for production workers, and 0.1 for construction workers.⁸

In order for businesses to succeed, they need to match the right employee to the right position. Accordingly, businesses invest heavily in searching for the right employees, through methods such as advertising their vacancies and thoroughly evaluating candidates. According to one survey, the average U.S. company spends $3,500 per new hire in job advertising costs and human resources staff, or $124 billion per year.⁹

Employer surveys report large costs from hiring difficulty, including overworked current staff, lower quality output, loss in revenue, and delays in product delivery.

Meanwhile, previous studies estimate that the average job vacancy lasts approximately 20 days.¹⁰ For companies, lengthy searches are a necessary if expensive cost of matching. Each day a job goes unfilled costs the enterprise the value of that potential worker, and this cost is higher for higher-paying and highly skilled positions, even as the cost of hiring a skilled worker who is the “wrong fit” is also higher. Employer surveys report large costs from hiring difficulty, including overworked current staff, lower quality output, loss in revenue, and delays in product delivery.¹¹ In short, companies have a strong incentive to fill vacancies as quickly as possible, but first they must identify qualified and willing candidates.

Job matching is an ocean of economic activity. Across the United States, 4.7 million workers are hired in a typical month.¹² That churn dwarfs the number of net new jobs created, which has averaged just 160,000 per month since 2010.¹³ Economists have a well-developed theory of job search to guide their thinking on labor market matching and its macroeconomic consequences, and the three leading theorists in this field were awarded the Nobel Prize in 2010.¹⁴ In this framework, hiring is determined by both supply and demand, not just the number of vacancies—a measure of demand. In particular, when there are many unemployed workers looking for work, such as during a recession, jobs vacancies are filled quickly, leading to many new hires for a given level of vacancies. Likewise, when unemployment falls, as in the recovery from the recession, vacancies become harder to fill.¹⁵

Yet, job openings data do not garner the same level of attention from researchers and the media as do other measures like GDP growth, job growth, and the unemployment rate. The reason may be the lack of detailed data that are collected and reported by the government. The Bureau of Labor Statistics’ Job Openings and Labor Turnover Survey (JOLTS) does not provide any information on openings by occupation or even detailed industry. Nor does it provide regional or even state-level information. It is probably not very helpful to a laid-off auto worker in Michigan to know that there is a job opening somewhere in the Midwest in the manufacturing sector, but that is the most detailed information provided by the underfunded JOLTS program. The pioneering work cited above has relied on private-sector data sources, such as that provided by the Conference Board.¹⁶

The analysis here confirms the broad theoretical predictions of “search theory”—that hiring difficulty is affected by both supply and demand—and provides new insight into the characteristics of jobs that take long periods to fill. This report goes beyond previous studies by considering a much larger and richer set of factors that influence job duration—including the local labor market, the value of specific skills required to do the job, and the company doing the recruiting.

After summarizing previous research and describing the data, this paper will analyze trends in hiring difficulties by skill, education, and occupation, with a focus on STEM occupations and the skills common to them. The results show that there is a national relative shortage of workers with STEM skills, and the shortage is roughly as severe now as before the recession. Generally, more valuable STEM skills—measured by salary offers—are more difficult to fill. The problem is particularly acute in specific regions with few STEM workers or few unemployed STEM workers. These results imply that enhanced STEM training and education would have enormous benefits for individuals, firms, and regions.

Summary of Previous Studies of Shortages and Vacancies

The formal economic analysis of job vacancies as a sign of shortage issues goes back at least to the 1950s. Kenneth Arrow and William Capron argued that unfilled vacancies represent a “dynamic shortage” that will persist until wages rise sufficiently to attract enough workers to acquire the necessary skills.¹⁷ While the theory implies shortages will only be temporary, long-term shortages can persist in reality because financial incentives have a very weak effect on the learning of children, as recent experimental evidence demonstrates.¹⁸

Empirical research on job vacancies has been limited by a dearth of data, but a few published articles shed light on how vacancy data can be used to discern skill shortage. In the 1960s, economists analyzed openings data collected from government staffing agencies and found that highly skilled positions took longer to fill and that employers cited a lack of qualified candidates as the most common explanation for why jobs were hard to fill. Consistent with the claims of employers, the studies also documented fewer job applicants per vacancy for more skilled positions. This gap, moreover, varied widely across regions, suggesting that skill mismatches have an important geographic component.¹⁹

More recent work confirms that job vacancies requiring more education, training, or experience take longer to fill.²⁰ Economists have interpreted long vacancy periods as synonymous with shortage, after adjusting for job characteristics that would make the open position unattractive, such as low wages.²¹ Openings for higher-skilled positions receive fewer initial applications, prolonging the duration of the vacancy.²² There is a strong correlation between firm reports of shortage and the duration of vacancies.²³

Shortages, in this sense, can be attributed to firm factors—such as rapidly increasing demand—or regional labor market factors—such as the presence of few skilled workers and low unemployment rates.²⁴ Shortages are particularly common in firms using advanced technologies, suggesting that skill shortages will persist unless there is continued advancement in the attainment of technical skills.²⁵ Other evidence shows that raising wages may not be sufficient to fully address shortages. Firms that pay high relative wages often report shortages.²⁶ Finally, skill shortages appear to negatively affect firm productivity and growth.²⁷ To summarize, there is small but consistent academic literature establishing the usefulness of job vacancy data to interpret skill shortages and other labor market trends.

Despite this literature, however, a few scholars have recently argued that skill shortages are created by bad human resource practices or do not exist at all.

The first argument suggests that the adoption of flawed software algorithms used to sort through resumes are excluding many qualified candidates for skilled positions and thus leading firms to misdiagnose their recruitment challenges as shortage.²⁸ While probably true in some cases, it seems unlikely that human resource practitioners are systematically sabotaging—accidentally or not—their companies’ profits by excluding thousands of qualified workers for trivial reasons. Moreover, this software-based argument cannot explain why the duration of job vacancies fell during the recession and has since increased, as will be shown below. Nonetheless, the data analyzed below adjust for human resource practices specific to companies to test if this theory explains differences in vacancy duration across skill levels.

A related and more persuasive argument along these lines is that firms are not training as much as they used to and are putting too much pressure on workers to have the perfect skills before being hired. Systematic evidence is not available on this point, but a recent global survey found that only 10 percent of companies report offering additional training to current employees to help overcome talent shortages.²⁹

The second argument is that weak wage growth for engineering and computer occupations are inconsistent with shortage claims, and that there may even be a surplus of workers with computer and engineering skills.³⁰ Yet, evidence does not support these contentions.³¹ Nominal and real earnings for these positions have increased significantly and much faster than wages in other occupations at least since 2000, and even further back using consistent definitions of occupations or skill.³² A more likely explanation for wage patterns in recent years is that the Great Recession temporarily eliminated most shortage problems and reduced wages, but the resumption of economic growth is once again tightening the labor market and regenerating a long-run shortage. It often takes a few years for wages to adjust to market conditions.³³

Related to the second argument on whether a shortage exists, a recent analysis of an employer survey in the manufacturing sector concluded that a shortage of production workers was a problem only for a small minority of manufacturing firms.³⁴ Only a small percentage of firms in manufacturing (24 percent) report having vacancies lasting three months or longer. Yet, that low percentage includes firms with no vacancies, which by definition could not have a recruitment problem. Considering only firms eligible to have a recruitment problem (i.e., those with vacancies), these data show that two-thirds of firms report having a long-term unfilled vacancy. At the same time, these data show that most production jobs do not require many years of formal education or technical training, though the report did not examine vacancies for engineers or other professional STEM occupations.

Hard-to-fill vacancies take longer to fill often because of a lack of qualified candidates. Yet, there are also a large number of low-skilled occupations that take a long time to fill because employers face high turnover and a constant need to recruit new employees.

Job vacancy information is widely used outside of academia. Indeed, U.S. government labor market statisticians released a guidebook suggesting that lengthy openings durations for given occupations can be interpreted, with some caution, as indicating a shortage.³⁵ Many U.S. state labor departments now conduct their own job vacancy surveys, and some ask employers to report the duration of their job vacancies and which occupations are hard to fill and why. The conclusions that follow from these data are that hard-to-fill vacancies take longer to fill often because of a lack of qualified candidates. Yet, there are also a large number of low-skilled occupations that take a long time to fill because employers face high turnover and a constant need to recruit new employees, and so the employers maintain vacancy advertisements for long periods.³⁶ Thus, the interpretation of ad duration can generally be taken to be synonymous with hiring difficulty except for positions with very high turnover and typically low wages.

The state-level data suggesting skill imbalances are consistent with national surveys. According to Manpower, roughly half of U.S. companies reported difficulty filling positions in 2011 and 2012 because of lack of skills, and this number increased from just 14 percent 2010.³⁷ In 2013, the share reporting difficulty fell somewhat to 39 percent, but that remains above the global average and above what many European countries report.³⁸ Moreover, companies report that many of the hardest-to-fill jobs require STEM skills—skilled trades and technicians, information technology (IT) staff, financial analysts, and engineers. Another survey of human resource managers found that roughly half of large U.S. companies report a STEM shortage, to which the managers attribute lower productivity and firm growth.³⁹

Small businesses also report increased hiring difficulty. Since 1986, the National Federation of Independent Business has surveyed small businesses on a monthly basis. Its February 2014 report shows the share of small businesses reporting at least one unfilled vacancy has returned to pre-recession levels, and the share reporting a lack of qualified job candidates increased from 25 percent in October 2009 to 40 percent in October 2013.⁴⁰

To analyze characteristics of skill shortage in 2013 generally and at the most detailed level, this study uses data purchased from Burning Glass, a Boston-based labor market information company founded in 1999. Burning Glass aggregates job postings from approximately 15,000 Internet job boards (as well as company websites) and codes and categorizes those openings and their requirements to facilitate interpretation and analysis.

This study relies on a subset of Burning Glass data that uses job postings advertised on company websites for companies located in metropolitan areas. In all, the database contains 4.7 million job postings from company websites, representing 25% of all Burning Glass ads for 2013 in metropolitan areas. Fifty-two thousand different companies are represented. Unlike the full Burning Glass database, this subset includes information on the date the job vacancy was first posted and when it was taken down from the company’s website. The difference in days between when a job was first posted and when it was taken down provides a measure of its duration in days, which will form the basis for key measures used in this analysis.

Advertisement duration is not precisely the same concept as vacancy duration. Employers advertise to attract applicants and obtain a pool of suitable candidates. Then, during a selection period, they sift through that pool and conduct interviews.⁴¹ It is not clear at what point in this process they remove advertisements. Moreover, an ad may be re-posted months later after an unsuccessful round of interviews. These measurement issues are addressed more directly in the appendix.

Summary findings on vacancy duration are reported only for the first quarter of 2013. The reason for limiting the analysis to this period is that the data are cut off in November 2013. As a result, vacancy duration artificially appears shorter for advertisements posted later in the year simply because the ads have not been allowed as much time to last. For the more sophisticated econometric analysis, which undergirds the summary findings, ads posted from January to August (3.3 million in total) are utilized and the results are adjusted by the quarter in which the ads were posted.

In addition to vacancy duration, Burning Glass provides many other useful pieces of information about job openings. The data include occupational codes, industry codes, company names, and metropolitan location. Many ads also include information on educational requirements (55 percent of total), experience requirements (52 percent of total), and, to a lesser extent, salary (7 percent of total).

Summary tables and statistics for the duration of STEM and non-STEM vacancies by educational requirements use the minimum educational requirements listed on the advertisement collected by Burning Glass.

The more sophisticated econometric analysis takes a slightly different approach. To avoid discarding observations with incomplete information, the baseline econometric analysis imputes educational requirements to detailed (six-digit) occupations using the most frequent educational level of workers, as reported by the Occupational Information Network (O*NET), a collection of detailed occupational-based surveys. Results are also shown using only the Burning Glass educational requirements. Here, the average of minimum and maximum educational requirements will be used. For example, an advertisement showing a range of possible educational levels from a bachelor’s to a Ph.D. would receive a higher education score than one showing only a bachelor’s.

Burning Glass also lists specific “skills” and certifications required for the job as mentioned in the advertisement. This information allows one to distinguish skill levels between jobs within the same occupation. For example, one computer occupation may require fluency with Microsoft Windows, while another requires fluency in Java script or another programming language. These skills can be quantified by taking advantage of salary information associated with each job and calculating a mean salary for each skill. For example, if a skill—like Microsoft Powerpoint—appears in 5,000 ads and 100 list salary information, the mean salary from those 100 ads is used to quantify the value of the skill.⁴²

Since Burning Glass provides data on vacancy duration starting only for 2013 (at least presently), alternative job vacancy data sources were used to analyze broad trends in hiring difficulty.

The Conference Board’s Help Wanted Online (HWOL) data series has comprehensive historic coverage of job openings advertised on websites and Internet job boards starting in 2005. It distinguishes between job openings that are newly posted in a calendar month and those that are re-posted (or unfilled) from the previous month. In this report, the unfilled rate is the share unfilled from the last month divided by the share of total postings last month; it is an indicator of hiring difficulty. Brookings purchased HWOL’s occupational-level openings data from 2005 to February 2012 for the 100 largest metropolitan areas. This report focuses on the period just before the recession (2006-Q4) until the latest available data. National summary-level openings data are available for free from the Conference Board’s website.