Collaboration with Master’s Programs

Differences Between Master’s and Undergraduate Education

In the previous chapter, we discussed what organizations should and should not expect when working with undergraduate students in data science. A general summarization of undergraduate education might be that undergraduate students “learn how to learn” through a combination of general education courses, major required courses and major elective courses. Consequently, it is often the internship or capstone course - the “high-impact practices” - that uniquely enable undergraduate students to develop applied skills during their academic programs via real- world projects.

Master's-level education might be summarized as "learning how to do what people do".

As the number and proportion of people in the United States with a bachelor’s degree has increased over time, so too has the number and proportion of people with a master’s degree. In 2019, 33-6% of the U.S. population over the age of 25 had earned a bachelor’s degree¹. This was up from 28% of the population in 2010 and up from just 4.6% in 1940. By 2019, approximately 10% of the U.S. population had earned a master’s degree, up from just under 7% in 2010 and less than 5%

Figure 4.1 Master's degrees conferred by U.S. colleges and universities.

in 2000. The number of master’s degrees conferred by U.S. academic institutions has almost doubled since 2000 and almost tripled since 1980 (see Figure 4.1)².

While the reasons for the increasing number of people obtaining master’s degrees are varied, increased salary potential is unquestionably part of the trend. People who have master’s degrees are generally paid more money than those who do not. Tliis is true for graduates of analytics and data science programs - but potentially not at the level that you might expect. More on that later in the chapter.

The Rise of Master’s Programs in Analytics and Data Science

Master’s-level programs in analytics and data science formally entered the educational marketplace in 2007 In 2016, there were roughly 60 analytical master’s programs, and by 2020 that number had surpassed 300 programs. See Figure 4.2. Rarely in academia has such a rapid evolution occurred - of all the things that universities are known for, quickly and efficiently responding to the needs of the marketplace has historically not been one of them. In Figure 4.2, there is a differentiation between ’‘business analytics”, “data analytics” and “data science”. While some “business analytics” programs are simply repackaged MBA programs, many of these programs were purposefully architected from scratch, such as the nation’s first formal analytics program at North Carolina State University through the Institute for Advanced Analytics.

Reiterating the definitions that we presented in from Chapter 1, generally, “data scientists” are considered to be a more technical subset of analytics professionals;

Figure 4.2 Growth of analytics and data science master's degree programs in the United States.

where analytics professionals apply a wide range of quantitative methods to derive insights and prescribe actions, data scientists will have deeper computer science skills needed to acquire, clean or transform, and model unstructured or continuously streaming data, regardless of its format, size, or source³. Both analytics and data science professionals are likely to have a master’s degree, with data scientists equally as likely to have gone on to earn a PhD¹. See Figure 4.3- The PhD in data science and how analytical organizations should consider engaging these doctoral programs is addressed in Chapter 5-

With this unprecedented growth in master’s-level programs, it would be reasonable to question if supply is in danger of outpacing demand. Actually, demand for analytical talent and supply of analytical talent have been closely matched⁵ - this is good news because unemployed alumni are not a desired outcome for any university program. See Figure 4.4. This close correlation begs the question if supply is actually driving demand. Given the disruptive nature of technology, the typically slower adoption of larger organizations to embed it in their workflows and the need to demonstrate value-driven use cases, this may make sense. As companies hire analytics and data science talent, the work outputs of that talent have great potential to create improved operational efficiencies. Consequently, more resources are devoted (i.e., data scientists hire more data scientists) and the supply and demand

Figure 4.3 Distribution of data scientists and analytics professionals by education (2020).

become closely correlated. Regardless of the reason, Figure 4.4 ultimately demonstrates that the need for talent is still growing year over year - with the highest volume happening at the master’s-level.

Salaries are also keeping pace. According to a longitudinal salary study of analytics and data science professionals, entry-level employees with an undergraduate

Figure 4.4 U.S. data scientists supply and vacancies - entry-level, $80K+ Salary.

Table 4.1 Distribution of Base Salaries of Analytics and Data Science Professionals in the United States by Job Level and by Education

earning a MS in Analytics, and an increase of almost $15,000 when pursuing a MS in Data Science. Lower level managers - those with small teams and well-defined project scope — saw an increase of almost $6,000 after earning a MS in Analytics, and, again, about a $15,000 increase when pursuing an MS in Data Science. Again, in this context, data science professionals are generally more technical than are analytics professionals. See Table 4.1.

This proliferation of master’s-trained individuals developing varying levels of expertise in analytics, data science, data engineering, predictive modeling, and machine learning has led to confusion in the hiring market for such talent and likely frustration for hiring managers when looking for talent — particularly talent with such strong salary demands.

A study completed in 2018 examined 603,424 job postings for “Analytics”, “Big Data”, “Business Analyst”, and “Data Scientist” for the three-year period from January 2015 to December 2017⁶. While there were expected differences — with technical skills like programming and scripting languages more likely to be required for “Data Scientists” versus “Business Analysts” — there were commonalities - many positions required “communication and interpersonal” skills. See Table 4.2.

Table 4.2 summarizes the analytical skills that are in demand, but how does this compare to the “supply” of master’s-level talent that is generated by universities? The same study examined what was actually being taught across 64 master’s- level programs in analytics and data science in 2015- See Figure 4.5-

It is important to note that universities lean heavily into the applied project course to help master's-level students develop communications and interpersonal skills that are most in-demand across all sectors of the market and for all types of analytical positions.

Illustrations have been created especially for this book by Charles Larson.

Table 4.2 Percentage of Documents (Job Ads) that Contain at Least One Term from a Custom Topic by Job-Title Keyword (Analytics, Business Analyst, and Data Scientist)

Table 4.2 Percentage of Documents (Job Ads) that Contain at Least One Term from a Custom Topic by Job-Title Keyword (Analytics, Business Analyst, and Data Scientist) (Cont.)

Data source: Burning Class Technologies (2018).

The concept of “mapping” skills (e.g., communications) into particular courses (e.g., the applied project course) are common to all universities. These “curriculum maps” are helpful for faculty and administration to ensure limited overlap between programs, but also to aid accrediting bodies like the Association to Advance Collegiate Schools of Business (A ACSB) in their evaluation of programs.

Figure 4.5 Average distribution of credit hours for master's-level data science and analytics programs by topic.

Illustrations have been created especially for this book by Charles Larson.

We have presented the curriculum map for the Health Data Science degree program at the University of New Hampshire. See Figure 4.6. In this curriculum map, the columns are the courses and the rows are the skills and competencies. Note that the map starts with more quantitative skills and moves toward interpretive skills, followed by application skills, and finally to communication skills. Thus, by looking at the first four columns and then letting your eyes blur to the right (blurring is

Figure 4.6 Curriculum map for MS in health data science program at the University of New Hampshire.

easy to do sometimes when looking at these maps), but excluding the electives, you can see that skills taught tend to cluster from upper right to lower left as courses progress. The curriculum culminates with the practicum experience (applied project course), which is the most immersive and applied course in the sequence. In that course, students are putting their skills into practice and within the realistic confines of existing structures (both data and organizational) and simulating impact across the project metrics. The electives (not shown) are a mix of the two, typically providing a more rounded blend of skills and application. This is one example. Different programs will emphasize different courses, skills, and competencies.

Do not be shy about asking to see a curriculum map - it is the best way to see holistically what is being taught in a data science program.

All told, the primary way that companies engage with master’s-level programs — and the frequent starting point for many university engagements - is through applied graduate project courses. To illustrate how these project courses work in more detail, we present you with two case studies shared by two university analytics programs.

CASE STUDY 1 AN ENHANCED RELATIONSHIP OF MUTUAL BENEFIT

A large energy company based in the Southeast has frequently hired students from the Masters of Science in Applied Statistics and Analytics Program (MSASA) at Kennesaw State University (KSU). The hiring managers agreed that the topics and skills taught in the classroom were particularly well-aligned with the analytical needs of their team. However, the relationship was relatively informal; the energy company would sponsor student events, hackathons, and analytics project competitions, and the faculty would forward job postings to the students when positions became available. In 2015, the energy company approached the Chair of the Department of Statistics and Analytical Sciences about enhancing the partnership through a new format that would allow the analytics team to interact more frequently and engage in regular, ongoing projects with the master's students. The goal was to not only support the program in a more formalized way, but also to create a pipeline of "known" talent for the energy company to deepen the bench of their analytics team.

The analytics project course, sponsored by the energy company, was launched in Spring 2016. The first semester the course was offered six second year MSASA students enrolled. In the context of the course, students worked either individually or on a team to classify customers and the likelihood of purchasing energy-efficient products and services.

Over the 15-week semester, the students met regularly with the Information Technology Team and Business Managers to clarify questions, verify results, and provide project updates. Meetings occurred approximately every other week online and in person. At the end of the semester, the students were invited to travel to the company's headquarters - wearing their best suits - to present their final models and recommendations to the senior leadership team.

From that class, one student was offered an internship which evolved into a full-time position.

Almost every semester since, the project course has resulted in at least one student receiving an offer for a graduate internship or full-time position.

Through the analytics project course, the MSASA students had the opportunity to work on a wide range of business problems. Course problem topics have included:

• 1. Predicting potential customers for key products and services (different products and services in different semesters)
• 2. Estimation of a predicted power outage duration
• 3. Product bundling analysis
• 4. Market sizing
• 5. Solar plant production
• 6. Wind energy impacts
• 7. Event (outage, de-rates, curtailments) validation
• 8. Gas usage pattern recognition and abnormality detection

In the context of the project course, students have used a combination of modeling approaches including logistic regression, generalized linear models, time series analysis, recommendation system, association analysis, market basket analysis, principal components analysis, к-means clustering, and exploratory data analysis (EDA). Student evaluations of the course have reflected five consistent themes of learning:

• The project course helps students to develop confidence with translating a business problem into a quantifiable analytics problem with measurable outcomes.

Students realize that effective communication is foundational to the success of an analytical project. Students reported learning that what "clients" actually need may not necessarily be consistent with what they say they need.

• • The ability to communicate very technical results to a non-technical audience is a necessary skill for someone who is highly computational.
• • Learning to work effectively in teams is critical. While the students in the course were all enrolled in the MSASA program, their undergraduate majors included computational disciplines like computer science and mathematics, but also non-computational disciplines like psychology and sociology. Students expressed appreciation for learning how to work in teams with diversity of thought, approach, skills, and problem-solving approaches.
• • Students developed an appreciation for flexibility and tolerance for changes in scope. More than once, teams received a last-minute file two weeks before the final presentation. Inevitably, the server would crash as they were trying to generate their final deliverables.

In addition to establishing a rich pipeline of "known" talent, the energy company frequently realized innovative ideas and applicable solutions from the class - although this was not their primary goal. Several of the models developed as part of class projects - such as the model developed to improve the estimation of power outage duration - have been implemented into the company's application to improve customer experience. Compared to a one-semester internship position, ongoing sponsorship of the course was a much lower cost engagement and generated more innovative solutions due to competition-orientation amongst multiple project teams. In addition, employees of the energy company have gone back and formally enrolled in the MSASA program.

CASE STUDY 2 GETTING TO THE "RIGHT" QUESTION

At Oklahoma State University (OSU), all students enrolled in the Masters of Business Analytics and Data Science program (MSBAnDS) must enroll in BAN 5560: Research and Communications. MSBAnDS is a cohort-based curriculum and as such enrolls 35-40 students every fall semester. While many enrolled students have several years of working experience, some are entering just after their undergraduate program. While the students have various backgrounds and skills, most lack project management or leadership experience. Every year, between three and six companies work with the BAnDS program to provide students an opportunity to gain this experience with real-world business data while being supervised by experienced faculty. Students are placed into interdisciplinary teams of 4-5 students at the beginning of their first semester and remain in the same company- based groups for the first academic year.

Teams are designed to ensure balance across several criteria:

• 1. Programming skills
• 2. Statistical skills
• 3. Industrial/domain focus
• 4. Project needs

The students' performance in the course is assessed across six deliverables:

Deliverable 1 - Project Proposal 10%

Deliverable 2 - Draft Report/Presentation 13%

Deliverable 3 - Project Management Process 12%

Deliverable 4 - Practice Presentations (3 practice sessions) 21%

Deliverable 5 - Final Report 21%

Deliverable 6 - Final Presentation to Client 23%

Team assessments are conducted every 1.5 to 2 months as part of the project management process; all students rate their teammates on communications, planning, team goals and shared experiences among others. Students receive anonymized comments and feedback from their teammates. These evaluations are used to conduct course correction meetings with faculty throughout the project cycle.

In 2017, one of the analytics projects was sponsored by a state licensing and regulation agency in Texas. The agency had recently undergone a data warehousing project and was interested in learning where their customers overlapped across different licensing groups. At the time of this project, faculty were outlining the outputs of the project for the students but not the business problem that it was trying to solve. For example, students were given the following information at the beginning of the fall semester:

The agency would like to have the following types of analysis run on the data. The first two are considered the critical components. The Marketing analysis would be nice to have but are of lesser importance at this time.

• • Demographics and location
• • Compare and contrast participants in each program
• • Churn analysis
• • Recency; Frequency; Monetary Value (RFM) analysis

While the students ultimately gave the company what they outlined, the project suffered from a woeful lack of creativity; the project was too prescriptive.

This was evident in presentation quality and report quality. The final projects were substantively less impressive than projects from previous years. Did the students meet the criteria? Yes, but not with the passion that the program likes to instill in future data scientists. After a debrief with the program administrators, we learned the following points that we would like to share with organizations considering similar project courses with master's programs:

• 1. Thinking about the end goal in mind is fine but consider starting students off with the actual problem the company is trying to solve. While these students (and faculty) eventually backward engineered the problem, it was a long and painful process to ultimately determine that what they project sponsor really wanted was to improve customer experience while improving revenue.
• 2. Asking the right questions as a consultant is never easy. Analysts and research-types often get stuck on the "what" or "how" and are less focused on the "why". We teach our students to think about the business problem first and all items must lead back to the business problem.
• 3. Please do not prescribe the outputs; let the students figure out what will work to meet the criteria. For example, when the company wanted to know "How often are the same people participating in multiple licensing areas", do not then specify that this is churn analysis. This is like giving a student the test question with the answer embedded.
• 4. Working with company points of contacts are just as important for faculty as it is for students. We have worked with this person on several other projects, presentations, and conferences and fostering relationships over time is important not only for analytics students but also for the program. Maintaining long-term relationships is an important investment - for faculty, students, the program, and the sponsoring organization.