Data Scientists vs. Domain Experts

Ten years later, there is speculation that the role of data scientist may be short-lived.

In a 2012 Harvard Business Review article, Thomas Davenport and DJ Pati stated: Data Scientist: The Sexiest Job of the 21st Century. In 2022, the authors re-posted with title: Is the data scientist still the sexiest job of the 21st century? They haven’t taken a position on it, but they makes a helpful comment:

The role was relatively new at the time. Yet, as more and more companies tried to make sense of Big Data, they realized they needed people who could combine skills in programming, analysis, and experimentation. At the time, this demand was mostly limited to the San Francisco Bay Area and a few other coastal cities. Startups and tech companies in these fields seemed to want all the data scientists they could hire. We felt that the need would grow as larger companies embrace business analytics and new forms and volumes of data.

The prototypical data scientist in 2012 was recruited among PhDs in fields like physics, statistics or mathematics or other quantification disciplines with experience in experimentation in their respective research fields. As the saying goes, their skills were necessary but not sufficient. A large data science project, from start to finish, involves too many steps and too many skill areas for one person. Often responsible for the entire effort, the data scientist was expected to provide expertise in the following areas:

  • Statistical and computer analysis
  • machine learning
  • deep learning
  • Processing large data sets
  • Data visualization
  • Data Conflict
  • Math
  • Programming
  • Statistics
  • Exploratory data analysis
  • Visualization of results for presentation including BI, Powerpoint and in-person presentation

Of The Future of Data Scientists: Will the Next Decade See Complete Extinction?

What does data science involve?

To begin with, data science is the art of extracting relevant information, major insights, and knowledge from a given set of data. Therefore, the output of any data science project is mostly a slide chart or PPT that basically wraps up the whole scenario for business leaders to make decisions, or for a group of technical and product experts to conclude how to work on a site.

The key steps that drive the workflow of any data science project are mainly:

  • Collect lots of data (for verified sources)
  • Analyze the data and iterate it several times to develop a good understanding of the data
  • Suggest hypotheses or actions that include periodic analysis and updating of data.
  • Organizations are now looking to build a healthy data science framework by hiring large, consistent and deep teams, benchmarked by industry guidelines in place.

Due to the likelihood that a data scientist will be proficient in many of these requirements, according to Noah’s gift:

Many data science teams have results not delivered which can be measured in return on investment by managers. For complex data engineering tasks, you need five data engineers for one data scientist.

The pace of automation software is quite dramatic and will affect the nature of data science work, including machine learning. All Major cloud seller invested heavily in some type of AutoML initiative. A data scientist is no longer characterized by coding skills, which is confirmed by the growing importance of no-code and auto ML arrangements like DataRobot, Amazon, Dataiku, Google Cloud, Databricks, H20, Rapid Miner and Alteryx. Does this mean data science is a bad career choice? Without a doubt, the position is evolving. Data scientists should seek to improve their skills in areas that are not automatable:

  • communication skills
  • Expertise applied to the domain
  • Creation of revenue and business value

The tight recruiting process has shifted to critical thinkers, analysts and problem solvers who understand all the nuances of the business, its area of ​​expertise and its various collaborators. Therefore, mere know-how of multiple software packages or the ability to churn out a few lines of code wouldn’t do the job.

The only sure thing is change, and changes are happening in data science. As the job title of data scientist will recede, the job of data scientist will be distributed to machine learning engineers, data engineers, AI wranglers, AI communicators, business ethics officers , AI Product Managers, AI Production Administrators, and AI Architects. What today’s data scientists should be doing to stay relevant is adopting soft skills. They need to move away from tasks that can be easily automated – feature engineering, exploratory data analysis, trivial modeling – and towards tasks that defy automation and produce an AI system that has a measurable business impact with verifiable business metrics and improved revenue.

Companies that want to stay ahead of the game can embrace the pragmatism and automation of machine learning tasks to gain a clear strategic advantage.

My point of view – what can we conclude about the fate of data scientists?

Obviously, data science can never go away. However, tenured positions and data scientist roles will surely notice a dynamic shift. In about ten years, industry domain specialists, business analysts and data savvy field experts become experts in data science by training in AI and ML.

These specialists will be able to imbue the analysis with their deep industrial knowledge, whether they know how to code or not. Their titles will reflect their aptitude instead of the methods they perform.

I don’t believe data scientist was the sexiest job of the 21st century in 2012, and it isn’t now. In fact, we are already seeing a drop in job ads for data scientists and a corresponding increase in ads for breaking down the original design of data science.