AI Can Write Code. It Still Can't Think Like a Data Scientist.
Will AI Replace Data Scientists?
It's one of the most common questions prospective students, career changers, and working professionals are asking as artificial intelligence rapidly transforms the workplace. The short answer is no — but AI is changing what it means to be a successful data scientist.
In this first-person perspective, University of Virginia M.S. in Data Science (MSDS) Residential program student Ryan Dallas shares how an AI-assisted class project challenged his assumptions about large language models (LLMs), coding assistants, and the role of human expertise. His experience illustrates why critical thinking, machine learning knowledge, and sound engineering judgment remain essential skills in the age of AI — and why advanced education in data science is about learning to work with AI, not compete against it.
Key Takeaway: AI can accelerate coding and analysis, but it cannot replace the human judgment, critical thinking, and technical expertise that data scientists use to evaluate models, solve complex problems, and make informed decisions.
The AI Mistake That Changed My Perspective
As we were wrapping up Big Data Systems, the first class of the summer semester, I was exhausted. On the due date of a major project deadline, my team ran headlong into a hurdle we should have seen coming but to which we had become inured. The short version of the story is my project team sought to design a system in which four LLMs fed inputs into a fifth LLM along with a set of weights to produce a theoretically better overall summary.
In addition to being complex subject matter, this course also required us to use an AI coding engine for development rather than manually developing code ourselves for assignments. The regular homework assignments were pretty straightforward, so as usual, we fed our crafted project prompt to the AI and let it do its thing.
As we compiled the outputs from the AI, I was horrified to discover the graphs that were supposed to show the training performance curves for all five LLMs each only had a single data point. These models take a long time to train — even the smaller models with millions of parameters can take hours. How could this have happened? We needed to submit our slide deck by 5 p.m., and I wondered if we would have enough time to come back from this.
Why Human Judgment Still Matters
So, what actually happened? I forgot I am a human with sentience and opposable thumbs; and the task I had delegated to a computer algorithm is complex and difficult, even for a human. We like to think LLMs are capable of almost anything, but they are computer algorithms like any other: they do exactly what they are “told” to do, no more, no less.
I gave an LLM a hyper-detailed prompt with five paragraphs across 19 lines, and some details slipped through the cracks. I tried to offload my responsibility as a data practitioner, and a relatively simple LLM did the best it could with a task it does not actually understand. It captured data it “thought” was appropriate, but it does not know what that actually means — at what intervals it should capture data, how to recognize a problem with the data collected, etc.
Even after correcting the engine to capture and display data, it was re-generating the charts after each data point was collected — is that the most efficient way to do that task? All of the individual component models were over fit (i.e., performance improved up to a certain point in the training, and then performance tanked). Because it didn't know to monitor for performance optimization, it just ran all of the training and validation samples to collect data. The coding engine did exactly what I told it to do.
This is one personal example of a phenomenon academic research has demonstrated empirically: AI models are super-capable, but they cannot fully replace humans.
How UVA's MSDS Program Prepared Me
Humans are still needed in the loop to ensure systems and processes accelerated by AI are still achieving the desired end-result. This is why advanced education in data science is still important. At UVA, the MSDS program’s curriculum exposes us to a broad spectrum of data science skills with an emphasis on critical thinking.
Relevant to this anecdote, Machine Learning I taught us how to interpret model performance; Machine Learning III taught us how to read, understand, and develop code to implement complex AI models; and Data Engineering II taught us to be mindful of prompt and context engineering when working with AI coding engines and distributed computing tasks.
What Employers Are Looking For
Zooming out from academics, I can also describe how not being able to rely solely on AI appears in the job market. During my job hunt, I was never asked about AI-assisted coding — I was expected to understand and talk through the concepts and methodologies myself. In contrast, a classmate of mine tells me exposure to AI-assisted coding tools was a frequent feature of the job postings he encountered. So, we can infer exposure to these tools has a practical utility, but being able to use them may be insufficient to secure work in the data science field.
Final Thoughts: AI Won't Replace Curious Data Scientists
The story does have a happy ending: my team was able to re-run model training and capture performance metrics, and our presentation and project report both turned out well.
That said, some valuable lessons were learned, and I now have a story to back up what we already intuitively knew: AI cannot do everything, and advanced education in data science can keep you ahead of the curve when you start to feel like AI is coming for your job. Stay in school, kids.
Learn more about the full-time, in-person Residential MSDS at the University of Virginia. Request more information, connect with Admissions, or start your application today.
