This decade of data drove the university’s new experiment in artificial intelligence.
Dr. Finn and her team built a neural network, a mathematical system that can learn skills from huge amounts of data. By tracking down patterns in thousands of cat photos, a neural network can learn to identify a cat. By analyzing hundreds of old phone calls, it can learn to recognize spoken words. Or by examining the way teaching assistants evaluate coding tests, it can learn to evaluate those tests itself.
The Stanford system spent hours analyzing examples from old midterms and learning from a decade of possibilities. Then it was ready to learn more. With just a handful of additional examples from the new exam offered this spring, it could quickly grasp the task at hand.
“It sees many kinds of problems,” said Mike Wu, another researcher who worked on the project. “Then it can adapt to problems that it has never seen before.”
This spring, the system returned 16,000 feedbacks, and students agreed to the feedback 97.9 percent of the time, according to a study by Stanford researchers. For comparison: In 96.7 percent of the cases, the students agreed to the feedback from the human teachers.
Mr Pham, an engineering student at Lund University in Sweden, was surprised that the technology worked so well. Although the automated tool failed to evaluate one of its programs (presumably because it had written a snippet of code unlike anything the AI had ever seen), it identified both specific errors in its code, including what was in the computer programming and math as. known to be a fence post error and suggestions for correcting it. “You seldom get such well-thought-out feedback,” says Pham.
The technology was effective because its role was so clearly defined. When taking the test, Mr. Pham was writing code with very specific goals, and there were only so many ways he and other students could go wrong.
But with the right data, neural networks can learn a number of tasks. This is the same basic technology that identifies faces in the photos you post on Facebook, the commands you type into your iPhone, and translates from one language to another on services like Skype and Google Translate. The Stanford team and other researchers hope that these techniques can automate education in many other ways.