How students teamed up to decode 2,000-year-old texts using AI

What began as a challenge to decipher texts buried for centuries under the ashes of Mount Vesuvius quickly turned into a historical breakthrough using AI and GitHub tools. Three students—Youssef Nader, Luke Farritor, and Julian Schilliger—worked together across time zones and borders to unlock the secrets of 2,000-year-old scrolls, ultimately winning the Vesuvius Challenge and earning $700,000 in prizes. Their work showcases the powerful intersection of AI, open-source collaboration, and the drive to solve mysteries that have puzzled scientists for centuries.

About the challenge

In March of 2023, a group of leading technologists created the Vesuvius Challenge, a competition to decipher the Herculaneum Papyri which were buried after the eruption of Mount Vesuvius 2,000 years ago. Due to carbonization, these scrolls cannot be opened without falling to pieces. So how do we access these ancient texts?

In an interview with NPR’s Simon Scott, Brent Seals, a computer science professor at the University of Kentucky, spoke about the virtual unwrapping of scrolls and evolution of technology in this realm. This technology has been around for decades, but there was a breakthrough in 2015 which led to the reading of a scroll in the Dead Sea Scrolls collection. The scientists used tomography and X-rays and found success with the Dead Sea Scrolls, but were unsuccessful when it came to Herculaneum texts. “Not only were those scrolls difficult to apply virtual unwrapping to, but the ink from the ancient world did not readily show up in the scans that we made, and we needed an AI-based approach to be able to see that ink,” Seals said. Once the scientists captured these scans of the Herculaneum papyri, technologists from around the world set out to analyze them.

See Brent Seals speak about the process of using virtual unwrapping to recover some of the text on the Herculaneum and how this led to the creation of the Vesuvius Challenge.

Meet the Students

With hundreds of engineers around the world working on this, it’s no surprise that the winning group met and worked together completely online. Youssef Nader, Luke Farritor, and Julian Schilliger teamed up through the challenge’s Discord server and chatted about the other projects they had worked on under the Vesuvius Challenge. Luke won the First Letters Prize and reached out to Youssef upon his being named runner-up. While we weren’t able to sit down with Luke, he described the process of realizing he had found the first letter in this interview with his school, University of Nebraska-Lincoln.

Click to hear Luke explain his experience and the moment he realized that he had discovered the first letters in the Vesuvius Challenge.

At this point in the competition, Luke was working on segmentation and Youssef had created AI detection models. Shortly thereafter, the two teamed up with Julian who made a breakthrough automating the segmentation working extensively with GitHub Copilot.

“There are so many benefits to using GitHub as a student, from a free GitHub Pro account to the use of GitHub Copilot in Visual Studio Code,” Julian said. “For the Vesuvius challenge, I had to write pipeline most of the time. I’d write one piece of code that I needed to use to achieve the next goal and if I knew what I wanted to write, I could use the auto completion tool to help me write faster. It was a huge time-saver!”

We also sat down with Youssef who shared insight into his experience.

“We were working in different time zones, so sometimes, we had to intentionally overlap with each other which led to some very late nights and early mornings,” he said. “All of it was worth it because not only did the team win the challenge, but they were able to create special memories together. I remember a night or so before the submission, Julian sent me an exciting message that around 1600 cm^2 of the scrolls were being segmented by his software. We spent the better part of the night hunting for the title of the scroll.”

Youssef recalled the morning that everything started coming together. “I had almost given up hope. I had tried so many different things. In the morning, I was running one last experiment and to my surprise, it worked. There were some parts that made me feel connected to this 2000 year old scroll in a way. It required tracing the writing of this ancient writer on the scroll and finding smart ways of figuring out what letter it would be based on ink deposits.”

These three students realized the dream of all papyrologists from 1754 AD onward. Papyrologist Gianluca Del Mastro recalled meeting Luke in Kentucky for the First Letter Prize. “I saw this young student in front of me. It amazed me as I was expecting someone older. It made me realize we have entered a new world of information technology in which it’s possible to make new discoveries even if you are very young.”

The team invites everyone to take a look at their winning team’s code on GitHub. Having used GitHub for many years and utilizing the tools in the Student Developer Pack, Youssef and Julian felt it was the perfect place to share their team’s findings. “This challenge from the very beginning was to foster collaboration even in the face of competition. Housing our code on GitHub was the only thing that made sense so the community can continue to build and to have easy access to collaborate and push progress forward,” said Youssef.

Check out Youssef’s blog post for a more technical overview.

After the challenge

Dr. Del Mastro had the chance to meet two of the winning teammates after his team sponsored their flight out to Naples, Italy to see the scrolls in person. It was the first time that Youssef and Julian were able to meet in person. “It was surreal to meet in person after spending so much time collaborating over the internet,” Youssef shared. While in Naples, the two went to a conference where they were able to meet some of the professors who were behind the evaluation of their work. Youssef happily reports that he, Julian, and Luke are still in touch and hope they can all work on a project together in the future.

Left to Right: Aya Elzoheiry, Youssef Nader, Julian Schilliger, Marzia D’Angelo, Claudio Vergara, Fabrizio Diozzi, Alessia Lavorante.
Front: Rossella Villa

The experience was life changing in so many ways. Not only did the three winners help uncover part of the past, but one of them found his future. Julian shared that through the challenge, he’s met so many wonderful teachers and mentors who opened his eyes to all the work to be done at the intersection of code and history. Since completing the challenge, he accepted a full-time role working at the Vesuvius Project where he spends his time decoding the scrolls and learning new information about the ancient past.

Julian said, “Youssef, Luke, and I won this grand prize, but this is only a small piece in the ongoing efforts to decode the scrolls. Lots of people have worked on this before 2023 and there is plenty left to be done in 2024.” If you’re interested in getting involved, check out their Discord.

The post How students teamed up to decode 2,000-year-old texts using AI appeared first on The GitHub Blog.

Source: Read More