When Randy Bilyeu disappeared, he was hunting for the Fenn Treasure, a chest allegedly filled with gold, precious stones, and jewelry, supposedly hidden in the Rocky Mountains north of Santa Fe, New Mexico.
In 2010, millionaire art dealer (and former Vietnam fighter pilot) 79-year-old Forrest Fenn filled a bronze chest with rare metals, jewels, and artifacts, and then hid it in the mountains. Later that year, he published his autobiography, The Thrill of the Chase, which included a 24-line poem that he says contains the clues necessary to track down the treasure chest. Since then, he’s become something of a global celebrity; in 2013, he appeared on NBC’s Today Show to issue some new clues about the place where the chest had been hidden. Bilyeu happened to catch the episode on TV and became obsessed with finding the Fenn treasure—against all odds and his friends and family’s better judgement.
Soon after New Years Eve 2016, Bilyeu arrived in Santa Fe, a city that hosts “Fennboree” camping weekends a every year, and where $100 will buy a map signed personally by Forrest Fenn. On Jan. 3, Bilyeu checked out of his motel and bought an inflatable raft. On Jan. 5, he went out into the mountains, leaving a message with a friend that he’d be back tomorrow. But he didn’t, and after a week and a half of unsuccessfully trying to contact Bilyeu by phone, his ex-wife Linda called the cops.
The next day, local police found his raft and starving terrier named Leo. But there was no trace of Bilyeu. A few weeks worth of search-and-rescue missions also came to nothing. Linda Bilyeu didn’t give up though and organized a group of volunteers to keep the search up. One of them was Jerry Snyder, a retired Drug Enforcement Administration special agent and founder of the Find Me Group, a nonprofit organization based in Chandler, Arizona, offering professional help in finding missing people.
“Since 2002 we’ve had over 400 such cases. Find Me relies heavily on volunteers, so I’ve been thinking about the way to use their time as efficiently as possible,” says Snyder. “I came up with the idea of an artificially intelligent system that could be fed with all the evidence we’ve gathered on a particular case and which would give us some approximated location of the person in question on that basis. Sadly, I was no computer genius.”
So Snyder contacted the nearby Arizona State University and eventually got in touch with Paulo Shakarian, an assistant professor and head of the Cyber-Socio Intelligent Systems Laboratory there. Shakarian, a West Point graduate, specializes in a technique called “geospatial abduction.”
Essentially, it’s an artificial intelligence system that figures out the current location of someone (or thing) using a data set of known previous locations. For example, geospatial abduction can pinpoint the location of a bear’s cave using the coordinates of animal’s droppings, or a serial killer’s address using the coordinates of known killings. Serial killers usually attack within six miles from their home, and bears will stay within the same distance of their cave when they go out on their daily hunts or bathroom trips. Shakarian has designed algorithms that take information like that into account, ingest data points, and, after ruling out obviously impossible locations like lakes, rivers and so on, come up with the most feasible solution to current whereabouts. As with most algorithms of this sort, the more data—the more killings or droppings—the more likely for the solution to be correct.
The technology proved its worth in counterinsurgency operations in Afghanistan. An AI system that Shakarian co-designed with scientists from the University of Maryland (called called SCARE-S2, for Spatio-Cultural Abductive Reasoning Engine System 2), was able to locate insurgent leaders and their major supply depots.
Now, Shakarian began adapting SCARE-S2 into something he was calling MIST, for Missing Person Intelligence Synthesis Toolkit. The idea, he says, “was to pull the same trick off with finding missing people.”
Input coordinates are pretty clear when it comes to tracking down bears, serial killers, and even insurgents. But in the Bilyeu case, there were no certain data points. Instead, Shakarian and Snyder brought in around 20 experts to make educated guesses as to Bilyeu’s current location. Then they fed those coordinates into the algorithm.
But there was one other problem: in the world of mysterious disappearances and hidden treasures, “experts” are often eccentric. “We use the expertise of retired law enforcement and skilled search and rescue professionals, even the talents of hyper-intuitive individuals, trying not to miss anything that could be a valuable contribution,” says Snyder. By “hyper-intuitive individuals” he means psychics, mediums, and one man who describes himself as a “certified forensic astrologist.” Snyder insisted that MIST take their input into account, so Shakarian and his computer scientist students had to find a reasonable way of dealing with this. Their idea was beautifully simple.
Modern AI systems are often trained rather than programmed. In other words, they learn from examples and not rules. MIST was no different: they took 24 closed missing persons cases from Find Me’s files, and fed them to their AI software. Then, the AI “compared coordinates provided by each expert with coordinates of the location where a missing person was actually found,” says Shakarian. “On that basis MIST ascribed a weight to each expert’s input.”
Thus, when the AI was deployed on the Bilyeu case, it already knew its way around the Snyder’s team—whose guesses were likely to be pretty accurate and whose needed to be taken with a grain (or two) of salt, and how best to weigh outliers against team agreement on specific data points. Using that information, the AI took the experts’ estimates, and spit out what it believed to be Bilyeu’s whereabouts. Towards the end of July 2016, they passed MIST’s coordinate guesses on to the police in Santa Fe.
A few days later, an engineer working for the US Army Corps of Engineers stumbled on Bilyeu’s remains, on bank of the Rio Grande river, under a thick tanglement of branches and covered with leaves. The location matched MIST predictions—and in fact, the area had been searched more than once before, but because of all vegetation, the body had been missed to that point. Sadly, no AI in the world can pinpoint the exact heap of leaves that in such case needs to be turned. Nevertheless, the finding was encouraging: “At the end of the day MIST has passed its real world test with flying colors,” Snyder says.
Now, the team wants to tweak MIST so that it could work in homicide and human trafficking cases as well. Snyder has already approached several law enforcement agencies in the US and INTERPOL in Europe for access to their databases. But even if those projects don’t get off the ground, MIST could have a huge impact: 4,000 people go missing in the US every day. According to the research published by Shakarian and Snyder, a team using MIST can find a missing person average two days faster than a team without it. “The sooner we find them, the more likely we find them unharmed and alive,” says Snyder.