Siri Answers 83% of Questions Correctly in Test, Beating Alexa But Trailing Google Assistant

In an annual test comparing Google Assistant, Siri, and Alexa on smartphones, Loup Ventures' Gene Munster found that ‌Siri‌ was able to correctly answer 83 percent of questions, beating Alexa but trailing behind Google Assistant.

Munster asked each digital assistant 800 questions during the test to compare how each one responded. Alexa answered 79.8 percent of questions correctly, while Google Assistant answered 92.9 percent of questions correctly.

aitestresults
Compared to last year, ‌Siri‌ has seen improvement. During the July 2018 test, ‌Siri‌ answered 79 percent of questions correctly compared to the 83 percent of questions answered right this time around. Alexa last year was at 61 percent while Google Assistant was at 86, so there have been digital voice assistant improvements across the board.

siriimprovementsovertime
This test covered smartphones specifically, comparing iPhones and Android devices. Munster says that smartphones were isolated from smart speakers because while underlying technology is similar, "use cases vary." ‌Siri‌ was tested on an iPhone running iOS 12.4, Google Assistant on a Pixel XL, and Alexa in the iOS app.

Questions were based on five categories and all assistants were asked the same 800 questions. Each question set was designed to "comprehensively test a digital assistant's ability and utility." Some of the sample questions across each of the categories:

  • Local - Where is the nearest coffee shop?
  • Commerce - Order me more paper towels.
  • Navigation - How do I get to Uptown on the bus?
  • Information - Who do the Twins play tonight?
  • Command - Remind me to call Jerome at 2 pm today

‌Siri‌ did best in the command, local, and navigation categories, faring less well in the information and commerce categories. ‌Siri‌ actually won out in the command category, but trailed behind Google Assistant in other categories.

questionsbycategory

Siri continues to prove more useful with phone-related functions like calling, texting, emailing, calendar, and music. Both Siri and Google Assistant, which are baked into the OS of the phone, far outperformed Alexa in the Command section.

Munster says that the continued rate of improvement "continues to surprise" based on the notable improvements that each voice assistant has demonstrated over the course of the last few years.

In the future, Loup Ventures expects to see further improvements from extending the feature sets of each voice assistant.

Top Rated Comments

JetLaw Avatar
60 months ago
I'm not sure about the methodology they used to reach these results, but my anecdotal experience is that Alexa does a MUCH better job than Siri on almost everything I've tried to use them for.
Score: 36 Votes (Like | Disagree)
airwalk331 Avatar
60 months ago
Does anyone else feel like this is crap? 99.8% of the time Siri doesn’t work for me (though my friends’ Alexa does). Period. It’s even worse when I try to use it and there are 2 or more apple devices in the vicinity.
Score: 34 Votes (Like | Disagree)
Blackstick Avatar
60 months ago
Interesting... in my un-scientific testing, Alexa consistently beats Siri at recognizing what I want, and more importantly, the gibberish requests that my 4 year old spouts "Alexa play more Frozen!" (kill me)
Score: 32 Votes (Like | Disagree)
MallardDuck Avatar
60 months ago
* Local - Where is the nearest coffee shop?
* Commerce - Order me more paper towels.
* Navigation - How do I get to Uptown on the bus?
* Information - Who do the Twins play tonight?
* Command - Remind me to call Jerome at 2 pm today

And the responses:

Here's what I found on the web for coffee soup
I can't sort paper trowels in order
Playing Uptown girl by billy joel
I don't know who will win the game tonight
I've updated your reminder to call home


Most of Siri's brain-deadness is in the voice recognition portion of the system.
Score: 32 Votes (Like | Disagree)
diego.caraballo Avatar
60 months ago
Siri better than Alexa? Fake news
Score: 27 Votes (Like | Disagree)
mediotanque Avatar
60 months ago
This is ********. Siri is the dumbest Ai at the moment. Even Cortana works better than Siri.
Score: 14 Votes (Like | Disagree)

Popular Stories

reset password request iphone

Warning: Apple Users Targeted in Phishing Attack Involving Rapid Password Reset Requests

Tuesday March 26, 2024 4:34 pm PDT by
Phishing attacks taking advantage of Apple's password reset feature have become increasingly common, according to a report from KrebsOnSecurity. Multiple Apple users have been targeted in an attack that bombards them with an endless stream of notifications or multi-factor authentication (MFA) messages in an attempt to cause panic so they'll respond favorably to social engineering. An...
maxresdefault

Apple to Launch New iPad Pro and iPad Air Models in May

Thursday March 28, 2024 11:07 am PDT by
Apple will introduce new iPad Pro and iPad Air models in early May, according to Bloomberg's Mark Gurman. Gurman previously suggested the new iPads would come out in March, and then April, but the timeline has been pushed back once again. Subscribe to the MacRumors YouTube channel for more videos. Apple is working on updates to both the iPad Pro and iPad Air models. The iPad Pro models will...
Generic iOS 18 Feature Purple

iOS 18: What to Expect From 'Biggest' Update in iPhone's History

Wednesday March 27, 2024 11:10 am PDT by
At least some Apple software engineers continue to believe that iOS 18 will be the "biggest" update in the iPhone's history, according to Bloomberg's Mark Gurman. Below, we recap rumored features and changes for the iPhone. "The iOS 18 update is expected to be the most ambitious overhaul of the iPhone's software in its history, according to people working on the upgrade," wrote Gurman, in a r...
maxresdefault

Apple Announces WWDC 2024 Event for June 10 to 14

Tuesday March 26, 2024 10:02 am PDT by
Apple today announced that its 35th annual Worldwide Developers Conference is set to take place from Monday, June 10 to Friday, June 14. As with WWDC events since 2020, WWDC 2024 will be an online event that is open to all developers at no cost. Subscribe to the MacRumors YouTube channel for more videos. WWDC 2024 will include online sessions and labs so that developers can learn about new...
apple maps 3d feature

Apple Maps May Gain Custom Routes With iOS 18

Tuesday March 26, 2024 3:10 pm PDT by
Apple may be planning to add support for "custom routes" in Apple Maps in iOS 18, according to code reviewed by MacRumors. Apple Maps does not currently offer a way to input self-selected routes, with Maps users limited to Apple's pre-selected options, but that may change in iOS 18. Apple has pushed an iOS 18 file to its maps backend labeled "CustomRouteCreation." While not much is revealed...
General iOS 17 Feature Orange Purple

Apple Releases Revised Versions of iOS 17.4.1 and iPadOS 17.4.1 With Updated Build Number

Wednesday March 27, 2024 5:59 am PDT by
Apple on late Tuesday released revised versions of iOS 17.4.1 and iPadOS 17.4.1 with an updated build number of 21E237, according to MacRumors contributor Aaron Perris. The updates previously had a build number of 21E236. The revised updates are available for all iPhone and iPad models that are compatible with iOS 17 and iPadOS 17, but they can only be installed via the Finder app on macOS...
applephilschiller

Apple's Phil Schiller Works 80 Hours a Week Overseeing App Store

Wednesday March 27, 2024 2:03 pm PDT by
With the App Store and app ecosystem undergoing major changes in the European Union, The Wall Street Journal today shared a profile on App Store chief Phil Schiller, who is responsible for the App Store. Though Schiller transitioned from marketing chief to "Apple Fellow" in 2020 to take a step back from Apple and spend more time on personal projects and friends, he is reportedly working...