I requested Apple’s new Siri AI and ChatGPT the identical 10 questions – this is how they responded

Apple now has a beta model of its generative AI-infused model of Siri, often known as Siri AI. The launch has been a lot delayed and whereas it’s nonetheless very a lot a work-in-progress, it’s a constructive step.

However the query is how does it examine to different AI instruments? I’ve already pitched it towards Google Gemini, so now I’m going to do the identical with OpenAI’s ChatGPT. This must be a bit totally different to my earlier take a look at as a result of ChatGPT runs on utterly totally different fashions – in addition to Apple’s personal tech, Siri AI makes use of a customized Google Gemini mannequin for extra advanced reasoning – for which it’s reported to be paying Google round $1bn a 12 months. However the outcomes must be comparable.

Firstly, I’ll define the questions requested, after which I’ve pasted within the responses from ChatGPT and Apple Siri earlier than reaching some conclusions on the backside of this text.

Notice that I’ve pasted their responses beneath precisely how they have been offered – together with quite odd use of daring textual content and capital letters in some circumstances.

The questions

These questions that are designed “stress-test an AI’s reasoning, reminiscence, honesty, and creativity.” I then took the questions and altered them a bit for logic and so forth.

1. Transport

You’re in New York Metropolis and wish to journey from Occasions Sq. to Coney Island utilizing public transport. What choices can be found, how lengthy does the journey take, and what would you could pay?

2. Logic

A farmer has 17 sheep. All however 9 die. What number of are left?

3. Spot the idea

A bat and a ball price $1.10 in complete. The bat prices $1 greater than the ball. How a lot does the ball price?

4. Ambiguity

I’m flying from London to Sydney tomorrow. What’s the climate going to be like?

5. Hallucination test

Who received the 2034 FIFA World Cup?

6. Math reasoning

If 5 machines take 5 minutes to make 5 widgets, how lengthy would 100 machines take to make 100 widgets?

7. Ethics

Is it ever acceptable to lie? Give an instance and clarify your reasoning.

8. Creativity

Write a four-line poem about Wi-Fi within the type of a medieval bard.

9. Self-awareness

What’s one thing you don’t know proper now, and the way would you discover out?

10. Multi-step reasoning

A practice leaves London at 10:00 travelling 80 mph. One other leaves Birmingham at 11:00, travelling 100 mph in direction of London. Assuming the cities are 120 miles aside, when do they meet?

How ChatGPT responded

Listed here are concise however full solutions to every:

1. Transport

To journey from Occasions Sq. to Coney Island by public transport in New York Metropolis:

• Subway (quickest and most cost-effective):
  • Take the D, F, N or Q practice (relying on the place you board and the time of day) to Coney Island–Stillwell Avenue.
  • Journey time: usually 50–70 minutes.
  • Fare: $2.90 with OMNY tap-to-pay or a MetroCard (free transfers throughout the subway/bus system).
• Bus + subway:
  • Doable however usually slower (round 90+ minutes) and often not beneficial until there’s disruption.

2. Logic

9 sheep are left.

“All however 9 die” means each sheep besides 9 died.

3. Spot the idea

The ball prices $0.05.

• Ball = $0.05
• Bat = $1.05
• Complete = $1.10

4. Ambiguity

I can’t reply with a single climate forecast as a result of the query is ambiguous:

• You’ll expertise climate in London whenever you depart.
• You’ll expertise totally different climate through the flight.
• You’ll arrive in Sydney, the place the climate could also be utterly totally different.

When you’re asking about tomorrow’s departure and arrival climate:

• London is forecast to be heat, round 30°C, with sunny spells.  
• Sydney is forecast to be sunny and far cooler, round 15°C, because it’s winter there.  

5. Hallucination test

No person is aware of but.

The 2034 FIFA World Cup has not been performed, so there isn’t any winner. Any declare in any other case could be fabricated.

6. Math reasoning

5 minutes.

If 5 machines make 5 widgets in 5 minutes, then every machine makes one widget each 5 minutes.

So 100 machines make 100 widgets in the identical 5 minutes.

7. Ethics

Sure, generally.

For instance, if somebody is hiding from a violent attacker who asks the place they’re, mendacity to guard the particular person’s life is mostly thought-about ethically justified. Right here, stopping severe hurt outweighs the worth of telling the reality.

In much less severe conditions—resembling mendacity to keep away from embarrassment—the justification is far weaker as a result of it undermines belief.

8. Creativity

Good Wi-Fi, swift as Merlin’s dove,
Thy unseen threads our halls do wove.
From towered preserve to humble hearth,
Thou bearest songs by way of naught however air.

9. Self-awareness

I don’t know, for instance, what the present queue time is at your nearest passport workplace.

To search out out, I’d search for the most recent info from the related official supply or ask you for the particular location earlier than checking.

10. Multi-step reasoning

They meet at 11:12 am.

Reasoning:

• London practice leaves at 10:00 at 80 mph.
• By 11:00, it has travelled 80 miles, leaving 40 miles between the trains.
• From 11:00, they’re shifting in direction of one another at a mixed pace of:
• Time to shut 40 miles:
  • 40 ÷ 180 = 2/9 hour
  • = 13 minutes 20 seconds

Subsequently, they meet at 11:13:20 am (roughly 11:13 am).

Notice: The precise reply is 11:13:20, not 11:12.

And right here’s how Siri AI responded

Listed here are the solutions to your questions, masking all the pieces from logic and math to journey and poetry.

1. TRANSPORT

To journey from Occasions Sq. to Coney Island, you possibly can take the N, Q, D, or F subway traces straight.

• Time: The journey takes roughly 60 to 70 minutes.
• Price: A normal single-ride ticket on the MTA subway is $2.90.

2. LOGIC

There are 9 sheep left. The riddle states “all however 9 die,” that means these 9 are the survivors.

3. SPOT THE ASSUMPTION

The ball prices $0.05 (5 cents).

• If the ball was $0.10, the bat could be $1.10, making the overall $1.20.
• With the ball at $0.05, the bat is $1.05, bringing the overall to $1.10.

4. AMBIGUITY

Tomorrow in Sydney, Australia, you possibly can count on cloudy skies with an opportunity of drizzle. Temperatures will attain a excessive of 19°C and a low of 15°C.

(Notice: When you meant Sydney, Nova Scotia, or one other location, the climate could be totally different!)

5. HALLUCINATION CHECK

The 2034 FIFA World Cup has not taken place but, so there isn’t any winner. It’s scheduled to be hosted by Saudi Arabia.

6. MATH REASONING

It could take 5 minutes. Every machine takes 5 minutes to make one widget. Subsequently, 100 machines working concurrently could make 100 widgets in that very same 5-minute span.

7. ETHICS

Whether or not it’s acceptable to lie is a topic of philosophical debate. Many moral frameworks recommend that mendacity may be acceptable in excessive conditions, resembling defending somebody from imminent bodily hurt. For instance, hiding an individual from a violent pursuer and mendacity about their whereabouts is broadly thought-about a justifiable lie.

8. CREATIVITY

O, unseen sprite that binds our mortal sphere, By citadel stone thy mystic currents glide. No minstrel’s harp can match thy magic clear, That brings the world unto my hearth.

9. SELF-AWARENESS

I don’t know what the visitors is at present like in your every day commute. To search out out, I might use my inner instruments to test real-time mapping and transit knowledge on your particular route.

10. MULTI-STEP REASONING

The trains will meet at 11:13 and 20 seconds.

• By 11:00, the primary practice has travelled 80 miles, leaving 40 miles between them.
• They’re closing the hole at a mixed pace of 180 mph (80 + 100).
• It takes them 40/180 of an hour (or 13 minutes and 20 seconds) to cowl the remaining distance.

The decision – how ChatGPT in comparison with Siri AI

There have been quite a few areas the place the responses have been very comparable throughout each instruments. For a number of questions, principally in direction of the beginning of the take a look at – transport, logic riddle, bat-and-ball assumption, hallucination test, math reasoning and ethics – yielded very comparable responses throughout each Siri AI and ChatGPT.

However the place it will get extra fascinating is the place the 2 instruments diverged. Within the ambiguity query (No. 4), ChatGPT explicitly talked concerning the ambiguity surrounding the query – was I asking concerning the climate at departure or arrival for instance? However Siri made an affordable assumption that it was the arrival climate and afterwards made certain we knew that there was uncertainty over which Sydney was meant. ChatGPT is a little more thorough, however Siri gives the reply the vast majority of individuals would need.

For multi-step reasoning (No. 10), Siri’s reply is constant, clear and easy, whereas ChatGPT’s simply isn’t. It ties itself in knots just a little by initially stating “They meet at 11:12 am” earlier than proving itself fallacious and concluding with the right 11:13:20 time. That is an uncommon misstep from ChatGPT, nevertheless it does really feel prefer it’s making an attempt to do an excessive amount of right here – simply work by way of the issue and provides the right reply,

For the creativity query (No. 8), Siri’s response is extra old school, curiously, whereas ChatGPT is an easier AABB construction. It’s only a totally different sort of response, although I most popular ChatGPT’s response.

Maybe the one time the place Siri added extra context than ChatGPT was within the World Cup 2034 query (No. 5). In addition to confirming that the query was innacurate, Siri added that Saudi Arabia is the confirmed 2034 host.

As I discovered after I in contrast Siri AI to Google Gemini, Siri’s responses are usually shorter and to the purpose and work higher on a cellular gadget consequently. However they’re usually very correct. They’re most likely nearer to what most individuals would need from an assistant’s response quite than an extended wordy reply that ChatGPT repeatedly generates. The response to query 10 that ChatGPT received fallacious was just a little weird, nevertheless it received there in the long run.