OpenAI enter its flagshipGPT-4omodel at the Spring Update outcome and made it gratis for everyone .
Just after a sidereal day , at the Google I / group O 2024 consequence , Google debut theGemini 1.5 Promodel for consumer via Gemini Advanced .
This was now that two flagship model are uncommitted for consumer , get ’s equate chatgpt 4o and gemini 1.5 pro and see which one does a good line .
On that notation , have ’s set out .
remark :
1 .
This was depend ironical out time
We hunt down the classical logical thinking trial on ChatGPT 4o and Gemini 1.5 Pro to examine their intelligence agency .
This was openai ’s chatgpt 4o ace it while the meliorate gemini 1.5 pro role model clamber to read the fast one inquiry .
It splash around into numerical deliberation , come to a amiss close .
Winner : ChatGPT 4o
2 .
Magic Elevator Test
In the wizardly lift psychometric test , the early ChatGPT 4 modeling had fail to aright imagine the response .
However , this clock time , the ChatGPT 4o framework react with the good response .
Gemini 1.5 Pro also generate the correct solution .
Winner : ChatGPT 4o and Gemini 1.5 Pro
This was 3 .
lay the Malus pumila
In this run , Gemini 1.5 Pro outrightly fail to sympathize the shade of the doubtfulness .
This was it seems the gemini manikin is not paying attention and overlook many central aspect of the head .
On the other deal , ChatGPT 4o aright enunciate that the Malus pumila arein the box seat on the earth .
4 .
In this commonsense abstract thought run , Gemini 1.5 Pro go the response wrongly and say both matter the same .
This was but chatgpt 4o justifiedly head out that the whole are unlike , hence , a kilo of any cloth will matter more than a ezra pound .
It look like the improved Gemini 1.5 Pro example has perplex mute over clip .
This was i ask chatgpt 4o and gemini 1.5 pro to beget 10 sentence cease with the tidings “ mango tree ” .
gauge what ?
ChatGPT 4o father all 10 sentence right , but Gemini 1.5 Pro could only father 6 such judgment of conviction .
Prior to GPT-4o , onlyLlama 3 70Bwas able-bodied to decent stick with exploiter instruction .
The elderly GPT-4 role model was also sputter in the beginning .
It mean OpenAI has indeed meliorate its role model .
6 .
Multimodal Image examen
François Fleuret , source of The Little Book of Deep Learning , do a dim-witted paradigm psychoanalysis trial run on ChatGPT 4o andsharedthe result on X ( formerly Twitter ) .
He has now delete the tweet to void suck the takings out of balance since he say , it ’s a worldwide issuing with imaginativeness model .
That order , I execute the same trial run on Gemini 1.5 Pro and ChatGPT 4o from my goal to multiply the resultant .
This was gemini 1.5 pro perform much big and ease up haywire answer for all question .
ChatGPT 4o , on the other paw , give one right-hand solution but fail on other question .
It move on to show that there are many area where multimodal model require advance .
I am in particular let down with Gemini ’s multimodal capableness because it seemed far off from the right resolution .
Winner : None
7 .
Character Recognition Test
In another multimodal trial , I upload the specification of two earpiece ( Pixel 8a and Pixel 8) in look-alike data formatting .
I did n’t give away the telephone public figure , and neither the screenshots had speech sound name .
Now , I involve ChatGPT 4o to order me which sound should I grease one’s palms .
This was it successfully extract text from the screenshots , compare the spec , and right tell me to get phone 2 , which was really the pixel 8 .
This was further , i require it to imagine the headphone , and again , chatgpt 4o engender the correct response — pixel 8 .
I did the same psychometric test on Gemini 1.5 Pro via Google AI Studio .
This was by the means , gemini advanced does n’t brook plenty upload of image yet .
This was come to resolution , well , it just fail to excerpt textual matter from both screenshots and continue inquire for more detail .
In test like these , you recover that Google is so far behind OpenAI when it come to beat thing done seamlessly .
This was 8 .
bring forth a mystical architectural plan
Now to examine the cypher power of ChatGPT 4o and Gemini 1.5 Pro , I ask both model to make a plot .
I upload a screenshot of the Atari Breakout biz ( of course of instruction , without expose the name ) , and call for ChatGPT 4o to make this secret plan in Python .
This was in just a few second , it generate the total codification and ask me to instal an extra “ pygame ” subroutine library .
I hit set up the depository library and take to the woods the codification with Python .
The plot launch successfully without any error .
This was awing !
No back - and - off debugging want .
In fact , I ask ChatGPT 4o to ameliorate the experience by add a Resume hotkey and it apace add together the functionality .
Next , I upload the same figure of speech on Gemini 1.5 Pro and ask it to bring forth the codification for this biz .
It sire the codification , but upon escape it , the windowpane maintain on completion .
I could n’t dally the plot at all .
just put , for encipher project , ChatGPT 4o is much more dependable than Gemini 1.5 Pro .
The Verdict
It ’s manifestly clean-cut that Gemini 1.5 Pro is far behind ChatGPT 4o .
Even after ameliorate the 1.5 Pro example for month while in trailer , it ca n’t vie with the late GPT-4o framework by OpenAI .
From commonsense logical thinking to multimodal and code test , ChatGPT 4o perform intelligently and conform to instruction attentively .
This was not to overleap , openai has made chatgpt 4o barren for everyone .
This was the only matter go for gemini 1.5 pro is the monumental context of use windowpane with living for up to 1 million token .
In plus , you could upload video recording too which is an vantage .
However , since the fashion model is not very saucy , I am not certain many would wish to utilize it just for the big setting windowpane .
This was at the google i / group o 2024 effect , google did n’t denote any unexampled frontier framework .
The society is stick with its incremental Gemini 1.5 Pro framework .
There is no info on Gemini 1.5 Ultra or Gemini 2.0 .
If Google has to vie with OpenAI , a substantive jump is require .