August 2nd, 2024

Google Gemini 1.5 Pro leaps ahead in AI race, challenging GPT-4o

Google has launched Gemini 1.5 Pro, an advanced AI model excelling in multilingual tasks and coding, now available for testing. It raises concerns about AI safety and ethical use.

Read original articleLink Icon
Google Gemini 1.5 Pro leaps ahead in AI race, challenging GPT-4o

Google has launched Gemini 1.5 Pro, an advanced artificial intelligence model, now available for early testing through Google AI Studio and the Gemini API. This release marks a significant advancement in AI capabilities, quickly achieving the top position on the LMSYS Chatbot Arena leaderboard with an ELO score of 1300, surpassing competitors like OpenAI's GPT-4o and Anthropic's Claude-3.5 Sonnet. The model excels in multilingual tasks, mathematics, complex prompts, and coding, and has also secured the top spot on the Vision Leaderboard, highlighting its multimodal capabilities.

Gemini 1.5 Pro builds on the previous Gemini 1.5 model, featuring an extensive context window of up to two million tokens, allowing it to process large amounts of information effectively. This could enhance enterprise operations in data analysis, software development, and customer interactions. However, the release raises concerns about AI safety, ethical use, and potential misuse, intensifying the ongoing debate about the pace of AI development.

Google's decision to involve the community in testing reflects a trend towards open development in the AI industry. As Gemini 1.5 Pro presents both opportunities and challenges for businesses, its real-world performance will be closely monitored. This release signifies a pivotal moment in the AI arms race, as Google aims to redefine the capabilities of AI systems and challenge its competitors in the tech landscape.

Related

Gemini's data-analyzing abilities aren't as good as Google claims

Gemini's data-analyzing abilities aren't as good as Google claims

Google's Gemini 1.5 Pro and 1.5 Flash AI models face scrutiny for poor data analysis performance, struggling with large datasets and complex tasks. Research questions Google's marketing claims, highlighting the need for improved model evaluation.

How it's Made: Interacting with Gemini through multimodal prompting

How it's Made: Interacting with Gemini through multimodal prompting

Alexander Chen from Google Developers discusses Gemini's multimodal prompting capabilities. Gemini excels in tasks like pattern recognition, puzzle-solving, and creative applications, hinting at its potential for innovative interactions and creative endeavors.

Google's Gemini AI caught scanning Google Drive PDF files without permission

Google's Gemini AI caught scanning Google Drive PDF files without permission

Google's Gemini AI scans Google Drive PDFs without consent, sparking privacy concerns. Users struggle to disable this feature, raising questions about user control and data privacy within AI services.

Gemini AI caught scanning Google Drive hosted PDF files without permission

Gemini AI caught scanning Google Drive hosted PDF files without permission

Google's Gemini AI scans PDFs on Google Drive without consent, raising privacy concerns. Users struggle to disable the feature, possibly linked to Google Workspace Labs settings. Lack of transparency emphasizes privacy risks.

Gemini Pro 1.5 experimental "version 0801" available for early testing

Gemini Pro 1.5 experimental "version 0801" available for early testing

Google DeepMind's Gemini family of AI models, particularly Gemini 1.5 Pro, excels in multimodal understanding and complex tasks, featuring a two million token context window and improved performance in various benchmarks.

Link Icon 11 comments
By @nycdatasci - 6 months
I don't trust lmsys leaderboard anymore. I tried playing with the new model and on my very first prompt it told me a map that I asked it to analyze was sexually explicit. First prompt! Do they have a QA team?

Image was similar to this: https://i.pcmag.com/imagery/articles/03Rqi88BidPCECjR5HeA0ex...

By @jonomacd - 6 months
There is such a strong narrative against Google currently. It's really interesting seeing goalpost move and excuses made for this latest leaderboard. The fact of the matter is Google, openAI and anthropic all have excellent models with similar capabilities.
By @lispisok - 6 months
Anecdote but recently chatgpt has been giving way too many factually incorrect answers to tech related questions and Gemini has been getting them right.
By @testfrequency - 6 months
Gemini is still the king of being far too serious and incorrectly assuming bad intentions about almost every single interaction I have with it.

At least with GPT and Llama, positive intent is assumed or questioned, but with pushback I’m able to move forward.

By @ipaddr - 6 months
Terrible writing full of fluff. "With this release, Google has thrown down the gauntlet, challenging its competitors and pushing the boundaries of what’s possible in AI."
By @Oras - 6 months
Google made many mistakes with AI that their models do not get attention anymore.

When Anthropic Sonnet doing great at text and coding, OpenAI with their reach via web, mobile, and desktop app, and free deepseek interface with great coding model, who needs to turn to Google with their 20 years old looking interface?

To start, they need to start making modern interface and modern billing to catchup rather than tying it to GCP

By @jerojero - 6 months
For a while I stopped being impressed.

I mean, don't get me wrong, I think these models are pretty good as they are and they can be useful if they could run on devices natively which seems to be something that might be happening. That's exciting.

But in terms of these models getting better... I don't know. I think we've been doing very incremental upgrades rather than big changes for a while (a while being like 1 year... but that's how fast this tech has moved).

By @teleforce - 6 months
Instead of having a single AI Inference engine perhaps we can have two for cross checking in mitigating hallucinations.
By @FractalHQ - 6 months
GPT-4o was left in the dust by Claude months ago…