GPTs Are Maxed Out
OpenAI's Orion model is expected to underperform compared to GPT-4, raising concerns about financial viability and diminishing returns in AI advancements, while new paradigms are being explored for future development.
Read original article
OpenAI's anticipated next-generation AI model, referred to internally as Orion, is expected to be less revolutionary than previously claimed by CEO Sam Altman. While Orion does outperform its predecessors, the improvements are not as significant as the leap from GPT-3 to GPT-4. This raises concerns about the model's cost-effectiveness, as it may require more resources to operate, potentially disappointing shareholders. The scaling laws that AI companies have relied upon—believing that increasing model size and computational power would yield better performance—are now being questioned. Experts, including OpenAI's Noam Brown, have suggested that the financial feasibility of developing increasingly complex models may be reaching its limits. The article also references cognitive scientist Gary Marcus, who has long criticized the deep learning paradigm and its limitations, suggesting that the industry may be facing diminishing returns. As AI companies explore new paradigms, such as OpenAI's o1 model, which focuses on real-time computation and reinforcement learning, the future of generative AI remains uncertain. The potential for Orion to disappoint users and investors looms large, with critics already predicting a downturn for generative AI if expectations are not met.
- OpenAI's Orion model may not deliver the expected performance improvements over GPT-4.
- The financial viability of developing larger AI models is being questioned.
- Experts warn of diminishing returns in deep learning advancements.
- New AI paradigms, like OpenAI's o1, are being explored to address current limitations.
- The future of generative AI is uncertain, with potential backlash from users and investors.
Related
The Subprime AI Crisis
The generative AI industry, especially OpenAI, faces financial challenges, seeking $6.5 billion while projecting $5 billion losses in 2024, with low adoption rates and skepticism about new model o1's capabilities.
OpenAI will fall the hardest when the AI bubble bursts, analyst says
OpenAI raised $6.6 billion, reaching a $157 billion valuation, but faces challenges from competition and price erosion. Analysts warn of a potential bubble and question its growth sustainability.
OpenAI's Lead over Other AI Companies Has Largely Vanished
OpenAI's competitive edge in AI has diminished as models like Anthropic's Claude 3.5 and Google's Gemini 1.5 match or surpass GPT-4o, while inference costs decline significantly due to competition.
OpenAI plans to release its next big AI model by December
OpenAI will release its new AI model, Orion, by December 2024, initially limiting access to select partners. Orion is expected to be much more powerful than GPT-4 amid company restructuring.
OpenAI's new "Orion" model reportedly shows small gains over GPT-4
OpenAI's upcoming "Orion" model shows limited improvements over GPT-4, attributed to a lack of quality training data. The AI industry faces stagnation, but CEO Sam Altman remains optimistic about future developments.
10 commentsThere is no AGI. No one is creating Minds from Iain M Banks books. It's the world's best autocomplete. It's if you look at the Earth once from space, and "recreate" it by making a giant disc that looks like that view of the Earth. Nothing behind it is like intelligence as we would think of it.
AI is going to do some amazing things still. AGI is just an unbelievably high bar.
Yes! The biggest usability issue with almost all the AI products I’ve used has been the termination problem, especially with fixed price offerings like ChatGPT. The LLM doesn’t actually have any time to “think” (however you might want to interpret that) except for the attention mechanism that visits each token.
I want to be able to query an llm, give it access to web search, and give it a time limit so that it keeps going, consuming sources a hundred pages deep, “thinking” and writing a report for me until it’s given a signal to terminate.
Gary Marcus has been saying this since at least 2012:
> Yet deep learning may well be approaching a wall, much as I anticipated earlier, at beginning of the resurgence (Marcus, 2012)
To me it feels like having continually predicted rain throughout the longest dry-spell in history. Eventually you'll be right (assuming that we'll move to some other paradigm in due course), but it's hardly prophetic.
I think that the challenges of intelligence and non-shallow reasoning will inherently involve fighting against diminishing returns (which is what the supposedly broken scaling laws predict) regardless of technique. Same as for computer graphics - doubling the compute doesn't double the perceptual quality, but that doesn't necessarily mean it's hitting a wall.
This is a self-contradicting paragraph. It cites quantum mechanics understanding as breaking down Moore's law while ignoring the fact that it has implications on our so-called "laws" of physics in just the same way.
I can imagine the author having originally written "physics" instead of "gravity" but needing to re-write the paragraph to squirm away from the contradiction they set up for themselves. I feel that this presentation is not intellectually honest.
A lot of people really really really want this to fail.
Related
The Subprime AI Crisis
The generative AI industry, especially OpenAI, faces financial challenges, seeking $6.5 billion while projecting $5 billion losses in 2024, with low adoption rates and skepticism about new model o1's capabilities.
OpenAI will fall the hardest when the AI bubble bursts, analyst says
OpenAI raised $6.6 billion, reaching a $157 billion valuation, but faces challenges from competition and price erosion. Analysts warn of a potential bubble and question its growth sustainability.
OpenAI's Lead over Other AI Companies Has Largely Vanished
OpenAI's competitive edge in AI has diminished as models like Anthropic's Claude 3.5 and Google's Gemini 1.5 match or surpass GPT-4o, while inference costs decline significantly due to competition.
OpenAI plans to release its next big AI model by December
OpenAI will release its new AI model, Orion, by December 2024, initially limiting access to select partners. Orion is expected to be much more powerful than GPT-4 amid company restructuring.
OpenAI's new "Orion" model reportedly shows small gains over GPT-4
OpenAI's upcoming "Orion" model shows limited improvements over GPT-4, attributed to a lack of quality training data. The AI industry faces stagnation, but CEO Sam Altman remains optimistic about future developments.