Where OpenAI’s LLM is headed - The direction of several companies(1)

This article was written on October 20, 2024. You can also view the Korean version.

Why LLM?

Many people are focused on generative AI, but at the heart of it all is the Large Language Model (LLM), because it is the new intelligence that will be as good as, and perhaps better than, humans. Sam Altman, CEO of OpenAI, has dubbed this era “The Intelligence Age.”¹ It’s no exaggeration to say that LLMs are at the center of this new era. In this article, I will introduce the direction of OpenAI’s LLM development. I will post about other companies in the future. There are many companies developing LLM foundation models, but I will introduce OpenAI(GPT-4), Meta(LLaMA), and Korean companies Naver(HyperClova), SKT, and KT.

OpenAI: New Intelligence, Practical AI

OpenAI launched the ChatGPT service and started the generative AI boom. And it’s still driving the evolution of LLMs. Honestly, OpenAI’s progress since November 2022, when they launched ChatGPT, has been phenomenal.

• On November 30, 2022, launched the ChatGPT service based on GPT-3.5
• March 14, 2023 GPT-4 released.
• May 13, 2024 GPT-4o released.
• June 18, 2024 GPT-4o mini released.
• September 12, 2024 GPT-o1 mini, preview released.

Right now, it seems like OpenAI is building AI in two directions: (1) new intelligence that will go beyond humans, and (2) the practicalization of that intelligence. However, in the case of OpenAI, it feels like the direction is more focused on (3) intelligence itself.

A new intelligence beyond humans

OpenAI features AGI in its mission. OpenAI’s mission is “to ensure that artificial general intelligence benefits all of humanity.”² It’s almost as if AGI already exists, right? That’s because they’re really aiming to develop AGI. In February 2023, OpenAI published a post called Plans for AGI and Beyond.³ You can see that ChatGPT is just a waypoint on the way to that destination.

AGI is general purpose AI. Whereas traditional AIs specialize in specific areas (Go, dog/cat recognition), AGIs can make comprehensive reasoning about a wide range of things, just like humans. A few years ago, there were many observations that such an AGI was far off, but now no one doubts that we will have one in the near future.

As an intermediate point on the road to AGI, GPTs have gotten remarkably better at reasoning with each version. ⁴ Just comparing the most recent models, o1 and 4o, the difference is staggering. We used to have benchmark metrics to evaluate LLMs, but now we just measure their performance on human-solvable problems. On PhD-level science questions, we see that LLMs do better than humans.

Practical AI

Another direction is practicality. It’s getting faster and cheaper. Faster means you can reason in real time. Cheaper means it can be used in more applications. These are essential conditions for real-world use.

GPT-o1 mini API requests cost 10x less than GPT-4, even with significantly better inference compared to GPT-4. GPT-4o is twice as fast as GPT-4. GPT-o1 has slowed down as it has improved its inference capabilities. However, when OpenAI releases a new model, it also releases a mini version, which has similar performance but is much faster, showing that they also considers practicality.

OpenAI focuses on AGI only

OpenAI seems to be focused only on the journey to create AGI. This means they don’t seem to be thinking much about making it commercially viable. OpenAI is still running a huge deficit. By 2024, it’s projected to have a $5 billion deficit, and by 2026, it’s projected to have a $14 billion deficit. Sam Altman says of OpenAI’s deficit, “I don’t care. As long as we stay on a trajectory that creates much more value for society than that, and as long as we can figure out how to develop AGI, that’s worth it.”⁵

However, the future doesn’t look bleak for OpenAI, as it’s closer than anyone else to AGI while creating intelligence that’s overwhelmingly superior to its competitors. On 10/2/24, OpenAI raised $6.6 billion in funding.⁶ Microsoft, Nvidia, and Softbank have invested a lot of money in OpenAI. On top of all that, OpenAI also partnered with the mighty Apple on 6/10/24 to include ChatGPT in iOS.⁷ I think Apple has recognized that they can’t create an intelligence like ChatGPT in a short period of time. I’m excited to see it in iOS sometime next year.

Of course, there’s a problem. It’s going to take a lot of money to build AGI. On 9/27/24, OpenAI announced plans to turn into a for-profit company. ⁸ My guess is that it’s because they need the money. OpenAI was originally a non-profit organization, but after the transition, it looks like they will have a for-profit company under the non-profit umbrella. I’m not sure if it’s because of the transition to for-profit, but some technical staff left, including CTO Mira Murati.⁹

Can OpenAI create AGI? Can this AGI contribute to human prosperity? We’ll see, but the moment they transition to a for-profit business… I think it’s going to be hard.

---