AI: open-source models threaten the profits of big tech contenders

In the 1960s, a scientist at the Massachusetts Institute of Technology created a natural language processing program that could mimic human conversation. Named ELIZA, it was an early iteration of the chatbots sweeping the tech sector this year. Elijah was not a profitable endeavor. Neither are the current versions.

Generative Artificial Intelligence has clear transformative potential. Chatbots developed using large language models (LLMs) can allow seamless communication between humans and computers.

The question for investors is whether proprietary LLMs can reliably make money for big tech. An open source LLM can be a cheaper option for businesses looking to develop bespoke applications.

There is no formal definition of LLM. They are described as programs trained on large amounts of data available online and able to predict the next word in a sentence.

As computing power has increased, AI has become capable of unsupervised learning from unstructured data. They give such answers that surprise even their creator.

LLM complexity has escalated. In 2020, OpenAI released its Generative Pretrained Transformer 3 or GPT-3. There were 175bn parameters in this LLM.

The more parameters, the more data an LLM can process and generate. Google’s PaLM, which powers its Bard chatbot, has 540bn. The latest version of OpenAI’s LLM is GPT-4. The company hasn’t specified the number of parameters. Pundits believe that 100 tonnes would be an accurate figure.

The processing power required for such an LL.M. is enormous. The general rule is that the larger the data set used, the better the performance. This, in theory, limits the LLM to a small number of well-funded companies.

But niche applications can function using smaller data sets. Bloomberg GPT, intended to aid in the analysis of information on Bloomberg data terminals, has 50 billion parameters. Toronto-based start-up Foghere AI’s base model LLM has 52 billion parameters.

Of greater concern to companies like Google is the open source LLM. Meta gave away its system, LLaMA, as open-source software that could be copied and used by anyone. Smaller LLMs can be built on top of this.

Assuming enterprise users decide there is little difference between proprietary and open source LLMs when developing their own AI services, Google and OpenAI will lose their early mover advantage before they have a chance to break even. Is.