Enterprises will need AI governance as large language models grow in number

With the number of large language models (LLMs) in the market expected to grow and branch out, businesses will need a governance framework to manage their generative artificial intelligence (AI) applications. 

Organizations will require layers of intelligence that pull together internal and external capabilities, said Frederic Giron, Forrester's vice president and senior research director. 

Also: 3 ways to secure the best AI partner for your business

This approach will encompass the use of paid and open-source LLMs from third parties, such as OpenAI's ChatGPT, Anthropic's Claude, and Meta's Llama, and embedded AI tools, such as Salefsforce Einstein GPT. Organizations will also have their own AI models, including using generative AI, tapping general-purpose and specialized LLMs, and running various AI applications alongside key processes, policies, and business rules. 

The approach will be underpinned by structured and unstructured data, with the latter expected to double amid the adoption of generative AI as companies deploy more conversational experiences for customers and employees, said Giron, who was speaking at the research firm's 2024 predictions briefing this week. 

Also: Two divergent skills that matter in an AI world: Math and business development

User response and behavior should also be fed into a feedback loop and used to fine-tune the system. 

These requirements underscore the need for businesses to have a generative AI application architecture to govern and ensure the use of these tools is safe and efficient, he said. 

This framework should connect the application pipes, orchestrate requests into outputs, and pave the input and output gateways, so the organization can control what data goes into the AI models and ensure the responses comply with the rules the business has set. 

The complexities around AI governance mean it might take a while before businesses will see real results from their adoption of a framework. 

Forrester predicts the transformative impact of generative AI will benefit just 30% of Asia-Pacific firms over the next year. Giron pointed to key challenges related to data governance, quality, and infrastructure.

Also: 4 ways generative AI can stimulate the creator economy 

To help businesses plug the gaps, he noted that service providers are investing in transforming how they operate and deliver their service models, including expanding their industry partnerships and releasing new platforms, such as AI studios and model comparisons. 

This investment will drive better pricing models and, over a longer term, impact commercial models. The results will be more outcome-based and solution-based pricing structures, among others. 

Also: Businesses need a new operating model to compete in an AI-powered economy

The analyst added that 56% of organizations expect employee productivity to be the leading use case for generative AI, followed by 48% that point to software development and testing. Another 48% see generative AI as an enabler of self-service data and analytics. 

Unsurprisingly, generative AI is the biggest tech thunderstorm to hit in 40 years, according to Dane Anderson, Forrester's senior vice president of international research and product. 

It follows previous "inflection point" technologies that saw cloud computing emerge in 2009 and the mobile internet and smartphone wave rise in 2007, Anderson said. The World Wide Web inflection took hold of the 1990s, while personal computers made their mark in 1981. 

Some of these innovations brought about greater changes than others, creating both opportunities and challenges, he added. With the emergence of generative AI, the analyst predicted that "static" websites will gradually be abandoned over the next 20 years. 

Also: Generative AI will far surpass what ChatGPT can do. Here's everything on how the tech advances

Users will instead evolve to prompt or ask a query, to which they will get a response that is continually updated in the backend -- powered by generative AI -- and customized for an improved interactive experience. 

These changes to websites will further impact search, which will no longer be as central or critical as it is now, Anderson said. 

Such significant transformations will play out over several years. In the shorter term, the anticipated emergence of more LLMs in the market means organizations will need to carefully assess their options and determine which models are best suited for the outcomes they want. 

Anderson also noted the potential for more market players to start embedding generative AI capabilities for free into their existing customer enterprise applications. 

Ultimately, the value for businesses is not in the layer where LLMs operate, said Leslie Joseph, Forrester's principal analyst, as this market segment will be commoditized as more LLMs pop up, he added. 

Joseph urged software vendors to start integrating generative AI features into their products, rather than offering these tools primarily as their version of a ChatGPT equivalent. This refined approach will help drive a workplace environment where generative AI capabilities are more ingrained into how employees work and make the technology more affordable for businesses, he said.