Large language models & generative AI - oh my!

Introduction

The field of artificial intelligence (AI) has long endured a troubled reputation concerning the promises made and results delivered. This history has led to numerous periods known as "AI Winters," during which funding from government agencies and the venture community dwindles significantly. The recent hype surrounding large language models (LLMs) and generative AI raises the question of whether we may be facing yet another cycle of inflated expectations and disappointments.

Understanding AI Winters

AI Winters occur when grand promises issued by researchers and startup founders fail to materialize into stellar results. The cycle usually begins with high expectations, followed by funding from government and private sectors. However, when the anticipated outcomes do not meet expectations, funding cuts ensue, leading many researchers to abandon the field. This historical pattern raises concerns regarding the sustainability of current AI advancements.

An example of this cycle can be illustrated by the AI hype cycle, where extravagant publicity and promotion escalate expectations beyond reality. The cycle often culminates in unrealistic expectations, leading to disappointment and subsequent funding cuts, leaving the industry in a state of disarray.

What Are Large Language Models?

Large language models are deep learning constructs that leverage massive datasets, sometimes comprising as many as 50 billion web pages or even more. At the heart of LLMs lies the Transformer model, consisting of neural networks with encoder-decoder architectures and self-attention capabilities. This enables unsupervised training and the ability to process sequences of data in parallel, significantly reducing training times when utilizing a GPU (Graphics Processing Unit).

Foundation models, fully pre-trained large language models, can be further specialized to create generative AI applications capable of producing new text, images, audio interpretations, and more. Noteworthy generative AI applications include ChatGPT, DALL-E, Midjourney, and, more recently, Google Gemini.

Generative AI Applications in Action

For example, ChatGPT allows users to request new text based on provided inputs. Users often test its capabilities by asking for personal biographies. While earlier versions struggled with accuracy, improvements in ChatGPT 4.0 brought more satisfactory results, only requiring minor modifications.

Midjourney, another generative AI application, creates images based on textual descriptions. In a recent personal project aimed at crafting unique place cards for a dinner party, prompts were given to Midjourney to generate images featuring turkeys in various humorous and thematic settings. The resulting images showcased Midjourney's impressive capabilities, although some ethical considerations regarding copyright arose.

As generative AI technology progresses, one prominent concern relates to copyright infringement and trademark issues stemming from the use of internet-sourced training data. The legal landscape surrounding these concerns can become complicated, as evidenced by lawsuits, including a notable case in which the New York Times sued Microsoft and OpenAI over copyright violations.

To mitigate potential legal troubles, companies might consider utilizing generative AI models that draw from proprietary archives. For instance, Disney could create a model trained exclusively on its internal art assets, rather than scraping data indiscriminately from the internet.

The Future of Generative AI

Despite the ongoing challenges, generative AI holds tremendous promise, particularly in entertainment and game development. The potential for rapid production of concept art and assets could transform workflows in these industries. As more companies enter the generative AI space, resolving copyright and trademark issues will be critical for future advancements and, potentially, for avoiding another AI Winter.

Many start-ups are appearing in this domain, with significant interest from investors. As more people and resources are committed to generative AI, there is hope that we can steer clear of historical patterns and instead pave the way for a bright future in the industry.

Keywords

• Artificial Intelligence
• AI Winters
• Large Language Models
• Generative AI
• Transformers
• ChatGPT
• Midjourney
• Copyright Issues
• Trademark Concerns
• Entertainment Industry

FAQ

1. What are AI Winters?
AI Winters occur when high expectations for AI fail to materialize, leading to reduced funding and interest in the field.

2. What are large language models?
Large language models are deep learning systems trained on massive datasets, widely used for understanding and generating human-like text.

3. What role do Transformers play in large language models?
Transformers, consisting of encoder-decoder architectures with self-attention mechanisms, are central to the functioning of large language models.

4. What is generative AI?
Generative AI refers to applications that create new content—such as text, images, and audio—based on the input data provided.

5. What are some examples of generative AI applications?
Notable examples of generative AI applications include ChatGPT, DALL-E, and Midjourney.

6. How are copyright and trademark issues relevant to generative AI?
Generative AI often uses datasets scraped from the internet, leading to potential copyright and trademark infringement, which raises legal concerns.

7. How can companies mitigate copyright issues with generative AI?
Companies might mitigate copyright issues by training generative models on proprietary data or their own archives rather than openly available information from the internet.