Preface
These are notes from an online talk by the three authors of
Main steps
Since the LLM industry is still in an early stage and no one has put forward a full solution yet, the authors of the book propose four steps that try to summarize the development flow of a GenAI product:
- Gen AI Superpower
- Prompt Design
- Define Trust Framework
- AI UI/UX
Step 1: Gen AI Superpower
At the very beginning, you have to confirm whether this product can only exist because GenAI enables it, rather than chasing the trend just for the sake of GenAI. So how do you confirm this? First, you have to know where GenAI’s strengths lie:
Areas where GenAI can add value
- Creative content generation
- Data generation, summarization, prediction
- Task generation, planning, execution
- Personalization
- Real-time interaction
- Access Democratization (not sure how to translate this exactly, but it roughly means that knowledge, skills and information are no longer hard to access)
So if you want to use GenAI to build a product, it is best to come at it from one of the angles above. Otherwise, you probably could already do it before the GPT era.
With these entry points in mind, the next step is to “measure” how well-suited this product is to be solved with GenAI.
Step 2: Designing prompts
Next, you have to design prompts for the GenAI product. There is plenty of content online about this, covering role, tone, response format, task examples and so on. However, if you want the GenAI’s responses to differentiate your product from others, you have to think about more details, and it really comes down to case by case (the book probably has more explanation here).
Step 3: Building a trust framework
With GenAI, on one hand the user’s input may be fairly private data, and on the other hand the generated content inevitably has some randomness. So how do you control GenAI’s generation in a way that lets users trust it and use it? There is no absolute method for this right now, but “trustworthiness” can be approached, validated and improved from these angles:
- Privacy: if user conversations are used to retrain the model, how do you guarantee personal information cannot be retrieved by other users through the model?
- Security: protecting the chatbot from being broken into by hacker techniques to leak data
- Harmlessness: making sure the chatbot does not produce harmful responses (violence, sexual content, etc.)
- Reducing bias: an LLM may already lean toward certain viewpoints in its training data, so how do you ensure the diversity of the data?
- Fairness: does the chatbot treat all users fairly (race, social status)
- Inclusivity: can the chatbot prioritize output that makes users feel welcomed and understood
- Accountability: are the chatbot’s capabilities, limits and decision process transparent
This part actually spans both the foundational general-purpose model and the application layer, but leans more toward the foundational model side.
Step 4: AI UI/UX
This is also at a very early stage right now, you could even say it has not really developed yet, because the product currently leading the market is just ChatGPT’s chat interface. But in the GenAI era, it will not be “an extension of existing designs”, it will be entirely new ways of interacting and presenting.
The authors list three research methods and directions:
- Wizard of Oz testing: users interact with what they “think” is a GenAI interface, when actually it is controlled by a human behind the scenes
- Diary studies: understanding how users’ perception and cognition of GenAI interactions change over time
- Ethical assessments: addressing the challenges of bias, transparency and security in GenAI development
Summary
Honestly it is a bit scattered. The most useful piece for me right now is the GenAI enablement checklist: do not do what does not need to be done, focus on a single point and iterate continuously, until the growth flywheel starts to spin.
The rest probably needs reading the book to really understand more deeply.
ChangeLog
- 20240416-init
- 20260501–translate by claude code