Please enable JavaScript.
Coggle requires JavaScript to display documents.
Lessons Learned from Creating GenAI Applications - Coggle Diagram
Lessons Learned from Creating GenAI Applications
Types of Applications
Natural Language Ops
Summarization
Categorization
Personalization
Generation
Delivery/Coding Assistants
Conversational Agents
Information Retrieval (RAG)
Challenges
Cost Explosion
Infra
LLM
Monitoring
Scaling
Human/Computer
Interaction
UX
How much should the LLM do?
Pre-processing/Post-processing
Feedback cycles
Personalization vs Generalization
LLM issues
Answer Quality
Reliability/Consistency
Cost
Performance/Latency
Model Reliability
Reducing costs with scale
Reducing dependence on LLM
AI Literacy
Team Capability
Guardrails
Use case mismatch
Bias/Fairness
Archtectural
Data Privacy & Security
Observability
Validations
Scalability
Integration with Legacy systems
Change Management
Tech Choices
Open Source vs Commercial
Build vs Buy
Hosted vs Local
Cloud Platform
Construction
Requirements Gathering
Feature creep
Tooling & Environment Setup
Code Quality
Integration
MLOPS
Compliance
Answer Validation
Tech/Model advancements
Lessons Learned
AI Literacy
Understand Limitations & Caveats
Encourage Experimentation
Continuous capability building in teams
Use case whetting
Establish Guardrails
Understand ecosystem
Human/Computer Interactions
Map application journey
Understand touchpoints
When to rely on LLM/Human
Feedback mechanisms on output
Build transparency
Architectural
Privacy/Security
Privacy by Design
Encryption & strict access control
Locally hosted/Guaranteed hosted systems
Local/Hosted
Cost, Control, Scalability, Compliance
Available competency
Hybrid models work well
Build vs Buy
Buy commoditized
Support needed
Build core necessities
Build to protect IP
Monitoring/Observability
Holistic logging
Monitor & analyse usage
Quality vs models
Periodic Audits
LLM Usage
Track Model output constantly
Estimate costs vs usage upfront
Hybrid LLMs
No great way to automate reply quality
Pigeonhole answers into output formats
Minimize answer sizes
Parallel calls to LLM
Use Agents & Classifiers
Construction
Modular development for models/prompts
Prompt Engineering
Profile application performance/timings
Metrics for answer reliability
Constant human feedback
Periodic model output reliability
Do not be creative with answers!
Engineering Rigor and Discipline
Define quality gates
Reduce token sizes
Rely on more pre and post processing