Sharing Our Internal Training Material: LLM Terminology Cheat Sheet!

Title: Introducing Our Internal LLM Terminology Reference Guide: A Comprehensive Cheat Sheet for AI Practitioners

In the rapidly evolving field of large language models (LLMs), staying aligned on terminology and core concepts is essential for effective collaboration, research, and implementation. Recognizing this need, our team developed an internal reference resource designed to clarify and unify understanding across various discussions, papers, and evaluations related to AI and LLMs.

Today, we are pleased to share this resource publicly in hopes that it may prove valuable to the broader AI community. The comprehensive cheat sheet offers a structured overview of fundamental topics, making it easier for practitioners to navigate the complex landscape of modern language models.

Overview of the Cheat Sheet

The document is organized into key thematic sections:

1. Model Architectures
2. Transformer: The foundational architecture powering most modern LLMs.
3. Encoder–Decoder: Used in models tailored for tasks like translation.
4. Decoder-Only: Models optimized for generative tasks.

5. Mixture of Experts (MoE): Architectures that leverage sparse routing for efficiency.

6. Core Mechanisms

7. Attention: The mechanism enabling models to weigh different parts of input data.
8. Embeddings: Techniques for representing words and tokens in vector space.
9. Quantization: Methods for reducing model size and computational load.

10. Low-Rank Adaptation (LoRA): Techniques for efficient fine-tuning.

11. Training Methodologies

12. Pre-training: Initial large-scale training on vast datasets.
13. Reinforcement Learning with Human Feedback (RLHF) / RLAIF: Fine-tuning approaches incorporating human preferences.
14. Quantized Low-Rank Adaptation (QLoRA): Combining quantization with LoRA for resource-efficient training.

15. Instruction Tuning: Fine-tuning models based on task-specific instructions.

16. Evaluation Benchmarks

17. GLUE: A suite for evaluating language understanding.
18. MMLU: A benchmark for assessing multilingual and multi-task understanding.
19. HumanEval: Measuring code generation quality.
20. GSM8K: A benchmark for mathematical problem solving.

Purpose and Audience

This resource is tailored for AI practitioners, researchers, and developers who frequently encounter inconsistent or scattered terminology across LLM literature and documentation. By providing clear, concise definitions and categorizations, we hope to facilitate better communication and understanding within the community.

We believe that shared knowledge accelerates innovation and reduces misunderstandings, paving the way for