Please enable JavaScript.

Coggle requires JavaScript to display documents.

LLM Systems Seminar (CS7670): Week05b, Optimizations (lottery) - Coggle…

- - - - making impossible possible
      - batching
      - caching
      - pipelining
      - hardware specialization
      - Principles and Methodologies for Serial Performance Optimization (OSDI'25)
  - - - Q: what is the computation here?
        
        NN = code + data
    - - Q: storage is less important?
- - - - Fast Inference from Transformers via Speculative Decoding, ICML'23
      - Accelerating large language model decoding with speculative sampling, 2023
      - SpecInfer: Accelerating Large Language Model Serving with Tree-based Speculative Inference and Verification, ASPLOS'24
      - trade computation for latency (pretty unique)
        
        Borrowed from Unlocking Efficiency in Large Language Model Inference:A Comprehensive Survey of Speculative Decoding
      - adoption: vLLM, Speculative Decoding
  - - - DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models
      - Why
        
        dilemma
        
        capability (model size)
        
        cost (mostly serving)
        
        core insight: sparsity
      - What
      - A Survey on Mixture of Experts in Large Language Models
      - OLMoE: Open Mixture-of-Experts Language Models
  - - - LoRA Without Regret
      - LoRA: Low-Rank Adaptation of Large Language Models
      - PEFT: Parameter-Efficient Fine-Tuning (vs. Full Parameter Fine-Tuning)
  - - - Tim Dettmers: 8-bit Methods for Efficient Deep Learning (talk)
        
        quantization
        
        challenge 1 for LLMs
        
        challenge 2 for LLMs
        
        result
  - - - goals
      - opportunities (for example)
  - - - Borrowed from TASO slides
      - TASO: Tensor Algebra SuperOptimizer