Please enable JavaScript.

Coggle requires JavaScript to display documents.

LLM Systems Seminar (CS7670): Week05a, FlashAttention (paper, HotCRP,…

- - - - the number of arithmetic operations per byte of memory access
  - - - Borrowed from Stanford CS336
      - Borrowed from this Blog
    - - Borrowed from NVIDIA doc
- - - - Pro side: Yes! The research community should care, even if the techniques might be irrelevant in a couple of years.
      - Con side: No! The research community should allow transient techniques to be disseminated through other channels (e.g., open-source communities), regardless of their current popularity.
- - - - DSA in DeepSeek V3.2
  - - - Q: Should it be floor or ceil?