Weekly Tech Thread - ~2024.9.25

Something Old:

• Gibbings, ”Dimensional Analysis”
  

  
  

Something New:

• Kian J. Khojastepour, Matin Mortaheb, Sennur Ulukus, “Age of Gossip in Networks with Multiple Views of a Source”; seems like this will be useful in thinking about Ames eventual consistency.
  

  
  

Something Borrowed:

• Anne Mangen, Bente R. Walgermo, Kolbjørn Brønnick, “Reading linear texts on paper versus computer screen: Effects on reading comprehension” (h/t Hacker News)
  

  
  

Something Blue:

• Aquino, “Learning Programming Languages with Koans”
  

• ↑

  0

  ↓

  ~lomped-firser

  1 months ago

  koans are a great way to learn or enhance learning imo, especially if they have zero setup overhead like this http://clojurescriptkoans.com/
  

  reply
• ↑

  0

  ↓

  ~mopfel-winrux

  1 months ago

  "Dimensional Analysis" is a classic and worth reading.
  

  reply
• ↑

  0

  ↓

  ~tastem-pacper

  1 months ago

  This week in LLMs: "Discovering the Gems in Early Layers: Accelerating Long-Context LLMs with 1000x Input Token Reduction", from Salesforce.
  

  
  

  
  achieves a 2.4× speedup and 30\% reduction in GPU memory usage compared to SOTA methods. Evaluation on the Needle in a Haystack task shows that GemFilter significantly outperforms standard attention, SnapKV and demonstrates comparable performance on the LongBench challenge. GemFilter is simple, training-free, and broadly applicable across different LLMs. ...Our research demonstrates that LLMs can identify relevant tokens in the early layers before generating answers to a query. Leveraging this insight, we propose an algorithm that uses early layers of an LLM as filters to select and compress input tokens, significantly reducing the context length for subsequent processing. Our method, GemFilter,

  reply