Evan Harmon - Memex

Language model

A Language model is a probabilistic model of a natural language. In 1980, the first significant statistical language model was proposed, and during the decade IBM performed ‘Shannon-style’ experiments, in which potential sources for language modeling improvement were identified by observing and analyzing the performance of human subjects in predicting or correcting text.
wikipedia:: Language model
  • Proprietary
    • Open Source
      • Benefits
        • Transparent/open - Open Source
        • Big one - you can fine-tune any open model however you want
        • Community contributions, etc
        • Can benchmark them according to how truthful they are
      • LLAMA2 - many/most open source models are based on this one
      • VICUNA
      • BLOOM
      • Running locally
    • Hugging Face
  • Training
Language model
Interactive graph
On this page
Language model