lexical token
Sign in to saveAlso known as token
in lexical analysis, atomic string with an assigned and thus identified meaning; can be either word, bi-gram or sentence
Also known as token
in lexical analysis, atomic string with an assigned and thus identified meaning; can be either word, bi-gram or sentence
Discovered by embedding cosine similarity (sentence-transformers MiniLM, 384-dim).