Rendered at 07:54:30 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
augment_me 2 hours ago [-]
Its cool, but it goes against the whole optimization that NLP has gone through with BPE and redundancy/repetition in large data in general.
Put this on a coding task or English and you will underperform against tokenizer that exploit multi-char patterns and can decode much faster.
Given where things are going with spec decode (I've seen 32 token spec-decode pop up in the frontier models), you will need a very domain-specific use for this to be useful imo like massive tabular data, but at that point you already have everything structured, and might as well use an LLM with polars/sql/arrow.
vatsachak 7 hours ago [-]
Oh man, that haskell quicksort tokenization has got me feeling great
Put this on a coding task or English and you will underperform against tokenizer that exploit multi-char patterns and can decode much faster.
Given where things are going with spec decode (I've seen 32 token spec-decode pop up in the frontier models), you will need a very domain-specific use for this to be useful imo like massive tabular data, but at that point you already have everything structured, and might as well use an LLM with polars/sql/arrow.