Original data study
How far does the HSK 4 vocabulary actually go across different kinds of Chinese text?
How this was collected
We match the same cumulative HSK vocabulary against five openly-licensed Chinese corpora, one per reading register (TV/film subtitles, news and web text, Chinese Wikipedia, classic vernacular novels, and classical Chinese), and compute what share of each corpus's Chinese-character word tokens that vocabulary covers. All five registers are computed by one identical script (scripts/build-genre-matrix.py); the subtitles register reproduces the original single-corpus coverage figures exactly, which is how we validate the method before trusting it on the other four. A register is included only once its corpus is licensed for this use and its HSK words are matched against it directly, never estimated.
This dataset was computed .
What the data shows
Registers compared
5
Lowest coverage
20.0%
Median coverage
48.2%
Highest coverage
69.8%
Average coverage
46.8%
The same HSK 4 vocabulary, 69.8% of Film and TV subtitles (Spoken / dialogue) but only 20.0% of Classical Chinese (wenyanwen) (Classical literary), a spread of 49.8% points depending only on which kind of text you read. Spoken dialogue leans on a small set of very frequent words; written and older registers spread probability across far more vocabulary and proper nouns the HSK list does not include.
Full data table
| Item | Cost | Source |
|---|---|---|
| Film and TV subtitles (Spoken / dialogue) | 69.8% | hermitdave/FrequencyWords (content/2018/zh_cn/zh_cn_full.txt) |
| News and web journalism (Journalistic / written-web) The Leipzig news crawl mixes news portals (Sing Tao, NYTimes Chinese, Nikkei Chinese) with forum text (bbs.voc.com.cn), so this register is online journalism plus discussion, not print-only news. | 54.4% | Leipzig Corpora Collection (Wortschatz, Universitaet Leipzig) |
| Encyclopedic (Wikipedia) (Expository / reference) | 48.2% | Leipzig Corpora Collection (Wortschatz, Universitaet Leipzig) |
| Classic literature (Ming-Qing vernacular fiction) (Literary narrative) Classic (Ming-Qing) vernacular fiction, not contemporary novels. Character names and archaic vocabulary are outside the HSK list, so HSK coverage of literary prose is genuinely lower than of speech. | 41.7% | Project Gutenberg (public domain) |
| Classical Chinese (wenyanwen) (Classical literary) Classical Chinese uses a different lexicon and grammar from the modern HSK vocabulary. The low coverage figure is the point: a modern HSK vocabulary does not unlock classical texts, and even this figure overstates comprehension because shared characters carry different classical meanings. | 20.0% | Project Gutenberg (public domain) |
Limitations
- Token coverage is not comprehension: recognising a large share of the words in a text is not the same as understanding every sentence, especially where grammar or a low-frequency content word carries the meaning.
- HSK ordering is not identical to any single corpus's own frequency ranking, and proper nouns are never part of the HSK list, so this is coverage BY the HSK vocabulary, not the best-possible coverage from that many words.
- The Leipzig news and encyclopedic corpora are licensed CC BY-NC 4.0; only the derived coverage percentages are published here, never the corpora themselves.
- The classical-Chinese segmentation is heuristic (jieba is trained on modern Chinese), so its match rate is the lowest of the five registers; the low coverage figure it reports is still the headline finding, not an artifact to explain away.
Update history
- - Film and TV subtitles (Spoken / dialogue) coverage computed against hermitdave/FrequencyWords (content/2018/zh_cn/zh_cn_full.txt)
- - News and web journalism (Journalistic / written-web) coverage computed against Leipzig Corpora Collection (Wortschatz, Universitaet Leipzig)
- - Encyclopedic (Wikipedia) (Expository / reference) coverage computed against Leipzig Corpora Collection (Wortschatz, Universitaet Leipzig)
- - Classic literature (Ming-Qing vernacular fiction) (Literary narrative) coverage computed against Project Gutenberg (public domain)
- - Classical Chinese (wenyanwen) (Classical literary) coverage computed against Project Gutenberg (public domain)
Read the methodology behind this data, or see the full per-level coverage-by-genre breakdown this study summarizes.