Kanji sets

First of all, the results will differ significantly depending on which kanji set (our universe) is analyzed. We’ll investigate two such sets:

1. The set of kanji taught in Japanese education, today called Jōyō kanji. Since 1945, Japanese texts generally omit furigana readings for Jōyō, meaning that adult readers are expected to know by heart the readings of this set. Our Jōyō analyses are based on the full set of 2141 characters (including the 5 that were dropped in 2010, namely 勺 銑 脹 錘 匁 ).
2. The largest set I have available, that is, all kanji in the current version of KanjiVG—a bit more than 6300. I don’t know if there’s any selection criteria for this set, but I expect they’re some of the most common, and six thousand is a good estimate of the knowledge of a proficient, literary-minded native reader.

Quantifying phonetic series

Within each kanji set, the basic variables to relate are:

• A few thousand kanji, where
• each kanji is made of one or more components, and
• each kanji has zero or more (on-yomi) readings.

A component series is set of kanji that include a certain component. Here are some examples from the Jōyō set:

For our purposes, a phonetic series is a set of kanji that shares a component and a reading. If we add each kanji’s readings to the table above, interesting patterns appear:

First, consider the 走-series. Not a single kanji in it has a shared reading! 走 is not a phonetic component at all, i.e. the 走-series is not a phonetic series.

Compare to the 包-series. All the six kanji that include 包 are pronounced hou. In other words, 包-hou is a phonetic series of size 6.

Now consider the 乍-series. Almost there! Four out of five kanji include the reading saku, but 詐 breaks the pattern; it’s only read as sa. In this case the 乍-saku phonetic series (size 4, 作搾昨酢) is smaller than the full 乍 component series (5, 作搾昨詐酢). We say this phonetic series covers 4 out of 5 kanji, or that it has a kanji coverage of 4/5 = 80%.

What about the 青-series? It does include a phonetic series, with 100% coverage: all characters do have a shared reading, sei. However, most characters also have extra, unpredictable readings, so that this component is less predictive than 包, and we should measure this. Call readings coverage the ratio of predicted readings to all the readings of all kanji in which the component appears (that is, all kanji in the larger component series). 青-sei is a phonetic series of size 7, with 100% kanji coverage but only 43.75% (7/16) readings coverage.

I hope it’s clear by now that 及-kyuu is a size-4 phonetic series with 100% kanji coverage and 80% (4/5) readings coverage.

Phonetic series with 100% kanji and 100% readings coverage (like 包-hou) are especially useful; these ratings mean that, whenever the component appears, one can be sure of all readings of the kanji. We call these perfect series. Second in importante are those with 100% kanji but less than 100% readings (like 及-kyū and 青-sei); let’s name them semiperfect series. If you see a semiperfect phonetic component, you can be sure of at least one of the kanji’s readings. Series with less than 100% kanji coverage are not as useful, since you have to memorize the exceptions anyway; these are imperfect series.

Attentive readers might have noticed that a single component can be on many phonetic series; 乍, for example, could also be described as a very imperfect predictor for sa, working for 作詐 but not 搾昨酢. It of course performs better as a predictor for saku, since in that role it get 80/66% for 4 kanji, rather than 40/33% for just 2. We’re now in position to choose the best series for a component or a kanji: the rating criteria will be:

• First, higher kanji coverage;
• Second, larger size (more kanji);
• And third, higher readings coverage.

Perfect and semi-perfect series

Here’s a selection of the better phonetic series for Jōyō kanji, including all those with 100% kanji coverage and size 3 or larger:

Here’s a similar selection for KanjiVG kanji (including non-Jōyō), limited to 100% kanji coverage and size 5 or greater:

It’s clear that phonetic series grow in usefulness as we venture outside Jōyō. This is probably because, as mentioned earlier, rarer characters are more likely to have been built with a phonetic structure.

Notice 吾-go is kind of a spurious series, because the component 吾 itself includes a better phonetic component, 五-go:

Imperfect series

Here are some of the top few phonetic series with imperfect kanji coverage, for Jōyō kanji:

For the KanjiVG set, there are more than 600 series between 75% and 99% kanji coverage, many of them quite large. Here’s a sampling of some interesting ones:

When we open to a larger universe, some series can be demoted—like our friend 包-hou, the best perfect predictor in Jōyō, which here becomes imperfect thanks to 雹 (haku, hyou—one semivowel from hou!) and 萢 (no on-yomi). Even then, 包-hou still manages an impressive 23 out of 25—arguably even better than the 6 out of 6 in Jōyō.

The imperfect series with kanji coverage smaller than 50% (not pictured above) are quite bad; in fact, most of these “series” are just statistical noise, with sets of hundreds of characters where two or three share a reading by chance. For example, 含 and 岩, both read as gan, share the component 口; but so do 449 other Jōyō kanji, so that 口-gan has an abysmal kanji coverage of 0.44%.

Coverage of kanji sets: how many are in phonetic series?

It’s interesting to measure the coverage of Jōyō kanji—that is, how many of its kanji are included in a phonetic series. The charts below graph this for the Jōyō and KanjiVG sets, choosing the very best phonetic series for each kanji (click for larger version):

You can read these charts horizontally, as an one-dimensional percent bar. The vertical dimension adds information about kanji coverage and (in darker shading) readings coverage. The very bad section (kanji coverage less than 50%) is shaded in red, and is in practice the same as having no phonetic hint. The hints are actually useful basically for the 100% section (colored blue), and perhaps a few of the better imperfect series (orange).

However, the charts above are a bit misleading because they’re not graphing the size (number of kanji) of each series, only their relative coverages. Many of the perfect series have a modest size of 2—just enough to know, for example, that 愁 is pronounced the same as 秋. However, if the entire graphs were adjusted for size, we’d have to use a log scale, because of how bad the red section is—kanji outside the phonetic series would dominate the image in a sea of gray. To avoid that, this magnification shows only the left part (up to 50% kanji coverage), allowing us to represent the series size in a natural way:

(Notice that the scales are not the same between different graphics!)

456 Jōyō means 21.29% are perfectly phonetic, and 1411/6394 = 22.06% of our larger set (the yellow, imperfect part goes up to 34.46% and 49.69%). How to reconcile this with the often-cited figure that about 90% of kanji have a phonetic component? It’s simple: most of those components can only be detected through historical analysis, because either the sounds and graphs have changed too much, or else they now only appear in a single kanji. For example:

• In 盆 bon, the top element 分 is a phonetic for the sound bon, coming ultimately from Old Chinese *pən. However, due to diverging phonetic changes, its role as a phonetic in 分粉紛雰 became fun (recall that Modern Japanese f- or h- was Old Japanese p-); in 貧, hin; and, in 頒, han. So the phonetic structure of 盆 (“dish” 皿 + sound *pən = *bˤən “bowl”) became obscured and approximate.
• In 浮 fu “to float”, the right component 孚 is a phonetic element giving the sound fu (Old Chinese *mbu). It also appeared by itself as 孚 fu “confidence”, and in 俘 fu “prisoner”; however, these two characters are not used in modern writing anymore, so that 浮 “float” remained alone as the single example of the phonetic 孚-fu.

Downloads

Fuller data sets are available in tab-separated text files (TSV), compatible with any spreadsheet software and easy to manipulate with Unix tools.

components_phonetic.jouyou.tsv and components_phonetic.kanjivg.tsv:

The main tables, with one phonetic series by line, ordered by goodness as described above.

kanji_phonetic.jouyou.tsv and kanji_phonetic.kanjivg.tsv:

The best phonetic series for each kanji.

components.tsv:

Extracted from KanjiVG, a decomposition of each kanji into components.

component_variants.tsv:

Components that are considered to be “the same” in this experiment (for example, 人 and 亻, or 水 氵 氺). These are generally calligraphic abbreviations, marked as Z-variants or semantic variants in Unicode. This list is partial.

base_components.jouyou.tsv and base_components.kanjivg.tsv:

A list of all components needed to build the kanji set which are not themselves part of the set. Later I’ll add a new post on this.