Visualizing Changes in Meaning

By Eric Bell,Natural Language Processing Software Scientist at Pacific Northwest National Laboratory and BBI Board Member, Dr. Seuss, Jeggings, and Dill?

“So the writer who breeds
more words than he needs
is making a chore
for the reader who reads.”

-Dr. Seuss

Humans are social creatures, and social implies communication.  We communicate in many forms: words, pictures, gestures, etc., but language, and words, are at the center of all communication.

From Dr. Seuss to William Shakespeare, humans have been making up and playing with words for centuries.

Words can change meaning over long periods of time. For example, the word “awful”, used to mean worthy of awe. Other words change meaning in a matter of moments. Would your definition of “computer” include your phone?

While today’s millennial may be chillaxing in their jeggings, unfriending vestiges of a failed bromance, computers around the world are racing to understand what all of the new words in the first part of this sentence mean.  If computers are going to understand humans, they must understand what words mean, and that includes when words change their meaning.

Researchers at Pacific Northwest National Laboratory have been examining this phenomenon, known in academic circles as semantic shift.

Words, like technology, are evolving faster than before. Communication in modern society is characterized by an always-on, constant barrage of global conversations.  This is compounded by the recent transition from face-to-face and point-to-point telephone conversations to multiple consumer communications in the rise of social media and mobile messaging platforms.

These changes in communication styles have given rise to a number of phenomena including emojis, live video streaming from around the globe, and a world of shopping with reviews at your fingertips.

The approach to understanding the meaning of a new word was traditionally accomplished by examining its relationship or shared grammatical constructions with known words. This approach is called statistical semantics. For example, from the phrase “wear jeggings to work” a computer can infer that jeggings are most likely a piece of clothing, because a clothing-related word often follows the verb “wear”.  When this sort of reasoning is applied over billions of social objects every day, new words can quickly be assigned some meaning.

As that meaning slowly changes with the adoption of words and the natural process of semantic shifts, a backward look in time at a word’s meaning today can help explain its origins and historical relevance.

At PNNL, scientists have been exploring the use of new terms in a variety of social data platforms, focusing on techniques from statistical semantics that aren’t platform or language dependent.

We’ve taken a technique called ‘temporal embeddings’ and combined that with novel visualization techniques to produce visual summaries of a word’s semantic shift, focusing on some words that have rapidly changed meaning in recent events.  Below we present a visualization of the term ‘ukrop’ which during the Crimean Crisis went from being a term describing the spice dill to an insult for Ukrainians. The path illustrates how the meaning of ‘ukrop’ meanders through a concept space in a relatively short period of time.


Here’s another example of how the word ‘fire’ shifted from ‘a natural disaster’ to ‘gunfire’.



Both of these examples are based on examination of social data during the Crimean Crisis, gathered from VKontakte, Europe’s largest social networking service.

Like words themselves, our process is evolving. And we want you to help. Do these images help explain the origin of a term?   These depictions are snapshots of an interactive system that lets you look at multiple words in the same concept space or representation of word meanings.   We invite the industry to help take the next step, the exploration of a similar phenomenon in the significance and meaning of visual social content.

So you readers who read
more words than we need,
fear not these new words
computers can help explain the misdeed.