What is TF-IDF… Can it read documents with just one word?

Is “apple” really an important word?

When reading a document, we quickly grasp the gist of the text by looking at the title or the first sentence. Computers do something similar: they look at words to determine what a document is about. However, not all words are equally important.

For example, in the sentence “I want to eat a banana,” the word “banana” feels more important than words like “I” or “want,” right?

There’s a technology that helps computers understand this too. That’s TF-IDF.

🍌 In simple terms, TF-IDF is a “word importance calculator.”

TF-IDF stands for Term Frequency - Inverse Document Frequency. It sounds complicated, but here's a simple explanation:

“Words that appear frequently, but only in this document, are the most important!”

If a word appears multiple times in a document → It is considered important in that document

But if that word appears in multiple documents? → It's just a common word and not considered important

🍎 Learning TF, DF, and IDF with examples

▶ TF (Term Frequency)

This looks at how many times a word appears in a document.

Document Content
Document 1 I want to eat an apple.
Document 2 I want to eat a banana.

Here, “eat” appears once in both documents, right?
“Banana” only appears in Document 2. Therefore, “banana” could be a more important word in Document 2.

▶ DF (Document Frequency)

This measures how many documents a word appears in.
“Eat” appears in both Document 1 and Document 2, so DF = 2.
“Banana” only appears in Document 2, so DF = 1.

▶ IDF (Inverse Document Frequency)

IDF is the opposite of DF!
Words that appear in many documents are common, so they are less important.
Words that appear in only a few documents are rare, so they are more important!

The IDF formula looks like this, but don't worry too much. We just need to interpret it.

IDF(t) = log(total number of documents / (number of documents containing the word + 1))
For example, if there are 4 documents in total and “apple” appears in only 1 document:

IDF = log(4 / (1 + 1)) = log(2) ≈ 0.693

🍇 So what is TF-IDF?

Now we're all set.
TF × IDF = TF-IDF

In other words, we multiply how often a word appears in a document (TF) by how rare the word is (IDF)

to find the truly “important words.”

🥝 Real-world example: Is banana really important?

The table below shows the number of times each word appears in each document (TF). (This is also called DTM.)

• Document 1
  Banana: 0
  Eat: 1
• Document 2
  Banana: 1
  Eat: 1
• Document 3
  Banana: 2
  Eat: 1

Now, let's calculate TF-IDF by multiplying the IDF values.

In Document 2, “banana” appears once, so TF = 1.
In Document 3, it appears twice, so TF = 2.

Let's assume that IDF is 0.287682 in both cases.

So:

Document 2: 1 × 0.287682 = 0.287682
Document 3: 2 × 0.287682 = 0.575364

Even if it's the same word, Document 3, where it appears more often, is considered more “important”!

🤖 Where is TF-IDF used?

Search engines: When you search for “I want to eat an apple,” documents with high TF-IDF scores are displayed first.

• Document classification: When distinguishing whether a document is a “sports article” or “entertainment news”
• Chatbots: When extracting important keywords from user questions

And it's no exaggeration to say that almost all the basics AI uses to understand text these days start with TF-IDF.

🚀 In conclusion: TF-IDF is a technology that assigns scores to words!

You may feel like you've seen too much information, but just remember one thing.

TF-IDF is a technology that tells you “how special this word is” in numbers.

When analyzing documents or building AI models in the future, using TF-IDF can enable smarter analysis.
Now, we’re one step closer to creating AI that can “read” documents through TF-IDF!

💡 Preview of Next Content

“What’s Next After TF-IDF?” — Methods for Calculating Document Similarity
“Building Your Own Search Engine: Let’s Start with TF-IDF!”
“Beyond TF-IDF… How Does BERT, a Deep Learning-Based Model, Understand Words?”

Reference: https://wikidocs.net/31698