TF-IDF Calculator
SEOs and content strategists reach for this calculator when a draft has to stand next to four competitor pages and show which words each one actually owns. Paste two to ten documents, one per URL you are studying, and every term gets scored by how often it repeats inside a single document against how rarely it appears across the whole set. Raw keyword density only reports per-page totals, so a word repeated on all five pages looks important while saying nothing about any one of them. TF-IDF inverts that: terms concentrated in a single document climb, terms shared by the set sink.
| Term | df | idf | Total tf | Score |
|---|
How It Works
Tokenizing lowercases each document and splits on anything outside letters, digits, and apostrophes, so don’t stays one token while hyphenated compounds break into pieces. With stopword removal on, tokens matching the built-in list of about 150 English function words are skipped and counted separately in the stopwords removed figure. Document frequency records how many of the pasted documents contain a term at least once, and inverse document frequency follows ln(1 + N/df), a smoothed form that stays positive even when a term appears in every document. Term frequency takes one of three shapes: raw count, sublinear 1 + ln(count), or augmented 0.5 + 0.5 × count/max. Each weighted tf then multiplies the shared idf, giving one cell of the score matrix per document. The vocabulary table sorts terms by the sum of those cells across the whole set, while each per-document list sorts its own column and keeps the top N entries. The CSV export writes term, df, idf, then one score column per document, ready to drop into a spreadsheet.
Before and After
corpus:
"alpha alpha alpha common beta"
"alpha common common gamma"
"common delta epsilon"
docCount 3 vocabularySize 6 totalTokens 11
term df idf totalTf scoreSum
alpha 2 0.9163 4 3.6652
common 3 0.6931 4 2.7726
beta 1 1.3863 1 1.3863
delta 1 1.3863 1 1.3863
epsilon 1 1.3863 1 1.3863
gamma 1 1.3863 1 1.3863
doc 3 top: delta 1.3863, epsilon 1.3863, common 0.6931
alpha has df 2, so idf 0.9163 sits under the df 1 termscorpus:
"the cat and the hat"
"cat hat"
stopwords kept:
vocabularySize 4
terms: and, cat, hat, the
stopwords removed:
vocabularySize 2 stopwordsRemoved 3
terms: cat, hat
"the" and "and" leave the vocabularyUse Cases
| Developer | Production | Workflow |
|---|---|---|
| Demoing tf-idf on a five-line sample corpus | Comparing your draft against three ranking pages | Turning top terms into a content brief outline |
| Feeding the CSV matrix into a scoring script | Tagging tickets or feeds by distinctive vocabulary | Finding underserved subtopics a draft never covers |
| Checking tokenizer options before shipping a pipeline | Rescoring a page set after a content refresh | Handing the term grid to an editor for review |
Common Mistakes and Troubleshooting
- Analyzing a single document. Every term then shares one idf value, so the ranking collapses into a plain word count; paste at least two texts before reading scores.
- Expecting production-grade language processing. The tokenizer is a simple split on non-alphanumerics with no stemming or lemmatization, so ranking and rankings remain separate terms.
- Pasting ten-word snippets. Sparse inputs yield a handful of terms whose scores swing on one word, which makes any cross-document claim hard to defend.
- Comparing raw scores across tf variants. Switching from raw to sublinear rescales every cell, so judge ranks within a single run rather than numbers between runs.
- Leaving stopword removal on for foreign-language text. The list covers English only, so uncheck the box and let other languages keep their function words.
- Pressing Download CSV before an analysis. The button stays disabled until a run succeeds, and editing inputs afterward leaves the previous export in place.
Best Practices
- Use one document per ranking URL when the goal is a gap analysis against a live result set.
- Keep each pasted document near 200 words or more so counts settle and single sentences stop skewing scores.
- Pair the top-N table with the keyword clustering tool when a brief needs expansion into themed sections.
- Export the CSV into a sheet and sort one document column at a time to build a term-by-page grid for editors.
- Re-run the analysis after every draft revision to watch which competitor terms your draft picks up.
- Combine the results with the keyword density checker for per-page totals, since TF-IDF scores set membership rather than a single page’s share.
Frequently Asked Questions
How is TF-IDF different from keyword density?
Density divides a term’s count by the word total of one page, so repeating a word always lifts that page’s density on its own. TF-IDF keeps a frequency half but multiplies it by how rare the term is across the documents you pasted. A word sitting on every page collects a low idf and a low final score even when its per-page count is high. The table therefore surfaces terms distinctive to each document instead of terms that are merely frequent.
How is inverse document frequency computed here?
For every term the tool counts how many of the N pasted documents contain it at least once, and that count is the document frequency. The formula on this page is the smoothed ln(1 + N/df), which keeps a term present in all documents above zero instead of dropping it to the 0 that the textbook ln(N/df) produces. Smoothing also keeps scores stable when you add another document to the set. df, idf, and the resulting scores appear side by side in the results table.
Why is stopword removal turned on by default?
Small function words (the, and, is) appear in almost every English sentence, so left in they pile up counts without pointing at any topic. The built-in list holds roughly 150 of these words, and matching tokens are skipped while still feeding the stopwords removed tally. Uncheck the box when you want full counts, which helps with stylometry or corpora full of names that happen to collide with the list. The toggle only changes which tokens enter the math.
How many documents should I paste?
Two to ten texts is a workable range, and the common setup is one document per ranking URL for the query you care about. With only one document every term shares the same idf, so the ranking collapses back into a plain word count. Terms repeating across several documents stay low while terms confined to a single page rise, which is where the comparison gets useful. Very short inputs still run, but tiny documents produce sparse tables.
When should I use sublinear or augmented tf?
Raw count multiplies the idf by the plain occurrence number, so a term repeated thirty times outscores one repeated three times by a factor of ten. Sublinear applies 1 + ln(count): repetition still helps, but the jump from three to thirty is compressed and one long paragraph stops taking over the table. Augmented scales counts between 0.5 and 1 relative to the most frequent term in that document, which suits short texts with one outlier word. Switch variants when a single repeated term dominates every score.
How do these results feed a content brief?
Each per-document list shows the terms that page already owns, so a term scoring high on competitors but missing from your draft marks a section worth writing. The vocabulary table ranks the whole set, giving a short candidate list for headings and subheadings. Download the CSV, open it in a spreadsheet, and sort by one document column at a time to build a term-by-page grid for editors. Re-running after each draft revision shows which gaps closed.
How does this differ from the keyword clustering tool?
The clustering tool groups a keyword list into themed buckets by co-occurrence, which helps expand seed terms before any writing starts. This calculator scores a vocabulary you provide by document frequency across pasted texts, so it works best after drafts or competitor pages exist. A typical workflow runs clustering first to plan sections, then TF-IDF scoring on the published set to see which terms separate the pages. The two answers differ: grouping versus differentiation.