Day 22 of 100 Days Coding Challenges: Python
I woke up today with a peculiar mission: build a text summarizer without relying on AI. Yes, I willingly traveled back in time—back to the days before ChatGPT could just nod and condense a thousand words into ten. Why? Curiosity, mostly. I wanted to know how the old-school extractive methods worked. Spoiler alert: they’re kind of charming in their own clunky way.
Did it work on the first try? Of course not. I wrestled with broken libraries, mysterious errors, and one particularly judgmental error message about “punkt_tab” that seemed to imply I should know what that is. I wrote one version, then another. Tried a different library. Threw a mini tantrum. Eventually, I ditched NLTK entirely, kicked open the spaCy door, and got a summarizer working. Is it perfect? Nope. Is it lovable in its quirky way? Absolutely.
Today’s Motivation / Challenge
You ever read something and think, “Wow, that could’ve been a tweet”? That’s the spirit behind today’s project. Summarizing text is a superpower—whether it’s for condensing long articles, making notes from journal entries, or just helping your brain cope with too much information. While AI can do it effortlessly now, building one from scratch is like learning to cook pasta before using a microwave: humbling but worthwhile.
Purpose of the Code (Object)
This code takes a chunk of text and extracts the most important sentences. Instead of “understanding” like AI, it just finds which words show up the most and grabs the sentences that use them. It’s a bit like highlighting the loudest parts of a conversation and calling it a summary—but hey, it kind of works.
AI Prompt:
“Write a Python program that summarizes text using classical NLP techniques. Do not use deep learning. Use word frequency and sentence scoring instead.”
Functions & Features
- Tokenizes text into words and sentences
- Removes common words like “the” and “is” (stopwords)
- Scores sentences based on word importance
- Extracts and returns the top N sentences as a summary
Requirements / Setup
pip install spacy
python -m spacy download en_core_web_sm
Minimal Code Sample
python
CopyEdit
doc = nlp(text)
sentences = list(doc.sents)
word_freq = {word.text.lower(): word_freq.get(word.text.lower(), 0) + 1 for word in doc if not word.is_stop and not word.is_punct}
summary = ‘ ‘.join([sent.text for sent in heapq.nlargest(3, sentences, key=lambda s: sum(word_freq.get(w.text.lower(), 0) for w in s))])
This grabs the top 3 most important sentences using word frequency as a guide.
Notes / Lessons Learned
Oh, the joys of punkt_tab. Today’s real adventure was less about Python logic and more about survival. The original plan was to use NLTK—but it had other ideas. It kept yelling at me about a missing “punkt_tab” resource, which sounded more like a 90s German punk band than a tokenizer. I redownloaded, wiped caches, and whispered sweet nothings to the command prompt. Nothing.
Eventually, I gave up and pulled in spaCy instead—and guess what? It just worked. Sometimes, letting go of the buggy route is the bravest choice. Along the way, I learned how to uninstall libraries cleanly, delete hidden cache folders, and even navigate the Windows file system like a seasoned hacker. It didn’t summarize exactly like a human or an LLM, but it got the job done. And now, I’m much more confident using Python from the command line. NotebookLM still wins at summarizing, but my little app gets a gold star for effort.
Optional Ideas for Expansion
- Let the user choose how many sentences they want in the summary
- Add a “read from file” button so you can summarize documents directly
- Highlight the summary sentences in the original text to show what was picked