What POS tagging does
Part of speech tagging labels each word with its grammatical category: noun, verb, adjective, determiner, and so on. It is a building block for parsing, NER, and many downstream tasks.
Why context matters
The same surface word takes different tags in different sentences.
- In I book a flight, book is a verb.
- In I read a book, book is a noun.
So a tagger cannot just look up a word in a dictionary; it must read the surrounding words.
Classic and modern approaches
- Hidden Markov models treat tags as hidden states and words as emissions, decoded with the Viterbi algorithm.
- Neural taggers encode the sentence and predict a tag per token, capturing long context.
- Standard tag sets include the Penn Treebank tags and the cross language Universal POS tags.
How it is evaluated
Accuracy per token is the usual metric, and strong models exceed ninety seven percent on clean English news text. Errors cluster on rare words and ambiguous function words.
Key idea
POS tagging assigns a grammatical category to each word using sentence context, since one word form can be a noun in one sentence and a verb in another.