When single features fall short
A ranker may know a user likes sports and that an item is about cooking, but the value lies in the combination: does this user like this item. Feature crossing builds combined features that capture interactions a linear model cannot represent from raw inputs alone.
Explicit crosses
- A cross feature multiplies or concatenates two raw features, like user country times item language.
- Crosses turn an additive model into one that can express conditional preferences.
- Hand crafted crosses encode domain knowledge but require effort and can explode in number.
Learned crosses
- Factorization machines learn pairwise interactions efficiently by sharing low dimensional vectors across feature pairs.
- Deep cross networks stack layers that generate higher order crosses automatically.
- Embeddings of categorical IDs let the network discover useful combinations during training.
The wide and deep pattern
A popular design pairs a wide part holding memorized explicit crosses with a deep part that generalizes to unseen combinations. The wide side nails frequent patterns; the deep side handles the long tail.
Watching for blowup
Crossing high cardinality features can create enormous sparse spaces. Hashing and embedding tame the dimensionality while keeping the interaction signal.
Key idea
Feature crossing lets rankers model interactions between features, combining explicit memorized crosses with learned ones so the model expresses conditional preferences raw features cannot.