NoLBERT: A No Lookahead(back) Foundational Language Model for Empirical Research
Published: Sep 1, 2025
Last Updated: Sep 1, 2025
Authors:Ali Kakhbod, Peiyao Li
Abstract
We present NoLBERT, a lightweight, timestamped foundational language model for empirical research in social sciences, particularly in economics and finance. By pre-training exclusively on 1976-1995 text, NoLBERT avoids both lookback and lookahead biases that can undermine econometric inference. It exceeds domain-specific baselines on NLP benchmarks while maintaining temporal consistency. Applied to patent texts, NoLBERT enables the construction of firm-level innovation networks and shows that gains in innovation centrality predict higher long-run profit growth.