Title: RowNet: A Memory Transformer for Tabular Regression

Abstract: Determining real estate value constitutes a complex structured regression challenge, driven by diverse feature categories, sparse regional influences, nonlinear interdependencies, and the practical necessity of identifying comparable properties. Conventional multilayer perceptrons (MLPs) process each data row as an independent vector, requiring them to infer locality, sensitivity to scale, and categorical alignment solely through supervised learning. While gradient-boosted decision trees offer robust baselines for tabular data, their splitting mechanisms focus on individual features and fail to explicitly capture the retrieval of analogous historical records. To address this, we introduce RowNet, a neural architecture grounded in retrieval methods designed for predicting price per square meter in real estate. RowNet encodes a query property by calculating pairwise similarity metrics against a memory bank of previously labeled assets. The model employs a two-stage retrieval process: the initial layer generates a coarse target estimate based exclusively on feature similarities, while the subsequent layer refines this by incorporating target-consistency features and leveraging multiple learned attention heads to identify complementary sets of comparable properties. The final prediction is synthesized by a mixture-of-experts module that integrates learned gating mechanisms, residual corrections, and regularization techniques targeting both entropy and head diversity.

Source: arXiv Generated at: 2026-06-04 00:00:00 UTC