How Far Do Simple Transformations Translate Across Text Embedding Models?

TL;DR - An arXiv study tests whether lightweight (e.g. linear) translators can map text embeddings between independently trained models, and finds that "latent universality" holds only for some model pairs, not universally. This matters for AI-to-AI latent communication that bypasses decoding to human-readable text.

• Evaluated nine text embedding models differing in architecture, pooling strategy, and training objective, in a realistic text setting rather than simplified benchmarks.
• Compatibility was measured on four axes: CKA similarity, downstream transfer, fidelity, and retrieval performance.
• Simple translators recovered meaningful shared structure and supported transfer for compatible pairs, but failed sharply for others.
• Compatibility depends jointly on architecture, training objective, pooling, and data distribution — contradicting broad claims of universal linear relatability between heterogeneous embedding spaces.