Discrete Geometry for Comparing and Transferring Neural Representations

Sammanfattning

Neural networks transform data through a sequence of intermediate representations,but comparing and transferring the structure of these representations remains challenging.This thesis develops a geometric framework for neural representations basedon manifold learning, representational similarity, and diffusion geometry, incorporatingtools from multi-view learning into this field for the first time. A key contributionis an exact Markov reformulation of a broad class of centered, scale-invariant RSMbasedsimilarity measures in terms of row-stochastic Markov matrices, which thenopens the door to manipulations from diffusion geometry.Building on this, the thesis introduces multi-scale variants of CKA and DistCorr,which compare powers of the associated Markov operators, and alternating-diffusionvariants, which fuse the Markov matrices of several layers into a single network-leveloperator. Empirically, these diffusion-based measures achieve state-of-the-art performancein accuracy and output correlation for both language and vision tasks acrossdifferent models, on the Representational Similarity (ReSi) benchmark. They alsoobtain the best results on an additional out-of-distribution challenge benchmark.The thesis further applies the geometric viewpoint to knowledge distillation. Agraph-Laplacian distillation objective is proposed, in which the student is trained tomatch the teacher’s sample geometry rather than activation coordinates. Together,these results show that operator-based discrete geometry provides a useful languagefor comparing, aggregating, and transferring neural representations.Keywords: