network sparsity, monosemanticity and expert specialization in MoEs

understanding experts in a MoEs through routing analysis and logit lens

features, neurons, polysemanticity and superposition

embeddings, attention and transformers architecture math

backpropagation, regularization, cross-entropy loss and more