Quantum mechanics and representation theory are very closely linked subjects since the Hilbert space of a quantum system with symmetry group G carries a unitary representation of G. To the extent that one has a way of quantizing a classical Hamiltonian system with G-symmetry, one has a way of constructing representations of G out of symplectic manifolds with G-action. This “geometric quantization” approach to constructing representations has been a very fruitful one.
For the case of G compact, connected, with maximal torus T (the crucial example to keep in mind is G=SU(2), T=U(1)), the “flag manifold” G/T (the 2-sphere for G=SU(2)) is a symplectic manifold (actually Kahler) and can be thought of as a classical phase space with G-symmetry. Choosing a representation of T (a “weight”) allows one to construct a line bundle over G/T, which turns out to be holomorphic. The Borel-Weil theorem says that irreducible G-representations are given by holomorphic sections of this line bundle, for “dominant” weights.
For weights that are not dominant, one gets not holomorphic sections, but elements in higher cohomology groups. These can be expressed either in terms of the sheaf cohomology of G/T with coefficients in the sheaf of holomorphic sections of the line bundle, or in terms of Lie algebra cohomology. This is known as the Borel-Weil-Bott theorem, which first appeared in:
Bott, R., Homogeneous Vector Bundles, Ann. of Math. 66 (1967) 203.
the Lie algebra version was further developed by Kostant in
Kostant, B., Lie Algebra Cohomology and the Generalized Borel-Weil Theorem, Ann. of Math. 74 (1961) 329.
Instead of using complex manifold methods and the Dolbeaut operator to construct cohomology classes, one can use spinors and the Dirac operator, with the representation appearing as the kernel of the Dirac operator (or, more accurately, its index). For this point of view, which fits in beautifully with equivariant K-theory and the index theorem, see:
Bott, R., The Index Theorem for Homogeneous Differential Operators, in: Differential and Combinatorial Topology: A Symposium in honor of Marston Morse, Princeton (1964) 71.
The Dirac operator approach to representation theory has been extended to some cases of G non-compact by various authors. In the last few years, Kostant has come up with a new version of the Dirac operator in this context which has quite interesting properties. He likes to work algebraically, so his Dirac operator on G is given as an element of U(Lie G)XCliff(Lie G), where U(g) is the universal enveloping algebra of the Lie algebra Lie G and Cliff(Lie G) is the Clifford algebra of Lie G. The Kostant Dirac operator is the standard one you would expect, with the addition of an extra cubic term. For the details of all this, see Kostant’s paper:
Kostant, B. , A Cubic Dirac Operator and the Emergence of Euler Number Multiplets of Representations for Equal Rank Subgroups, Duke Math. J. 100 (1999) 447.
Things get interesting when you consider the case of H a subgroup of G of the same rank (one example is H=T, another important one is G=S0(2n+1), H=SO(2n), where G/H is an even-dimensional sphere). Taking the difference of Kostant Dirac operators for G and H gives something that corresponds to a Dirac operator on G/H, which acts on the product of a G rep with the spinors associated to Cliff (Lie G/Lie H). For H=T, one gets back the old Bott-Kostant construction of representations, but with the Lie algebra cohomology replace by the index of a Dirac operator.
Part of this story is that one finds that, starting with an irreducible G-representation, the kernel of Kostant’s Dirac operator consists of a “multiplet” of H representations of size given by the Euler characteristic of G/H. The existence of these multipliets was first noticed by Ramond for the case H=SO(9), where SO(9) is the massless little group in 11 dimensions and the multiplets appear in the massless spectrum of N=1 11d-supergravity (the low energy limit of a conjectural M-theory). These SO(9) multiplets come about because SO(9) is an equal rank subgroup of the exceptional group F4, so for each irreducible F4 representation one gets a multiplet of SO(9) representations.
The first paper about this was by Gross, Kostant, Ramond and Sternberg, for more about this from a geometrical point of view, see a paper by Greg Landweber. For a discussion of the relation of this to supersymmetric models in physics, look up recent preprints by Pierre Ramond, one of which is by Brink and Ramond.
Greg Landweber has applied these ideas to loop groups, getting a beautiful interpretation in terms of loop group representation theory of certain N=2 superconformal models first studied by Kazama and Suzuki in 1989. This paper also contains a detailed exposition of the story both for finite dimensional groups and loop groups.
More recently, Freed, Hopkins and Teleman have used a modified version of the Kostant Dirac operator to give a proof of their theorem relating the Verlinde algebra and twisted K-theory. Their construction is quite beautiful and gives a new point of view on the whole story of the relation of geometric methods of quantization to K-theory and the index of Dirac operators. I’ll try and write something about this at some later date.