This post is about Serre duality for smooth, proper, Deligne-Mumford stacks \cX over a field k, which came up recently in a phone conversation with Max Lieblich (but please don’t blame him for what I am writing here). Disclaimer: I haven’t yet had time to think carefully about cohomology on algebraic stacks, so what I say in this post may be completely wrong or besides the point! Moreover, it is also very likely that you (= the reader) have told me a vastly more general and superior theorem and I am repeating it here in some kind of warped way. Please let me know if so.

What I want — and it is quite possible that I shouldn’t want this — is for every locally free sheaf E with dual E^* on a smooth proper \cX over k and every integer i a nondegerate k-bilinear pairing

- H^i(\cX, E) x H^{-i}(\cX, \omega^*_\cX \otimes_{O_\cX} E^*) —> k

Here \omega^*_\cX is (maybe, see below) an object of the derived category D(\cX) of \cX and the pairing should come from a map

- Tr_\cX : H^0(\cX, \omega^*_\cX) —> k

via the cuproduct as usual. The complex \omega^*_\cX and the pairings should have more properties, but let’s ignore this for now.

Here is an example: Consider \cX = B(G) over a field k of characteristic p where G is a cyclic group of order p. Then we see that H^i(B(G), O_{B(G)}) = H^i(G, k) is zero for i < 0, but nonzero in all degrees i = 0, 1, 2, … Thus we see that the complex \omega^*_{B(G)} cannot be contained in D^{+}(X) since if it were then its cohomology groups H^{-i}(B(G), \omega^*_{B(G)}) would be zero for all sufficiently positive i! This is really the main point I wanted to make, and maybe you should stop reading now and have a beer instead (or tea).

Let me explain what I think \omega^*_{B(G)} is in case G = Z/2Z and the characteristic of k is equal to 2. In this case k[G] = k[e] with e^2 = 0. In this case the category of quasi-coherent O_{B(G)}-modules is equivalent to the category of k[e]-modules, the tensor product of O_{B(g)}-modules corresponds to tensoring over k(!), and H^0 corresponds to taking the kernel of e. An injective resolution of k is the complex

- k[e] — e –> k[e] — e –> k[e] — e –> …

and it is clear that if you take the kernel of e, then you get k in each nonnegative degree with zero maps. I think that \omega^* is the “k-linear dual” of this complex. But we have to be careful when we do this because we are working with unbounded complexes. Since my brain doesn’t appear to be functioning very well right now, let me just try to say what I am thinking (and you can leave a comment if you think this is wrong). I want to think of the infinite complex above as the limit of the complexes L_n^* which are the stupid truncations of the complex above in degrees [0, n]. Then I say that

- \omega^* = colim_n Hom_k(L_n^*, k)

for some notion of colimit of complexes. Why does this work? Well, I’m not sure it does, but I checked that it works for the two interesting modules I can compute the result for in this case, namely E = k and E = k[e]. Note that both modules are selfdual so it is easy to see what you get on both sides.

Presumably, the correct thing to do is to take the homotopy colimit or something in the formula for \omega^* above. But I think a nice way to think about it is that \omega^* simply isn’t a complex, but a system of complexes. The next thing to try would be to look at a case where \cX is a global quotient \cX = [X/G] for some smooth proper X over k. Note that \cX —> B(G) is a smooth proper morphisms. Hence in this case we can presumably let \omega^*_\cX be the tensor product of the pullback of the system \omega^*_{B(G)} just constructed and the usual dualizing sheaf of X placed in degree -dim(X). Right?

[Edit 21:15: Replaced limit by colimit and vice versa, as per the comment of Bhargav below.]

[Edit March 10, 2011: See next post for a bit more of the underlying algebra.]

I am probably being very stupid here, but don’t the maps go the “wrong way” for the stupid truncations to define a colimit of a non-negatively supported complex?

OK, you are right. To fix this I switched limit and colimit. For some reason I think using the stupid truncations is the correct thing to do and using the canonical truncations isn’t. The confusing issue is that taking the actual colimit of the complexes (as it is written now) gives a complex which is quasi-isomorphic to k placed in degree zero and this is NOT what we want. But each of the complexes Hom(L_n^*, k) is a bounded complex of free k[e]-modules and its cohomology is really equal to k in degrees -n, …, 0 which is what we want. If we use the canonical truncations this doesn’t work…

If I don’t misunderstand things horribly, you are looking at ind-coherent sheaves in the sense of Gaitsgory. I will think about this a little more after lunch.