Here are two definitions as currently in the stacks project:
- A Serre subcategory of an abelian category is a strictly full subcategory closed under taking subquotients and closed under taking extensions.
- A weak Serre subcategory of an abelian category is a strictly full subcategory which is abelian, which has an exact inclusion functor, and which is closed under taking extensions.
Here the subquotients and extensions are taken in the bigger abelian category. The formal definitions can be found here.
Yesterday I realized I had confused these two notions. In some situations the first is more appropriate (e.g., the kernel of an exact functor is a Serre subcategory) and in others the second is better (e.g., given a weak Serre subcategory B of A the derived category D_B(A) makes sense).
Nomenclature: I think the notion of a Serre subcategory is pretty standard, in the sense that all of the definitions of a Serre subcategory of an abelian category that I have seen are equivalent to the one above (single exception: nlab). Serre used the same definition (in the case that the ambient category is the category of abelian groups). On the other hand, the notion of a “weak Serre subcategory” is nonstandard. In some papers/books the terminology “thick subcategory” is used for this, but unfortunately in many texts “thick subcategory” is synonymous with “Serre subcategory”. In fact, it seems that the notion of a “thick subcategory” is very malleable — there is no real agreement on what this term should mean, and, googling, I found at least one instance where this confusion led to a mathematical error. In the case of subcategories of a triangulated categories I decided to avoid using “thick” and I have used “saturated” just like Verdier does in his thesis. (Unfortunately, some authors use “saturated” to mean “closed under isomorphism”, but they seem in the minority.)
Is there a word, other than “thick”, we can use to describe weak Serre subcategories?