Let X be a smooth projective variety fibered by genus one curves. Physical heuristics imply that the Gromov-Witten generating functions for X should be modular forms. We prove this in several cases, using the Hodge theory of elliptic surfaces and the cohomological theta correspondence to extract enumerative counts of smooth rational curves over lines in the base.