High cycle fatigue in polycrystals is mostly governed by deterministic laws such as crystal plasticity, but also depends on probabilistic properties, such as random defects and crystallographic and morphological textures, which result in significant scatter of fatigue lifetime at the macroscopic scale. Thus, modeling fatigue phenomena so that the probabilistic density function of failure is anticipated, would be useful especially for very high cycle fatigue involving up to 10^9 cycles. To do so, the grain structure with crystal orientations should be considered in full field computations, which usually involve prohibitive computation cost therefore hindering numerical exploration of statistical distribution of fatigue failures. This paper therefore consists in developing a very fast full field mesoscopic model of polycrystals subjected to crystal plasticity during cyclic loading based on energy minimization techniques. As a result, the uniform plastic slip in each grain is obtained in the form of a relatively simple recursive formula, which guarantees short computation time even for very high cycle fatigue. The proposed approach has been validated against a classical crystal plasticity finite element model in 2D, and satisfying agreement is observed. In addition the model has be applied in combination with classical fatigue criteria to rapidly compute the fatigue lifetime and then derive probabilistic S-N curves, hence creating a substantial link between crystallographic and morphological textures on the one hand, and fatigue lifetime estimations on the other hand.