We consider orthogonal connections with arbitrary torsion on compact Riemannian manifolds. For the induced Dirac operators, twisted Dirac operators and Dirac operators of Chamseddine-Connes type we compute the spectral action. In addition to the Einstein-Hilbert action and the bosonic part of the Standard Model Lagrangian we find the Holst term from Loop Quantum Gravity, a coupling of the Holst term to the scalar curvature and a prediction for the value of the Barbero-Immirzi parameter.