A conforming finite element discretization of the Poisson problem with a pure Neumann boundary condition produces an algebraic problem with a singular stiffness matrix. Typically, the singularity is removed by specifying the solution at some point or by perturbing the Neumann condition to a Robin type condition. Here we propose and analyze a simple and efficient least-squares penalization method that ensures nonsingularity of the alegbraic problem and at the same time appears to minimize the condition number of the matrix. Numerical results illustrating our method are also included.