Hybrid High-Order (HHO) methods were introduced in (DiPietro & Ern & Lemaire 2014) as a new method in the class of Discontinuous Skeletal discretizations. The main features of HHO are the approximation of the solution with arbitrary order polynomials, support for fully polytopal meshes and easy $hp$-refinement. The HHO unknowns are placed both in the cells and on the faces of the mesh, in order to approximate a pair including the primal variable in the cells and its trace on the skeleton. In particular, these unknowns are used (i) by a reconstruction operator, which reconstructs a high-order field in the cell and (ii) by a stabilization operator, which weakly enforces in each mesh cell the matching of the traces of the cell functions with the face unknowns. These two operators are then combined in a local bilinear form which, after local static condensation, is assembled into a global problem posed only on the face unknowns. In recent years, the study of Galerkin methods suitable for polytopal meshes that avoid the use of a non-physical stabilization term has attracted the interest of the scientific community. In the context of HHO, stabilization-free variants exist and are based on the use of spaces from the Raviart-Thomas (RT) family. In particular, local RT-N\'ed\'elec spaces on simplicial submeshes were exploited in (DiPietro &Droniou &Manzini 2018). In addition, in (Abbas & Ern & Pignet 2019) RT spaces on simplicial meshes were employed, though optimal convergence was not attained. In this work, we introduce and study numerically a variant of HHO for elliptic problems, posed on 2D and 3D polytopal domains and free of an explicit stabilization term. The proposed variant preserves optimal HHO convergence rates and is based on introducing modified reconstruction operators mapping on polynomial spaces richer than the ones used in standard HHO methods.