In recent years, the nanoindentation test has been used to determine the parameters of the constitutive equation of metallic materials without much success. The variability in the nanoindentation test data, and a lack of repeatability of the test, make difficult to find a single set of parameters describing the mechanical behavior of the material. In this context, a research was emerged to differentiate the dispersion sources related to the specimen preparation, from the intrinsic heterogeneity of the material. To accomplish this aim, a series of nanoindentation tests was conducted on a specimen of 316L austenitic stainless steel, using both sharp and spherical indenters at different loads and different depths. These tests were accompanied by microstructural and topological characterizations of the imprint. The surface of the specimen was prepared in three different ways: manual polishing, vibratory polishing and electrolytic polishing. Each surface was carefully characterized by measuring residual stresses, roughness and hardening. Then, a comprehensive study of all parameters measured was performed. The results show that in the early stages, the dispersion is originated by the mechanical cutting process applied in the preparation of the specimen. And in the subsequent stages, the dispersion is linked to the microstructural heterogeneity of the material. From these results, an original methodology is proposed to avoid the problems related to the specimen preparation, and to obtain a nanoindentation data able to describe the local mechanical behavior of the material with an acceptable accuracy.