Second Order Properties of Nearest Neighbor Regression with Uniform Weighting Function for Strongly Mixing Processes

Patrick Rakotomarolahy

doi:doi:10.11648/j.ajtas.20261501.12

Research Article |

| Peer-Reviewed

Second Order Properties of Nearest Neighbor Regression with Uniform Weighting Function for Strongly Mixing Processes

Patrick Rakotomarolahy^*

Published in American Journal of Theoretical and Applied Statistics (Volume 15, Issue 1)

Received: 24 November 2025 Accepted: 11 December 2025 Published: 16 January 2026

Views: Downloads:

Download PDF

Share This Article

Twitter
Linked In
Facebook

Abstract

We examine the statistical properties of nearest neighbor regression function estimation beyond the classical i.i.d assumption. Second-order properties of this estimator for uniformly mixing processes have been derived in previous studies. Nevertheless, uniform mixing is a very strong form of dependence that is difficult to achieve. In contrast, strong mixing conditions are satisfied by a broad class of stochastic processes commonly encountered in theoretical and applied time series modeling. This paper focuses on the analysis of the second-order properties of the nearest neighbor regression estimator under strong mixing dependence measure. Under appropriate regularity conditions, including strong mixing assumptions and smoothness of the underlying density functions, we derive expressions for both the bias and the quadratic mean squared error (QMSE) of the nearest neighbor regression estimator with a uniform weighting scheme for estimating the unknown conditional mean function. Our results demonstrate that the QMSE of the nearest neighbor estimator attains the minimax-optimal rate for estimating a p-smooth regression function in a d-dimensional embedding space. The theoretical analysis integrates the dependence structure specific to strongly mixing processes with the geometric characteristics of k-nearest neighborhoods. This combination enables the identification of an optimal choice for the number of nearest neighbors that effectively balances the trade-off between bias and variance. Overall, these findings provide a rigorous theoretical foundation for the application of nearest neighbor regression methods to short-range dependent data. Furthermore, the explicit bias and variance characterizations lay the groundwork for establishing asymptotic normality, thereby enabling the construction of valid confidence intervals and supporting reliable statistical inference in dependent data settings.

Published in	American Journal of Theoretical and Applied Statistics (Volume 15, Issue 1)
DOI	10.11648/j.ajtas.20261501.12
Page(s)	12-18
Creative Commons	This is an Open Access article, distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution and reproduction in any medium or format, provided the original work is properly cited.
Copyright	Copyright © The Author(s), 2026. Published by Science Publishing Group

Keywords

Nearest Neighbor Regression; α-mixing Processes, Bias, Variance, Quadratic Mean Square Error

References

[1]	Biau G. and L. Devroye ( 2015). “Lectures on the Nearest Neighbor Method”, volume 1. Springer, New York. https://doi.org/10.1007/978-3-319-25388-6
[2]	Bosq, D. (1998), “Nonparametric statistics for stochastic processes: estimation and prediction”, Lecture Notes in Statistics, Springer-Verlag, 2nd eds, Berlin. https://doi.org/10.1007/978-1-4612-1718-3
[3]	Bradley, R. C. (1986), “Basic properties of strong mixing conditions”, In Dependence in Probability and Statistics, E. Eberlein and M. S. Taqqu (eds.), 165-192, Birkhauser, Boston. https://doi.org/10.1007/978-1-4615-8162-8 8
[4]	Carrasco, M. and X. Chen (2002), “Mixing and moment properties of various GARCH and Stochastic Volatility models”, Econometric Theory 18, 17-39. https://doi.org/10.1017/S0266466602181023
[5]	Collomb, G. (1984), “Nonparametric time series analysis and prediction: Uniform almost sure convergence of the window and k-NN autoregression estimates”, Mathematic Operational Statistics, Series Statistics.
[6]	Davydov Y. A. (1973), “Mixing conditions for Markov chains”, Theory Probab. Appl. 18, 312-328. https://doi.org/10.1137/1118033
[7]	Doukhan P. (1994), “Mixing: properties and examples”, Lecture Notes in Statistics 85, Springer Verlag.
[8]	Doukhan P. and S. Louhichi (1999), “A new weak dependence condition and applications to moment inequalities”, Stochastic Processes and their Applications 84, 313-342. https://doi.org/10.1016/S0304-4149(99)00055-1
[9]	Doukhan P. and O. Wintenberger (2007), “A new weak dependence condition and applications to moment inequalities”, arXiv: 0712. 3231v1.
[10]	Devroye L. P. (1982), “Necessary and sufficient conditions for the pointwise convergence of nearest neighbor regression function estimates”, Z. Wahrscheinlichkeneitstheorie Verw Gebiete 61, 467-481. https://doi.org/10.1007/BF00531618
[11]	Ferrara L., D. Guégan and P. Rakotomarolahy (2010), “GDP nowcasting with ragged-edged data: A semi-parametric modelling”, Journal of Forecasting 29, 186-199. https://doi.org/10.1002/for.1159
[12]	Guégan D. (1983), “Une condition d’ergodicité pour des modèles bilinéaires à temps discret”, CRAS Série 1, 297-301.
[13]	Guégan D. and N. Huck (2005), “On the use of nearest neighbors in finance”, Revue de Finance 26, 67-86. https://doi.org/10.3917/fina.262.0067
[14]	Guégan D. and P. Rakotomarolahy (2010), “ Alternative Methods for Forecasting GDP. In Nonlinear Modeling of Economic and Financial Time-Series”, Book Chapter, Chapter 8, 161-185, Emerald Group Publishing Limited.
[15]	Ibragimov, I. A. (1962), “Some limit theorems for stationary processes” Theory of Probability and Application 7, 349-382. https://doi.org/10.1137/1107036
[16]	Lin Z. Y. and C. R. Lu (1996), “Limit theory for mixing dependent random variables”, Kluwer Academic Publishers.
[17]	Mack Y. P. (1981), “Local properties of k-NN regression estimates”, SIAM Journal on Algebraic and Discrete Methods 2, 311-323. https://doi.org/10.1137/0602035
[18]	Mokkadem A. (1990), “Propriétés de mélange des modèles autoregressifs polynomiaux”, Annales de l’Institut Henri Poincare 26, 219-260.
[19]	Nowman B. and B. Saltoglu (2003), “Continuous time and nonparametric modelling of U. S. interest rate models”, International Review of Financial Analysis 12,25- 34. https://doi.org/10.1016/S1057-5219(02)00123-0
[20]	Peligrad M. (1986), “Recent advances in the central limit theorem and its weak invariance principle for mixing sequences of random variables (a survey)”, In Dependence in Probability and Statistics, E. Eberlein andM. S. Taqqu (eds.), 193-223, Birkhauser, Boston. https://doi.org/10.1007/978-1-4615-8162-8 9
[21]	Peligrad, M. and S. A. Utev (1997), “Central limit theorem for Linear Processes”, The Annals of Probability 25, 443-456. https://doi.org/10.1214/aop/1024404295
[22]	Puspitasari D. A. and Z. Rustam (2018), “Application of SVM-KNN Using SVR as Feature Selection on Stock Analysis for Indonesia Stock Exchange.” AIP Conference Proceedings 2023, 020207; https://doi.org/10.1063/1.5064204
[23]	Rakotomarolahy P. (2012), “Statistical properties of nearest neighbor regression function estimate for strong mixing processes”, available at https://hal.science/hal-00725841.
[24]	Rakotomarolahy P. (2020), “Asymptotic behavior of encompassing test for independent processes: case of linear and nearest neighbor regressions”, Cogent Mathematics and Statistics 7, 1805092. https://doi.org/10.1080/25742558.2020.1805092
[25]	Rosenblatt M. (1956), “A central limit theorem and a strong mixing condition”, Proc. Natl. Acad. Sci. USA 42, 43-47. https://doi.org/10.1073/pnas.42.1.43
[26]	Royall, R. M. (1966), “A class of nonparameteric estimates of a smooth regression function”, Ph.D. dissertation, Stanford University, Stanford.
[27]	Stone C. (1977), “Consistent nonparametric regression”, Annals of Statistics 5, 595-645. https://doi.org/10.1214/aos/1176343886
[28]	Stute W. (1984), “Asymptotic normality of nearest neighbor regression function estimates”, Annals of Statistics 12, 917-926. https://doi.org/10.1214/aos/1176346711
[29]	Yakowitz S. (1987), “Nearest neighbors method for time series analysis”, Journal of Time Series Analysis 8, 235-247. https://doi.org/10.1111/j.1467-9892.1987.tb00435.x

Cite This Article

Plain Text BibTeX RIS

APA Style

Rakotomarolahy, P. (2026). Second Order Properties of Nearest Neighbor Regression with Uniform Weighting Function for Strongly Mixing Processes. American Journal of Theoretical and Applied Statistics, 15(1), 12-18. https://doi.org/10.11648/j.ajtas.20261501.12

Copy | Download

ACS Style

Rakotomarolahy, P. Second Order Properties of Nearest Neighbor Regression with Uniform Weighting Function for Strongly Mixing Processes. Am. J. Theor. Appl. Stat. 2026, 15(1), 12-18. doi: 10.11648/j.ajtas.20261501.12

Copy | Download

AMA Style

Rakotomarolahy P. Second Order Properties of Nearest Neighbor Regression with Uniform Weighting Function for Strongly Mixing Processes. Am J Theor Appl Stat. 2026;15(1):12-18. doi: 10.11648/j.ajtas.20261501.12

Copy | Download

@article{10.11648/j.ajtas.20261501.12,
  author = {Patrick Rakotomarolahy},
  title = {Second Order Properties of Nearest Neighbor Regression with Uniform Weighting Function for Strongly Mixing Processes
},
  journal = {American Journal of Theoretical and Applied Statistics},
  volume = {15},
  number = {1},
  pages = {12-18},
  doi = {10.11648/j.ajtas.20261501.12},
  url = {https://doi.org/10.11648/j.ajtas.20261501.12},
  eprint = {https://article.sciencepublishinggroup.com/pdf/10.11648.j.ajtas.20261501.12},
  abstract = {We examine the statistical properties of nearest neighbor regression function estimation beyond the classical i.i.d assumption. Second-order properties of this estimator for uniformly mixing processes have been derived in previous studies. Nevertheless, uniform mixing is a very strong form of dependence that is difficult to achieve. In contrast, strong mixing conditions are satisfied by a broad class of stochastic processes commonly encountered in theoretical and applied time series modeling. This paper focuses on the analysis of the second-order properties of the nearest neighbor regression estimator under strong mixing dependence measure. Under appropriate regularity conditions, including strong mixing assumptions and smoothness of the underlying density functions, we derive expressions for both the bias and the quadratic mean squared error (QMSE) of the nearest neighbor regression estimator with a uniform weighting scheme for estimating the unknown conditional mean function. Our results demonstrate that the QMSE of the nearest neighbor estimator attains the minimax-optimal rate for estimating a p-smooth regression function in a d-dimensional embedding space. The theoretical analysis integrates the dependence structure specific to strongly mixing processes with the geometric characteristics of k-nearest neighborhoods. This combination enables the identification of an optimal choice for the number of nearest neighbors that effectively balances the trade-off between bias and variance. Overall, these findings provide a rigorous theoretical foundation for the application of nearest neighbor regression methods to short-range dependent data. Furthermore, the explicit bias and variance characterizations lay the groundwork for establishing asymptotic normality, thereby enabling the construction of valid confidence intervals and supporting reliable statistical inference in dependent data settings.
},
 year = {2026}
}

Copy | Download

TY  - JOUR
T1  - Second Order Properties of Nearest Neighbor Regression with Uniform Weighting Function for Strongly Mixing Processes

AU  - Patrick Rakotomarolahy
Y1  - 2026/01/16
PY  - 2026
N1  - https://doi.org/10.11648/j.ajtas.20261501.12
DO  - 10.11648/j.ajtas.20261501.12
T2  - American Journal of Theoretical and Applied Statistics
JF  - American Journal of Theoretical and Applied Statistics
JO  - American Journal of Theoretical and Applied Statistics
SP  - 12
EP  - 18
PB  - Science Publishing Group
SN  - 2326-9006
UR  - https://doi.org/10.11648/j.ajtas.20261501.12
AB  - We examine the statistical properties of nearest neighbor regression function estimation beyond the classical i.i.d assumption. Second-order properties of this estimator for uniformly mixing processes have been derived in previous studies. Nevertheless, uniform mixing is a very strong form of dependence that is difficult to achieve. In contrast, strong mixing conditions are satisfied by a broad class of stochastic processes commonly encountered in theoretical and applied time series modeling. This paper focuses on the analysis of the second-order properties of the nearest neighbor regression estimator under strong mixing dependence measure. Under appropriate regularity conditions, including strong mixing assumptions and smoothness of the underlying density functions, we derive expressions for both the bias and the quadratic mean squared error (QMSE) of the nearest neighbor regression estimator with a uniform weighting scheme for estimating the unknown conditional mean function. Our results demonstrate that the QMSE of the nearest neighbor estimator attains the minimax-optimal rate for estimating a p-smooth regression function in a d-dimensional embedding space. The theoretical analysis integrates the dependence structure specific to strongly mixing processes with the geometric characteristics of k-nearest neighborhoods. This combination enables the identification of an optimal choice for the number of nearest neighbors that effectively balances the trade-off between bias and variance. Overall, these findings provide a rigorous theoretical foundation for the application of nearest neighbor regression methods to short-range dependent data. Furthermore, the explicit bias and variance characterizations lay the groundwork for establishing asymptotic normality, thereby enabling the construction of valid confidence intervals and supporting reliable statistical inference in dependent data settings.

VL  - 15
IS  - 1
ER  -

Copy | Download

Author Information

Patrick Rakotomarolahy

LaMAF - Laboratory of Mathematics and their Applications of Fianarantsoa, University of Fianarantsoa, Fianarantsoa, Madagascar

Contact Email

http://orcid.org/0000-0002-7897-5356

Download PDF

Submit an Article

Sections

Plain Text BibTeX RIS

APA Style

Rakotomarolahy, P. (2026). Second Order Properties of Nearest Neighbor Regression with Uniform Weighting Function for Strongly Mixing Processes. American Journal of Theoretical and Applied Statistics, 15(1), 12-18. https://doi.org/10.11648/j.ajtas.20261501.12

Copy | Download

ACS Style

Rakotomarolahy, P. Second Order Properties of Nearest Neighbor Regression with Uniform Weighting Function for Strongly Mixing Processes. Am. J. Theor. Appl. Stat. 2026, 15(1), 12-18. doi: 10.11648/j.ajtas.20261501.12

Copy | Download

AMA Style

Rakotomarolahy P. Second Order Properties of Nearest Neighbor Regression with Uniform Weighting Function for Strongly Mixing Processes. Am J Theor Appl Stat. 2026;15(1):12-18. doi: 10.11648/j.ajtas.20261501.12

Copy | Download

@article{10.11648/j.ajtas.20261501.12,
  author = {Patrick Rakotomarolahy},
  title = {Second Order Properties of Nearest Neighbor Regression with Uniform Weighting Function for Strongly Mixing Processes
},
  journal = {American Journal of Theoretical and Applied Statistics},
  volume = {15},
  number = {1},
  pages = {12-18},
  doi = {10.11648/j.ajtas.20261501.12},
  url = {https://doi.org/10.11648/j.ajtas.20261501.12},
  eprint = {https://article.sciencepublishinggroup.com/pdf/10.11648.j.ajtas.20261501.12},
  abstract = {We examine the statistical properties of nearest neighbor regression function estimation beyond the classical i.i.d assumption. Second-order properties of this estimator for uniformly mixing processes have been derived in previous studies. Nevertheless, uniform mixing is a very strong form of dependence that is difficult to achieve. In contrast, strong mixing conditions are satisfied by a broad class of stochastic processes commonly encountered in theoretical and applied time series modeling. This paper focuses on the analysis of the second-order properties of the nearest neighbor regression estimator under strong mixing dependence measure. Under appropriate regularity conditions, including strong mixing assumptions and smoothness of the underlying density functions, we derive expressions for both the bias and the quadratic mean squared error (QMSE) of the nearest neighbor regression estimator with a uniform weighting scheme for estimating the unknown conditional mean function. Our results demonstrate that the QMSE of the nearest neighbor estimator attains the minimax-optimal rate for estimating a p-smooth regression function in a d-dimensional embedding space. The theoretical analysis integrates the dependence structure specific to strongly mixing processes with the geometric characteristics of k-nearest neighborhoods. This combination enables the identification of an optimal choice for the number of nearest neighbors that effectively balances the trade-off between bias and variance. Overall, these findings provide a rigorous theoretical foundation for the application of nearest neighbor regression methods to short-range dependent data. Furthermore, the explicit bias and variance characterizations lay the groundwork for establishing asymptotic normality, thereby enabling the construction of valid confidence intervals and supporting reliable statistical inference in dependent data settings.
},
 year = {2026}
}

Copy | Download

TY  - JOUR
T1  - Second Order Properties of Nearest Neighbor Regression with Uniform Weighting Function for Strongly Mixing Processes

AU  - Patrick Rakotomarolahy
Y1  - 2026/01/16
PY  - 2026
N1  - https://doi.org/10.11648/j.ajtas.20261501.12
DO  - 10.11648/j.ajtas.20261501.12
T2  - American Journal of Theoretical and Applied Statistics
JF  - American Journal of Theoretical and Applied Statistics
JO  - American Journal of Theoretical and Applied Statistics
SP  - 12
EP  - 18
PB  - Science Publishing Group
SN  - 2326-9006
UR  - https://doi.org/10.11648/j.ajtas.20261501.12
AB  - We examine the statistical properties of nearest neighbor regression function estimation beyond the classical i.i.d assumption. Second-order properties of this estimator for uniformly mixing processes have been derived in previous studies. Nevertheless, uniform mixing is a very strong form of dependence that is difficult to achieve. In contrast, strong mixing conditions are satisfied by a broad class of stochastic processes commonly encountered in theoretical and applied time series modeling. This paper focuses on the analysis of the second-order properties of the nearest neighbor regression estimator under strong mixing dependence measure. Under appropriate regularity conditions, including strong mixing assumptions and smoothness of the underlying density functions, we derive expressions for both the bias and the quadratic mean squared error (QMSE) of the nearest neighbor regression estimator with a uniform weighting scheme for estimating the unknown conditional mean function. Our results demonstrate that the QMSE of the nearest neighbor estimator attains the minimax-optimal rate for estimating a p-smooth regression function in a d-dimensional embedding space. The theoretical analysis integrates the dependence structure specific to strongly mixing processes with the geometric characteristics of k-nearest neighborhoods. This combination enables the identification of an optimal choice for the number of nearest neighbors that effectively balances the trade-off between bias and variance. Overall, these findings provide a rigorous theoretical foundation for the application of nearest neighbor regression methods to short-range dependent data. Furthermore, the explicit bias and variance characterizations lay the groundwork for establishing asymptotic normality, thereby enabling the construction of valid confidence intervals and supporting reliable statistical inference in dependent data settings.

VL  - 15
IS  - 1
ER  -

Copy | Download