Industrial Food Quality Analysis Using New k-Nearest-Neighbour methods

Omar Fetitah,Ibrahim M.Almanjahie,Mohammed Kadi Attouch,＊and Salah Khardani

1Laboratory of Statistics and Stochastic Processes,University of Djillali Liabes,Sidi Bel Abbes,22000,Algeria.

2Department of Mathematics,College of Science,King Khalid University,Abha,62529,Saudi Arabia

3Statistical Research and Studies Support Unit,King Khalid University,Abha,62529,Saudi Arabia

4Faculté des sciences de Tunis,Laboratoire des Réseaux Intelligents et Nanotechnologie,Tunis,Tunisia

Abstract:The problem of predicting continuous scalar outcomes from functional predictors has received high levels of interest in recent years in many fields,especially in the food industry.The k-nearest neighbor(k-NN)method of Near-Infrared Reflectance (NIR) analysis is practical,relatively easy to implement,and becoming one of the most popular methods for conducting food quality based on NIR data.The k-NN is often named k nearest neighbor classifier when it is used for classifying categorical variables,while it is called k-nearest neighbor regression when it is applied for predicting noncategorical variables.The objective of this paper is to use the functional Near-Infrared Reflectance (NIR) spectroscopy approach to predict some chemical components with some modern statistical models based on the kernel and k-Nearest Neighbour procedures.In this paper,three NIR spectroscopy datasets are used as examples,namely Cookie dough,sugar,and tecator data.Specifically,we propose three models for this kind of data which are Functional Nonparametric Regression,Functional Robust Regression,and Functional Relative Error Regression,with both kernel and k-NN approaches to compare between them.The experimental result shows the higher efficiency of k-NN predictor over the kernel predictor.The predictive power of the k-NN method was compared with that of the kernel method,and several real data sets were used to determine the predictive power of both methods.

Keywords:Functional data analysis;classical regression;robust regression;relative error regression;kernel method;k-NN method;near-infrared spectroscopy

1 Introduction

Near-infrared spectroscopy (NIRS) is a technique for measuring and analyzing reflection spectra in a class of wavelengths.Fig.1 illustrates the basic components of the NIR spectroscopy technique;this measurement technique is analytical,fast,and non-destructive is often used to measure some parameters in terms of spectrum absorbance.For example,in the pharmaceutical industry,it is used in the manufacturing process of a drug to control the active ingredient’s exact amount.In the food industry,the spectrum can be used to test the forage quality [1].In medical science,fluorescence spectroscopy can be used for cancer screening.Finally,in the food industry,for example,a method of classifying flour products based on resistance spectra of dough in bakeries.

Figure 1:Schematic representation of the near-infrared NIR spectroscopy-based setup,the basic

Although the NIR has given excellent results when used in various other fields such environment and the petrochemical industries,it remains relatively new for its use in virology.This method has also been used with great success for the identification of HIV-1 and the influenza virus.The advantage of using this method is that it does not require reagents or test kits that take a considerable time to perform these tests.For example,we mention the PCR (Polymerase chain reaction) or RT-PCR (reverse transcription-polymerase chain reaction) test that gives results in most cases for more than 2 h.

Usually,the NIR spectrometry is combined with some multivariate statistical models,such as the principal component regression or the partial least regression.To increase the accuracy of this procedure,we use the recent development in data science.Precisely,we combine the NIR spectrometry technology with big-data techniques modeling.The statistical modeling of big-data is an emerging topic of applied statistics.It has received considerable attention during the last decade.The development of the current technology provides a way to measure different types of instruments and the informatics tools that motivate this subject’s work.Besides,this advancement allows the researchers to recover big data being recorded over time.

One of the most advantages of this thematic is the fact that the statistical data can be treated as curves.Our main goal in this project is to develop a new software code induced from some recent statistical models adapted for NIR spectrometry data viewed as curves.The proposed models include the functional version of the PCR regression (principal component regression),and the PLS regression (partial least squares regression),etc.It is worth noting that the originality of the nonparametric analysis of functional statistics is that it links the probability structure to the topological structure to explore the most pertinent information about the data.An alternative to the preceding methods,we propose a new smoothing method constructed by the combination of the nonparametric functional regression methods and the kernel nearest-neighbor scheme.This new smoothing method keeps the robustness of the weighting functions.

Functional data analysis (FDA) arises mainly to resolve problems relating to time-like curves.In chemometric,it is usual to measure specific parameters in terms of a set of spectrometric curves that are observed in a finite set of points (functional data).In the past decades,spectroscopy has steadily gained importance as a rapid and non-destructive analytical technique in the domains of medicine,chemistry and pharmaceutical,environmental,agricultural,and food sciences.

Near-infrared spectrometry (NIR) provides benchmark examples coming from chemometrics.It is an analytical chemometric technology quick technique that involves subjecting a sample to infrared radiation to measure certain parameters of interest in terms of the absorbance spectrum;see,among others [2,3].Absorption spectroscopy is used as an analytical chemistry tool to determine the presence of a particular substance in a sample and,in many cases,to quantify the amount of the substance present.The utility of absorption spectroscopy in chemical analysis is because of its specificity and its quantitative nature.In spectroscopy,the measured spectra are typically plotted as a function of the wavelength or wave-number but analyzed with functional data analysis (FDA) techniques.Traditionally,spectral data are analyzed through multivariate statistical methods such as multiple linear regression (MLR),principal components regression(PCR),and partial least squares regression (PLS) [4,5],which consider the spectrum as a set of m different variables (curves).

There are many applications of the FDA in spectrometry.For example,these NIR spectra have been used in [6]to predict the oil content of the corn samples (multivariate calibration).In [7],the goal is to predict the composition (fat,sugar,and water content) of biscuit dough pieces using predictors of the NIR reflectance spectrum of dough pieces at 256 equally spaced wavelengths.In the food industry,the spectrum can be used to predict the fatness of a piece of meat (see [8]).NIR spectra are also used to study the forage quality assessment (see [9-12]for recent advances).

More precisely,this paper aims to use the functional Near-Infrared Reflectance spectroscopy approach to predict some chemical components with some modern statistical models based on the kernel and k-NN procedures.In this article,three NIR spectroscopy datasets are used as examples:Cookie dough,sugar,and tecator data.Specifically,we propose three models for this kind of data:Functional Nonparametric Regression,Functional Robust Regression,and Functional Relative Error Regression,with both kernel and k-NN approaches.

The paper is organized as follows.Section 2 describes the prediction problems and the data used.We discuss our results in Section 3.The conclusion is presented in Section 4.

2 Materials and Methods

2.1 Spectroscopic Analysis

Grid of measurements Near-infrared spectrometry provides benchmark examples coming from chemometrics.This is a non-destructive technology able to measure numerous chemical compounds in a wide variety of products (food industry,petroleum industry,wood industry,etc.);see among others [2,13-16].For instance,let us consider a sample of 72 cookie dough samples.Each sample is illuminated by a light beam at 700 equally spaced wavelengths(ω1,...,ω700)in the near-infrared range 1100?2498 nm.For each wavelengthωand each cookie samplei,the absorptionXi(ω)of radiation is measured.Theith discretized spectrometric curve is given byXi(ω1),...,Xi(ω700),and Fig.2 displays the 72 spectrometric curves.

Figure 2:Cookie dough 72 samples of near-infrared spectra

All these curves involve some continuum in their structure,even if they are observed at discrete points.The terminology of functional data refers to this continuous feature.Figs.3 and 4 give a benchmark example of such data for the food industry introduced in [17]:268 samples of sugar were dissolved,and the solution was measured spectrofluorometrically.For every sample,the emission spectra from 275 to 560 nm were measured in 0.5 nm intervals (i.e.,at 571 wavelengthsω1,...,ω571) theith discretized spectrometric curve is given byXi(ω1),...,Xi(ω571).We mention [18],who studied 215 finely chopped pieces of meat (tecator data).For theith piece of meat,one observes a spectrum of absorptionXi(·)sampled at 100 equally spaced wavelengthsω1,...,ω100from 850 to 1050 nm.

Figure 3:268 spectrometric curves sampled of the sugar data

Figure 4:The 215 NIR spectroscopy curves of the tecator data

Throughout these three examples,which will be our connecting thread,one can remark that the grid of measurements (i.e.,wavelengths) for the spectrometric curves is quite dense.

In chemometrics,there are often function-like absorbance or emission spectra—mainly for food samples—used to determine certain ingredients’content.The use of spectra function is typically much cheaper than alternative chemical analysis.

2.2 Statistical Analysis

This paper aims to present various ways of nonlinear modeling relationships in datasets containing functional data and discuss methodological aspects.We focus on the particular case when one regresses a scalar response on an explanatory functional variable.To fix the ideas,let’s present the mathematical formulation of our prediction problem.Indeed,assume that we aim to predict the content of certain ingredients:the sucrose content for the cookie dough,the quality ash in the percentage of the sugar given,and the fat content for the piece of meat.Denoted contents byYi,the spectrometric curves associatedXi.Note thatY’s values for the percentage of the sugar are discrete;Therefore,we will consider thatYis a continuous approximation.We assume that the output variableYand the input variableXare linked by the following regression formula

wherem(·)is an unknown operator modeling the relationship betweenXandYand the white noiseεrepresents an independent random variable ofXwith a symmetric distribution.The statistical challenge consists of proposing a relevant estimator.Here,we focus our attention on regression models such thatm(X)=E(Y|X)(i.e.,E(ε|X)=0),and propose in the following three models:Functional Nonparametric Classical Regression,Functional Robust Regression,and Functional Relative Error Regression.

2.2.1 Functional Classical Regression

The nonparametric estimation of the functional regression was initially studied by [19,20],who used the Nadaraya Watson method to estimate this statistical model.Precisely,the functionm(.)is explicitly expressed using the least square error criterion by

It follows that

m(x)=E[Y|X=x].

So,for all fixed curvesxwe predict the responseywith respect to the criterion in Eq.(2) by(x)(the classical kernel estimator ofm(x)) defined by

withKis a kernel function andhnis a non-negative real sequence.

2.2.2 Functional Robust Regression

This regression model is obtained by resolving the following optimization problem

ρis a real-valued Borel function chosen by the user according to the studied data.The model in Eq.(4) has been introduced in functional statistics by [21,22].The robustness is the main advantage of this model.It permits the analysis of the data even in the presence of the outliers.Its functional estimation is expressed by

2.2.3 Functional Relative Error Regression

This last regression is an alternative nonparametric regression to the least square regression model.It is recently considered in functional statistics by [23].It is defined by the following rule

The expression of this regression is explicitly given by

and its estimator is defined by

3 Results and Discussions

The performance of all the models mentioned above is closely linked with the use of different parameters involved in the estimation.We opted for the asymmetric quadratic kernel defined asThus,the smoothness of curvesXi(t)and the smoothing parameterhnare the most influencing parameters in this prediction issue.Concerning the norm ‖.‖,the distances between the smoothed curves are computed by

For basic materials on the latter notion,we refer the readers to [19].On the other hand,the bandwidth parameter,h,selection is a more important procedure for conducting the estimation.Our main goal is to compare two methods (kernel CV method and the k-Nearest Neighbors k-NN method) for our three estimators,and.In the following,we describe the use of these methods for our proposed estimators.

Using the kernel CV method,we obtain

and

wherehoptis the data-driven bandwidth obtained by a cross-validation procedure:

Using the method of k-Nearest Neighbors k-NN procedure,we obtain

and

wherehkoptis the bandwidth corresponding to the optimal number of neighbors obtained by a cross-validation procedure:

with

wherenTestis the length of the testing sample andindicate the prediction values of the estimatorscalculate atXi.The obtained prediction results are shown in Figs.5-7.

Figs.5-7 give an idea of the accuracy of the predictions corresponding to one run.They present the last 15th,16th and 20th of each data’s predictions,respectively:The observed values(black curve),the predicted values (dashed red for the kernel regression,and green for the k-NN one) are drawn.It is depicted in Figs.5-7 that there is a significant gain among the k-NN models compared to the kernel CV ones.The k-NN models for the classical,robust,and relative regression give better results than the kernel CV for the classical,robust,and relative regression.To further explore the performances of the six methods,we carry outM=100 independent replications,which allow us to compute 100 values for MSE and display their distribution through a bean plot.Figs.8-10 show the bean-plots of the MSE of the prediction values.Moreover,Tab.1 shows that the models in Eqs.(11)-(13) give small MSE followed by those in Eqs.(8)-(10).The same fact is confirmed by Tab.2,where we present the RMSE.

Figure 5:Prediction of the last 15 testing cookie dough samples

Figure 6:Prediction of the last 16 testing sugar samples

The values of RMSE are relatively stable and smaller for the three k-NN functional models,namelyas compared to the kernel CV models,namelyand.Although the performance of the studied models is varied,the variability of the MSE and RMSE are relatively stable for the three proposed models k-NN for the classical,robust,and relative regression as compared to that of the kernel CV for the classical,robust and relative regression models.

Figure 7:Prediction of the last 20 testing tecator samples

Figure 8:The bean-plots of the MSE of the prediction values by the six methods for the cookie dough data

The principal NIR data parameters were evaluated using a sample of 72,268,and 215 observations for the cookie dough,sugar,and tecator data,respectively.The results are summarized in Figs.5-10.The analyzed parameters are the sucrose content for the cookie dough,the quality ash in the percentage of the sugar given,and the fat content for the tecator,which are ranged between 9.95%?23.19% for the sucrose content,8%-33% for the ash and 0.9%-49.1% for the tecator fat,respectively.Such a data analysis was operated using six functional models:Functional Nonparametric Classical Regression,Robust Functional Regression,and Functional Relative Error Regression for both Kernel CV andk-NN procedures (i.e.,and

Figure 9:The bean-plots of the MSE of the prediction values by the six methods for the sugar data

Figure 10:The bean-plots of the MSE of the prediction values by the six methods for the tecator data

The comparison of both prediction plots in Figs.5-7 indicates that thek?NN method (green dashed curve) gives better prediction results than the kernel CV approach (red dashed curve).Figs.8-10 display various bean-plot which summarize the distribution of MSE computed over 100 experiments based onfrom left to right,respectively.That confirms the previous results,as we can see the distribution of MSE for the k-NN approach is small and very tight compared to the kernel CV method,as can be clearly seen in Figs.8-10.Based on the results in Tabs.1 and 2,it is clear that the best models (having a small MSE and RMSE) are

Table 1:MSEs by the six methods for each data

Table 2:Relative mean squared error RMSE by the six methods for each data

4 Conclusion

A review of the FDA methodologies,most used in chemometrics,has been presented in this work next to different applications,most of which are in spectroscopy where the absorbance spectrum is a functional variable whose observations are functions of wavelength.The work has been divided into two main parts that can be read independently.The first part (Section 2)presents a set of chemometrics applications in most of which the aim is to either predict a variable of interest from the NIR spectrum.The second part (Section 3) summarizes our functional models’results based on the proposed methods defined in Eqs.8-13.

In this work,an alternative approach to deal with spectrometric data has been suggested.This approach considers a spectrum as a function of the wavelength or wave-number rather than as a set of separate points.We combine the recent development in Chemistry and modern Statistics.Specifically,we use the NIR spectroscopy technology from Chemistry,which is an inexpensive,rapid,and accurate method.Moreover,it reduces the need for conventional wet Chemistry procedures.On the other hand,from modern statistics,we use some functional models that allow exploring all the information of the spectroscopy analysis where spectral data are viewed as curves.Specifically,we propose three models for this kind of data:Functional Nonparametric Regression,Functional Robust Regression,and Functional Relative Error Regression,with both kernel and k-NN approach to compare between them.On the real examples studied (Cookie dough,Sugar,and tecator data),we show that our method using the k-NN procedure is more efficient (gives better results in the sense of MSE) than those with Cross-validation.To conclude,models of intermediate dimensionality in the high-dimensional setting is undoubtedly a highway for deriving new useful statistical methods for the food industry.

Funding Statement:The authors thank and extend their appreciation to the Deanship of Scientific Research at King Khalid University for funding this work through the Research Groups Program under Grant Number R.G.P.1/189/41.I.M.A.and M.K.A.received the grant.

Conflicts of Interest:The authors declare that they have no conflicts of interest to report regarding the present study.