polyval and polyfit functions on noisy data

[tommyears]

Hi,

I have two sets of data, 'x' and 'y'. They are related to each other.

x = 239.18 239.18 239.18 486.63 486.63 237.9 237.9 412.33 412.33 412.33 412.33 367.51 367.51 367.51 367.51 367.51

y = 288.74 301.76 333.2 166.87 210.52 337.92 299.54 273.76 224.1 197.59 199.37 251.24 247.53 236.45 233.29 204.35

From looking at the data it can been seen that the values are corrupted by noise - this seems to be giving me problems when trying the use the polyfit and polyval functions to plot a 'line of best fit'. The reason I think is because x = 239.18 239.18 239.18 (all the same value) relate to slightly different values of y (due to noise) y = 288.74 301.76 333.2. The line fitting functions seem to have problems resolving this when the line of best fit is to be plotted.

I would appreciate any help in resolving this issue.

Regards

Tom Myers

 

[VisiGoth]

If you essentially only have 4 distinct x values, then you should only be asking for a polynomial with much less than order 3. That does not leave you with many choices. I will assume you only want a Y = mX + b solution. If you wanted a second order you should have more than 4 to be reasonable. I hope you did not want a cubic. If you did you might want to try non-linear filtering first. Perhaps a stack filter. Take the middlemost y of each of the nominal x values. The difficulty here is defining nominal x in general terms. A clustering algorithm could help. If this is your only data, than manually select the points. Or make a heavy quantization on the value of x, this clusters them to the same x value so that you can select the middlemost y for the each of the 4 quantized x's.

 

[Ali768]

dont know how critical your applicaion is but this is
just a guess/approx .........

try fitting the linear fit first and estimate the mid points for each y clustered set this will give u linear fit (y = -0.5106 + 430.82)with R2 = 0.7799 (not good) with all clustered y values lying within 18% error bars.

then estimate with these points a ploynomial (y = -4exp-17 x^2-0.5106x + 430.82) with R2 = 1.

x y (LINEAR) fit (POLY) FIT
239.18 288.74 308.69 308.685

486.63 166.87 182.35 182.308

237.90 337.92 309.35 309.339

412.33 273.76 220.28 220.256

367.51 251.24 243.17 243.147

 

[dm215]

The polyfit function in MATLAB uses a linear least squares fitting technique. In other words, it always finds the global minimum (or should, in theory). This means that if polyfit is giving you an unsatisfactory fit for both low and high order polynomials, you need to try some other type of function and write the optimization code for finding its parameters. Fortunately, this is not too hard (if you can find a suitable trial function). All you have to do is use the chosen trial function to write an m-file that calculates the norm of your error vector (xi - yi) as a function of the fitting parameters, then use fminsearch.