How to create an equally spaced 1D array?

I have two vectors, x and y, of the same length (more or less 2600x1 double). For each element x, a corresponding element y is acquired. x is a vector of points (1,2,3,4...), y contains the corresponding values, which describe a certain function. The problem arises because of the way I acquire these data; the x vector is built from a moving translation motor: for each step of the motor, a new point is recorded. For example, when the motor moves a 1 mm step, I acquire the first element of x, i.e. 1, and the corresponding element y (it doesn't matter what y is, but it depends on the position of the motor too). After another step of 1 mm, I record the second element of x (i.e. 2) with the corresponding element (that is the second element of y) and so on. The problem is that the scanning is actually irregular: the motor doesn't move on perfectly regular steps of 1 mm, but sometimes for example 0.8 mm, 0.9 mm or 1.1 mm (also in this case the x vector records points separated by one - 1,2,3,4,5...- because a new element of x is generated for any movement of the motor, of whatever step). I need to build an x vector, and the corresponding y vector, *as if I had a perfectly regular scanning motor of equally spaced steps . How can I do this? Shall I have to correct the x vector by creating an equally space vector? I need also a corresponding y vector, so also the y vector must be corrected in function of the x one. I know it's not trivial, if you want more details ask me. Thanks

Answer 1

[Roger Stafford]

You haven't made it clear if the y values are the result of some kind of measurement independent of x steps, but to have any hope of making the adjustment you describe, that would be absolutely essential. You have to have some kind of feedback from your machine in order to do what you ask. What you would be looking for are outliers among the y values that don't correspond with regularly=spaced x values, which in turn would be making assumptions about the continuity of the relationship between x and y. For some set of other y values on either side of a given y and assuming regularity of corresponding x values, you would observe the difference between the interpolated y against the given y at the central point. If they are too different, that y is to be regarded as an outlier and you would replace it with the interpolated value and assume regular steps in x.

It all sounds rather unreliable to me, because if there are uncertainties in y measurement it will be difficult to distinguish between these and outliers due to actual variation in x steps, and the x variations in turn will have to be rather widely-separated to make this work properly. In any case matlab makes available the function 'interp1' which could perform the needed interpolation for you.

[aurc89]

Yes, the y values actually depend on the position of the machine: a laser beam crosses an optical device placed on the motor, so when the motor moves, the detected power of the beam (which is collected in the y vector) changes. Your explanation is clear, but my problem is how to implement a code in Matlab, because I didn't manage to use interp1 properly.

[dpb]

Show your work, then, where you had a problem.

Do you have a measurement 'x' or is that the problem that you don't actually know what x is? It's not clear to me after rereading just what is actually measured and what you're after.

[Roger Stafford]

After thinking about your problem a little more carefully, I have changed my mind about using 'interp1'. What you need for detecting outliers in the y-values is 'polyfit', perhaps with a quadratic or cubic fit (n = 2 or 3). I am assuming you have already collected all the y values in a vector. For each y value, you can do a polyfit to get a best fit polynomial on a set of surrounding y values with the given y missing and assuming all the x values are equally spaced except of course at the position of the missing y. Then you can use 'polyval' to evaluate this polynomial at the missing value of x. If it differs too widely from the missing y value, then that missing y-value will be regarded as an outlier and replaced by the polyval value. In this way you can remove all outliers and have a smooth-looking y set of values that correspond to equally-space x values. Thus your problem is not one of creating equally-spaced x values. That is trivial. It is one of removing outliers in the y-values which are caused by the actual presence of irregularly-spaced x values.

If you are attempting to perform this computation as the values in y are being added, the one change you would make is to have 'polyfit' extend, not on both sides of a missing y but entirely to the previous y value side. Polyfit would then be predicting what the next y would be and comparing that with the measured y. If they differ too widely, the measured y would be replaced by the predicted value. Or perhaps you could devise a smart function which makes an intelligent compromise between the two conflicting values.

[aurc89]

dpb sorry, I've been not so clear: x just collects points in a progressive order (1,2,3...) independent from the real position of the motor, while I'd like to build a vector as if the motor moved with equally spaced steps.

Roger, thanks, maybe it could be a good idea. So, if x and y are the initial vector (both of length more or less 2600x1) what could be a Matlab code to obtain the new vectors xx and yy with the required conditions? It is difficult for me to iterate the process over all the length of the vector. Thank you very much

[aurc89]

This figure maybe explains better what I want to say: the blue curve is obtained from x and y vectors; x elements are 1,2,3,4,5..., but the real positions of the motor are given by the blue dots, which are not equally spaced, so in this way I'm plotting a wrong curve.

I want to build a curve like the red one, as if I had a motor moving exactly with same steps. How can I do this, starting from x and y vector and making a sort of interpolation ? I don't know how to implement such a code.

[Roger Stafford]

I repeat my earlier assertion. Your knowledge of x values is highly inaccurate. All you really have to work with in the way of accurate measurements are the y-values and these are obtained at irregular intervals of the actual motor position which x would represent if it had been measured accurately. You hope to adjust the y-values so as to best correspond to an evenly-spaced motor position. Because of this inaccuracy however, it is very likely inappropriate to attempt an interpolation process.

It is a difficult undertaking and only possible if there is not an excessive change in the actual nature of the y curve for a number of measurements - that is, the curve expressing how y varies with respect to the true motor position needs to be relatively smooth. (The image you show is rather discouraging in this respect.)

What I am recommending is to do a polyfit, p = polyfit(x,y,n), using, say n = 2 or 3, for a section of successive y measurements, considerably longer than n, in which the x are the theoretical successive integers you have described. Because of the actual irregularity of motor positions, the individual y values probably won't fit the quadratic or cubic curve you obtain very well, and the quadratic/cubic polynomial is (hopefully) a better representation of what the y's would be for equally-spaced motor positions than those you are receiving. Use polyval to adjust the center y value if it is a sufficiently bad fit - that is, it is an outlier - and move on one y value to the right and repeat this polyfit process, each time making a possible adjustment to the center y. You should put all the adjusted or non-adjusted center y values you get in this way in a separate array. This new array ought to better represent the quantity you would have measured for y if the motor positions had been regular. It may be necessary to repeat the whole process, obtaining yet other arrays of further adjusted y values until you have a reasonably smooth curve of their values.

(to be continued)

[Roger Stafford]

(continued from the previous comment because of its excessive length)

As you can tell from this description, you cannot hope for high accuracy by such a method, but that is inherent in the problem you face. You can obtain only so much information from y measurements, however accurately made, if the corresponding motor positions are inaccurately known.

I am not going to do the coding for this process. That is for you to carry out, Aurelio. It will no doubt require considerable trial and error to get the best combination of 1) the best number of points to include in the polyfit call, 2) the best degree n to use in polyfit, 3) the best number of total adjustments to make of the entire array, and perhaps the most crucial item of all, 4) the tolerance level you set to determine which y values are to be regarded as outliers. (You might want to use some weighted average of the two values in 4) instead.) It will probably be a lengthy process of repeated coding and observation of results, and I am not prepared to undergo that.

Answer 2

[dpb]

Create a new x set of points however desired--a couple of possible choices would be N points between the first and last actual points, alternatively, fix from some x0 to xN at whatever you wish the endpoints to be. You may or may not want to extrapolate to get there, your choice

x1=linspace(x(1),x(end),nPts);  % This is first alternative
y1=interp1(x,y,x1);             % interpolate to the new x1 points

doc interp1    % for more details, options on extrapolation, interpolation type, etc., ...

[aurc89]

It doesn't work like this :-( Thanks however

[dpb]

OK, but that certainly wasn't clear originally (to me, at least... :) )

Anyway, looking at your plots --

a) is 'x' known at any one (or more) point(s) at all? IOW, is there at least an origin and perhaps and endpoint that can be considered accurate?

b) is it to be assumed from the characteristic shape of the red curve that the actual response is to be linear?

Depending on the answers to the above, there are possibly some useful boundary conditions that can be used to aid in a solution similar to that proposed by Roger.

Also, what's known about the characteristics of the motor movement? Is it that the the motor moves at fixed speed and the sampling is simply asynchronous giving rise to the difference in actual positions or what? These kinds of ancillary pieces of info may also be helpful in modeling the output (I'm thinking "Kalman filter" maybe???)