Non-uniform noise distribution ruins filter attempt 8

[osbornera]

I have a differentially coupled DAQ Analog Input collecting data from a current sensing resistor at 500 samples per second.

I have a 25HP (VFD) variable frequence drive that is noisy nearby. Dang it! Yes, it's my fault it's so close. I didn't think it would be that noisy. Doh!

I drive the current to my UUT through the current sensing resistor as part of a test. If the VFD is off everything is beautiful. If it is activated, which it must be for my test, I am getting noise that is not randomly distributed around the signal; most of the data ~ 70% is correct (in the ballpark). ~ 1% of the data is too high (out of the ballpark, but not totally unreasonable) and the remaining ~ 29% of the data is too low (out of the ballpark, sometimes way out). When I plot this data I can *see* the *true* value of the waveform, it is *obvious* to my eye and brain what the data is *supposed* to be. Unfortunately, my eye and brain are not part of the requirements for this machine . . .

Any attempt to filter this data shows a plot that is too low, as you would expect, and is essentially out of the ball park, hence unusable. Is there a way to filter this data such that my output will be closer to what I earlier called the *true* values.

I've tried looking at rms values, medians, modes, means, butterworth and chebyshev filtering of this data. These methods work for all of my analog channels except for my current sensor and my thermocouples.

It looks like some kind of offset might help, but I have no idea how to calculate that kind of thing on the fly.

Ideas would be appreciated if you have any, I'm about ready to spring for some other kind of current sensor. I've used every bit of noise suppression that I could, given my ability and the geometry of my apparatus, which is pretty much set.

Regards,
Rick

 

[analogkid2digitalman]

What is the frequency content of the noise? If >250Hz that might explain the spurious signals (aliasing).

Is the noise induced by RFI/EMI or could it be a real spurious change in current that you are seeing.

What current range are you monitoring and with what voltage output from the shunt? Would a >V/A shunt be useful in increasing the S/N ratio (i.e. if you are only getting 100mV out of the shunt and the noise is +/-50mV, move up to a shunt that puts out volts of signal)

Have you tried an oscilloscope in differential input mode to verify signal at the shunt?

I would at least try to highly oversample and create an average. Sample at 10KHz and average every 100 data points for example.

Just some suggestions, hope they can help.

-Pete



Wheels within wheels / In a spiral array
A pattern so grand / And complex
Time after time / We lose sight of the way
Our causes can't see / Their effects.

 

[VEBill]

Perhaps you could create the SW equivalent of a histogram to pick out the highest peak (the 70% heap) and then exclude all data not associated with that highest peak. A complete algorithm is a bit more complicated and should have lots of error checks ('I dunno' needs to be an allowed output when there is no distinct peak).

Sometimes algorithmic-based filtering is a much better approach than math-based filtering. Most human-brain filtering is algorithmic-based as you've seen. Just implement SW that does what you do.

The other approach would be to try to fix the noise problem directly (shielding, passive filtering). AK2DM has some good points there.

 

[itsmoked]

Another idea.

The VFD's put all their noise out at whatever their modulation frequency is. Something like 7 or 10kHz. You could comb filter that frequency out.

Or phase lock loop the noise so you have it and then do synchronous filtering for whopping big reductions. You can then make sure your A2D system never takes readings at the switching points of the VFD. Use the last reading again if a reading is required or delay a reading that would coincide with a VFD switching.

Do you understand what I'm saying?

 

[osbornera]

All, thanks.

The frequency content is mostly the carrier frequency of the VFD 6kHz, but my noise floor seems to be below -60db which I thought was pretty good. I have been very careful with respect to grounding/shielding issues. I am in the process of building the final product which will have additional power filtering for the computer with the DAQ installed. I am hoping that this will have a positive effect. I have experimented with sampling rates from 500-8000 S/s. My waveform is a 1/8 Hz triangle so I have a bit of overkill installed. I'm thinking that the greatest part of the noise is charge injection from the switching of the Instrumentation Amp in the DAQ. . . That is a pet theory not backed by a great deal of evidence.

My current range is from 0-250 mA.

I have written an histogram, that's why I know the percentage of the distribution of the noise. I think I'm hearing you say that I can use that to help me create some kind of filter that will give more weight to the data in the bin with the good data and much less weight to the data that is pulling my output low. I like that solution, but gee whiz, it sounds an awful lot like work. . .

I'm going to look up synchronous filters, I think I tried to hack something together to do something like what itsmoked suggested.

Thanks, again all. Just writing this stuff down like this is great way to get my thoughts together. I appreciate your knowledgable input.
-rick

 

[BrianG]

If you have another A-D channel which can sample at the same time as the sense resistor measurement channel then perhaps you could you use a subtractive technique (like Invert + Add on a 'scope)

If the second channel had the same value sense resistor located next to the real one, but without the load current going through it, then it may give a reasonable set of data which is just the noise. This could then be subtracted from the wanted channel's signal+noise leaving (hopefully) just the signal.

 

[MrBananas]

I recommend looking into a technique called "Multirate Filtering." It is a DSP technique and it sounds like you are oversampling already, which is necessary for Multirate. It is a very common technique used in these sorts of situations where you have some wild data and you need to extract the "true" waveform.

You will need a DSP book to explain it to you (too long for here), but it's all software so you wouldn't need to make any hardware changes.

If you're able to make hardware changes, I would try using a differential amplifier as your input from the current sensor. That will give you very good common mode (i.e. noise induced from VFD) rejection, sometimes on the order of 80-100dB.

 

[VisiGoth]

osbornera said "The frequency content is mostly the carrier frequency of the VFD 6kHz, but my noise floor seems to be below -60db which I thought was pretty good."

If this is gathered after your A/D then you know the energy is low enought on average. Many comments above talk about filtering out the noise, which assumes the signal and noise are in different spectrums.

You must Nyquist sample at a rate high enough for your signal and you noise, not just your signal. So, you either have to have your analog Nyquist filter low enough with enough rejection, or you have to sample the interference at an adequate rate and DSP it out (freq filter). So if you VFD has important hamonics out to 60 kHz (likely) then you want to sample above 2 * 60 kHz / 0.8 (perhaps). Since you are not doing that you might try another trick.

If your noise is more impulsive in nature, you might be able to sync to it and only use the time samples that are not as corupted by noise.

You 8 kHz sample rate is simply not enough for even this trick, since you will want to only through out a small % of samples you will want a sample rate much higher than 6 kHz.

You r best bet seems to be the elimination of the interference using analog (BrianG and MrBananas gave good advice). Multirate is only an efficient technique, and it only works if you have a sample rate high enough (hundreds of kHz).

 

[itsmoked]

BrianG That's a novel idea sample The Noise and subtract it... But you would need a simultaneous noise and desired sampling converter.

 

[sreid]

The noise from VFD's (and switchers in general) is "Impulse Noise" due to the current spikes that happen at every switching edge. I'll bet your problem is common mode noise that drives both the differential inputs above and below their common mode range. Add clamp diodes to the signals and a common mode choke.

 

[Skogsgurra]

Another couple of cents...

You are certainly oversampling. If you have a triangle waveform at 0.125 Hz, then you probably are not interested in any component higher than 20 or 30 Hz. Triangle wave harmonics fall off as 1/(f^2) so leaving out frequencies above 12.5 Hz means an error less than 1(100^2) or 0.01 percent. Hard filtering (symmetrical!) at the DAQ card input seems to be the obvious way out.

On the other hand, the noise from the VFD is not easy to filter out. It contains HF components that travel through air quite easily. Using a shunt resistor has proven impossible in commercial inverters and is probably even more so in a limited production circuit. The solution is to measure with a hall element. There are many manufacturers; LEM has complete units that come calibrated and Allegro has units that you have to mount and calibrate yourself. I would use an LEM. They eliminate all problems you are having.

Gunnar Englund

http://www.gke.org

 

[osbornera]

My reason for oversampling is that my sampling frequency has a direct effect on loop execution time. I'm monitoring multiple analog input channels and I need to know, for example if I reach an overpressure condition so I can shut down quickly. I also need to stop sampling to do other things (possibly because I'm not an expert programmer or I don't understand the full capabilities of my DAQ). So I take a few samples, filter them, compare them to some limits, monitor my digital inputs, change my digital outputs if necessary, rinse, repeat.

If I sample at 100 S/s and take 10 samples, I've blown 100 msec and that seems like a long time to me. If I had less noise I wouldn't need so many samples. Also, if I knew the rate of change to expect for an overpressure condition that would be nice . . . Some of the unknowns for this system are probably even calculable, but I can't justify the time I would need to investigate every unknown. So I approximate, theorize, test, modify, ask, curse, rework, . . . I'm sure we all go through some version of this process.

On a Monday morning I realize that I'm feeling a little defensive/defeated as I try to summarize my "part time" project that I've spent almost 5 months of work on in the last year . . .

I love the hall effect sensor idea, I just might do that . . .

Thanks, again for the input. I appreciate the collegial tone of this forum.