Sorting out noisy signals 4

[Danielsp]

I am a Civil Engineer who's just started in the field, so please bear with me here. It's quite simple stuff.

How can we separate different frequencies in a signal in such a way that the information modulated into a given frequency is preserved? I mean, if you use a passband filter around a target frequency, you average out many wave cycles of that frequency and all the information stored there is lost.

At least that's what happens with a FIR filter (I've tried) and I suppose IIR filters would have the same effect. So, is there a different kind of frequency filter that keeps modulation intact? I tried a PLL as well but the net effect is pretty much the same. How is that problem solved in real life?

Thanks in advance!

 

[GregLocock]

Excel's FFT is pretty horrible, if you are new to Fourier you are better off learning to use a proper language (Matlab (not free) Python (free) Octave(free)), or in the simplest case, Mathcad or SmathStudio, which are free, have the shortest learning curve and are pretty.

Ideally you'd give us a sample of the data you have in numerical form and what you hope to extract from it.

Cheers

Greg Locock


New here? Try reading these, they might help FAQ731-376

http://eng-tips.com/market.cfm?

 

[Danielsp]

I couldn't figure how to quote and still I'm trying to pull some complicated stuff...

Once again, if you want to to recover a modulated signal, then you need a passband that is at least as large as the largest frequency in the modulation that you are unwilling to sacrifice. The fact that you insist on using rectangular waveforms for the modulation means that you need something like 10x the carrier frequency for a plausible bandwidth.

Right again, IRstuff! So my modulation of choice was not fit to my filter (or vice-versa). That is a very useful piece of information, thanks!

Let me give you some more info. My sampling rate is 44100Hz and I chose somewhat arbitrarily the carrier at 4900Hz (exactly 1/9 of the sampling rate). I'm dealing with sound being broadcast across a few meters around people and typical day-to-day noises, so I'd like the frequency to be as high as possible to be removed from the most important interferences, but at the same time I'd like to be as far away as possible from the 22050Hz limit so as not so sacrifice resolution. For now, 4900Hz is the compromise number, which we are free to change.

Back to our problem. I chose the amplitude rectangular modulation because it is easy to implement. But if my passband covers the 10x span from 4500Hz to 45000Hz (ok, as high as 22050 max), then I'll get all the unwanted noise above the carrier frequency, and I can't afford that.

All things said and done, it seems I need a modulation with the narrowest possible range of content in the frequency domain, right? Now, what would that modulation be?

 

[Danielsp]

If you're doing this on a computer, as a simulation, then you should see if you can include an FFT module (sometimes called a 'Tool Kit' in MS-Excel) to allow you to *see* (with your eyeballs) the same information in the frequency spectrum domain. If you don't do this, then you're working blind. If you can do it, then you'll find that everything will fall into place much quicker.

In the RF world, one of my quips is that, "If RF were purple, then it wouldn't be considered magic." In other words, once you can see it, it's easy.

Same sort of thing applies when trying to do signal processing. You *must* work in the frequency spectrum domain most of the time, only occasionally referring to the amplitude versus time domain as required (example, pop-noise filtering where they're more visible in the amplitude plot).

Yep, that's great advice, thanks! I just started some tests on Excel, which is my weapon of choice.
If I had done that from the start, I wouldn't have needed the insights IRstuff just gave me.

 

[Danielsp]

Excel's FFT is pretty horrible, if you are new to Fourier you are better off learning to use a proper language (Matlab (not free) Python (free) Octave(free)), or in the simplest case, Mathcad or SmathStudio, which are free, have the shortest learning curve and are pretty.

Yeah, maybe I should change. But I've been an Excel guy for such a long time...

Excel's FFT is pretty horrible, if you are new to Fourier you are better off learning to use a proper language (Matlab (not free) Python Ideally you'd give us a sample of the data you have in numerical form and what you hope to extract from it.

The data is going to be just a few bytes, a positive integer up to, dunno, 10^9.

 

[IRstuff]

Rather than try to bash some arbitrary solution into fitting your problem, perhaps you should start by describing what you functionally want to do, i.e., send how much data, how fast, how often, and how far. Then, and only then, perhaps you could consider looking at modem technology and how people were able to transmit up to 33 kbps over phone lines with a worst-case bandwidth of 3 kHz.

TTFN (ta ta for now)
I can do absolutely anything. I'm an expert!

https://www.youtube.com/watch?v=BKorP55Aqvg

faq731-376 forum1529 Entire Forum list

http://www.eng-tips.com/forumlist.cfm

 

[Danielsp]

Rather than try to bash some arbitrary solution into fitting your problem, perhaps you should start by describing what you functionally want to do, i.e., send how much data, how fast, how often, and how far. Then, and only then, perhaps you could consider looking at modem technology and how people were able to transmit up to 33 kbps over phone lines with a worst-case bandwidth of 3 kHz.

Yeah, I used the 33kbps phone lines. But remember we had very little noise there! The need for filters is what makes this case more difficult and much closer to radio: an environment with a lot of noise as loud as your signal and in VERY close frequencies.

So, fair enough: I would like to send some 30 bits in less than 0.02s (or so) every couple of seconds across some 10 or 20m using a sampling rate of 44100Hz. That makes 1.5kbps, which seems pretty reasonable. The times I set are first approximations deriving from the fact that there could be multiple senders (maybe 10 or 20) which are impossible to coordinate, so their messages are spaced out across time (probably at random intervals up to a couple of seconds) in a way they don't superpose (too often). It's the only time-share solution I can envision in a situation you can't sync the senders.

So, I didn't do the math but I figured that if you send 30 different signals that are 0.02s each at 2 or 3s varying intervals, there is little superposition and in 5s you probably had at least one instance of most signals not superposed to another. The limiting factor here is obviously how long I am willing to wait to get most signals right, which I set in 4 or 5s. Of course, we could use 0.03s or even more if we have to. But I am hoping we don't.

 

[VEBill]

"... I would like to send some 30 bits in less than 0.02s (or so) every couple of seconds across some 10 or 20m..."

There are plenty of off-the-shelf solutions for wee little radio links to perform such functions. The applicable modules are typically $1 or $2 from sellers on eBay. There's a whole ecosystem of this within the embedded "Arduino" community.

Edited to add: Andreas Spiess "The guy with the Swiss accent" on YouTube seems to specialize in this topic.

 

[GregLocock]

https://computers.tutsplus.com/tutorials/building-a-wireless-sensor-network-in-your-home--cms-19745

etc

You could assign a different channel to each sensor, or you could increase the complexity and actually ping each sensor in turn and get it to answer back.

I have got half a dozen Arduino wifi boards, but the project I bought them for is fairly low priority, so apart from a quick look to see what they were like (they need aerisls for my application) I can't tell you much about the practical side.


Cheers

Greg Locock


New here? Try reading these, they might help FAQ731-376

http://eng-tips.com/market.cfm?

 

[Danielsp]

There are plenty of off-the-shelf solutions for wee little radio links to perform such functions.

But I'm doing it over sound. How could I apply that?

 

[Danielsp]

You could assign a different channel to each sensor, or you could increase the complexity and actually ping each sensor in turn and get it to answer back.

But it is supposed to be a two way communication. All parties should be able to communicate with all others.

 

[MikeHalloran]

Over radio links, you could use TCP/IP, DNS, and DHCP.
... i.e., much of the hard networking stuff is already done.


Over sound links, maybe you could use time division multiplexing.
Divide each second into, say 32 periods.
Within all but one period, only one of up to ~30 transmitters can make noise; all listen.
Each sent message/squeak includes a few bits to designate the intended receiver,
a few bits to encode a message,
and a few bits to encode a checksum.
One address is for broadcast, message to everyone.
One timeslot is silent to synchronize everyone.
Everyone must send _some_ message within their designated time slot,
to show that that time slot is occupied and active,
so there must be a 'null message' for that purpose.
Identities 0..30 are preassigned to each unit.

You may need to add some fudge code to get all the transceivers into sync at startup,
like changing time delays from an arbitrary starting point until everyone receives
a complete cycle of messages with no checksum errors, i.e., no one is transmitting
when he shouldn't be.

Or equip everyone with radio receivers and use WWV for sync.

The above outline represents the core of something that might work,
BUT, if you are doing this among people,
expect complaints of annoyance about the buzzing if they can hear it,
and complaints of general malaise if they can't.

It's also going to be more expensive and more difficult than using stuff you can buy.



Mike Halloran
Pembroke Pines, FL, USA

 

[GregLocock]

Well the latest revelation rather knocks the entire project intio a different ballpark. If you're you have 2 way coms then collisions aren't an issue. I'm out of here until i see some sample data and a decent spec.

Cheers

Greg Locock


New here? Try reading these, they might help FAQ731-376

http://eng-tips.com/market.cfm?

 

[Danielsp]

Over sound links, maybe you could use time division multiplexing.
Divide each second into, say 32 periods.

Interesting ideas, Mike. But I'd have to be able to sync everybody with a given clock first, which I'm not sure is possible. Anyways, time multiplexing is solvable. But I'll never get there if the signal doesn't make it through the filter, which is where I am right now.

 

[Danielsp]

Well the latest revelation rather knocks the entire project intio a different ballpark.

Well, if y'all wanna help me along the whole project, that would be great! But I wouldnt ask that. All I'm asking is how to make a signal survive going through a filter.

If you're you have 2 way coms then collisions aren't an issue.

I'm sorry, what are you talking about?

I didn't talk about the whole project before because most of its features are irrelevant and I thought those information would throw people away from the one thing I came here asking for help - and that's exactly what is happening now.

The fact that it is 2way com doesn't change the problem at hand: how to extract a signal in a noisy environment since filters tend to distort it. I'm guessing that TVs, cellphones and radios have filters with very little distortion because you can afford to use thousands of wavecycles to produce a single bit of information, so modulation functions can be very, very gradual, with derivatives very close to zero, thus being almost perfect sines at short intervals, and those kinds of signals have almost no sidebands and will go pretty much undisturbed through a filter.

In other words, I tried rectangular modulation functions with frequency once or two times the carrier frequency. That created huge sidebands, far outside my passband. On the other hand, TVs, cellphones and radios have gradual modulation functions with frequencies that are at least 2 or 3 orders of magnitude inferior to the carrier. Right now, I'm guessing that's where my problem lies. If that is true, the solution is to use more gradual modulation functions (and lose A LOT of bandwidth, of course).

I was hoping I could have 1 or two bits per wave. But apparently that is too much to ask.

 

[IRstuff]

"I'm guessing that TVs, cellphones and radios have filters with very little distortion because you can afford to use thousands of wavecycles to produce a single bit of information, so modulation functions can be very, very gradual, with derivatives very close to zero, thus being almost perfect sines at short intervals, and those kinds of signals have almost no sidebands and will go pretty much undisturbed through a filter."

No, that's not what they do. They do a systems engineering analysis, come up with rational and sound design requirements, and THEN, they design.

You keep harping about noise, but you have produced zero information about it. How can you expect to come up with a workable solution if you don't provide ALL the relevant design specs and environments?

TTFN (ta ta for now)
I can do absolutely anything. I'm an expert!

https://www.youtube.com/watch?v=BKorP55Aqvg

faq731-376 forum1529 Entire Forum list

http://www.eng-tips.com/forumlist.cfm

 

[MikeHalloran]

But I'd have to be able to sync everybody with a given clock first, ...

Read my msg again; I outlined two distinct ways to get the clocks in sync.



Mike Halloran
Pembroke Pines, FL, USA

 

[Danielsp]

IRstuff, I dont have any formal and/or fixed "specs"! That's just a personal project of mine. All I know about the noise is: it's environment sound. Could be virtually anything, but in all likelihood, it's just any public environment.
What relevant info is there about this that is needed to solve the problem? I really don't know!

 

[IRstuff]

The performance of such a system is dependent on the SIGNAL TO NOISE RATIO (SNR); if you don't know the noise, then you can't properly design the signal. You are basically wasting people's time with an ill-formed problem.

TTFN (ta ta for now)
I can do absolutely anything. I'm an expert!

https://www.youtube.com/watch?v=BKorP55Aqvg

faq731-376 forum1529 Entire Forum list

http://www.eng-tips.com/forumlist.cfm

 

[Skogsgurra]

You started with this: "I am a Civil Engineer who's just started in the field, so please bear with me here. It's quite simple stuff".

A few (quite a few, actually) tried to help you on the right track. But not easy with the very vague specifications that you presented. A moving target, as it turned out to be. My answer: "Shannon and others formulated that a long time ago. It is about band-width of your filter. Modulation causes sidebands. Simple in AM and somewhat more complex in FM". Was an attempt to make you understand the basic concepts. Did you google Shannon and his basic work?

If you think that "It's quite simple stuff" then you may have to rethink.

BTW, I think that I shall pick up bridge designing. There can't be much to it. Seems to be quite simple stuff. The Romans built the Pont du Gard and medieval masonrys built Cathedrals. So why shouldn't an electrical engineer like me start designing bridges? I wonder.

You may have noticed the little flag "Report" in the lower right corner. Well this thread is very close to being just that. And disappear.


Gunnar Englund

http://www.gke.org

--------------------------------------
Half full - Half empty? I don't mind. It's what in it that counts.

 

[MacGyverS2000]

Gunnar, you didn't do any recent work in Florida, did you?

Dan - Owner

http://www.Hi-TecDesigns.com