Artificial intelligence algorithms 1

[rotw]

It is my understanding that artificial intelligence is based on various type of algorithms and that there are a variety of technologies currently competing in the industry, examples: artificial neural networks, genetic algorithms, fuzzy logic, etc. and combination thereof.

Now if I consider a basic example, such as artificial neural networks (e.g. classification problems, image recognition), it is known that these systems are subject to a learning phase (calibration to a certain set of data), once calibrated it can be exploited in industrial applications. So if an intelligent system is trained on a set of data it is expected that the system would extract a certain pattern or signal/information within that set data set. When applied outside of the training data set, subject to a certain limits, the system would return some sort of exploitable information.

Now I would like to take a very simplistic situation.

If we know the response of a system to A and B according to a certain pattern (say we have a linear and continuous transfer function), then at for a certain condition C (somewhere between A and B), we know that the corresponding response would be bounded and it shall be within system responses to A and B. If the behavior is very slightly non linear, we would expect a non linear response in accordance (not a change of order of magnitude for example), so forth and so on.

Where I am heading to is this, if we use an artificial intelligence system to predict or anticipate a response to a certain condition, can there be a "mathematical proof" that the information returned remains "bounded". In other words, what warranty do we have that any arbitrary system would not behave in a completely unexpected manner in terms of output when the input itself is within the operability limits set for the artificial intelligence system?

This is an excuse to ask a more broad question (and sorry if its very generic), nowadays we hear a lot about artificial intelligence systems which are quite present in our daily life, devices etc., so is there a mathematical foundation that underlies the design and deployment of these systems or is this all "trusted" based only on empirical validation ?

For the experts, please forgive my ignorance on the subject. This is more to trigger a discussion and learn, thanks.
I also posted in this section of the forum considering it is a future technology trend, but Ok please feel free to correct if this is inadequate.

 

[GregLocock]

No, if the particular set of inputs were not in the training set there is no guarantee that a neural net will not deliver an output outside the bounds of the outputs from the training set. Coniser a half sine wave. If you train an NN on the values at 0, pi/4, 3pi/4 and pi then the maximum output from the training set will be 0.7 If the NN is fed pi/2 it will hopefully return 1.

Cheers

Greg Locock


New here? Try reading these, they might help FAQ731-376

http://eng-tips.com/market.cfm?

 

[rotw]

GregLocok,

Does this apply to neural net only or is the case to other techniques as well?

Also what I meant by boundary of training set, may be (or may not be) slightly different, so I wanted to clarify.

There certainly is a boundary for the training set, but there should also be bounds within the training set. I explain.
Lets take a certain arbitrary transfer function that governs the behavior of my system - and say I want to make an interpolation using an artificial intelligence system like a neural net.

Assume I could determine mathematically (using tricks or whatsoever) that the response shall be within a minimum and a maximum although I cannot calculate what the value is exactly. In this case, if the response returned by the neural net is within the min and max, it is bounded - so this is what I meant.

In other words, I could be operating the neural net well inside the boundaries of the training data set, but locally the artificial neural net response could behave out of bounds - but if these have not been determined mathematically, there is no way I could figure out.

I thought this can have far-reaching consequences, because if a system is implemented and becomes autonomous in a life threatening decision, that system would certainly be fulfilling its function until it will not...so what justifies the fact that we trust it, because no matter how much training data you put in, at the bottom, there always will be "gaps" that we can never be sure of.

Not sure if the question here makes a real point and any sense in real world applications.

By the way, this is also quite different debate (if any) than the classic one of the artificial intelligence taking over control of humans (for the better or the worse).

 

[IRstuff]

AI is not just a set of algorithms, it's a actually a set of different technologies that might have little or nothing to do with each other, and might have no direct cross applicability. The simplest form of AI is algorithmic, mostly using fuzzy logic, and provide basic control systems. Expert systems may or may not be algorithmic, and are used primarily to answer questions, the way Siri does. Artificial neural networks (ANNs) at their simplest are categorization tools, is this thing A, or is it B, or is it C, etc. ANNs are used in many image processing systems to find items of interest within the image, such as facial recognition systems. As a general rule, ANNs cannot, by themselves be used for control functions, but can provide inputs to control systems.

Lane marking/boundary identification would be a good fit for an ANN, BUT, it's the control system that determines what the car should do if the ANN finds nothing or sees conflicting data that it cannot resolve. However, one cannot completely depend solely on that sort of system to maintain lane position, as the Mountain View fatality proves. The control system can use the ANN input, but must also use its other sensors to produce a complete answer. It's pretty clear from what one rather foolish driver proved, is that the Tesla Autopilot blindly followed the strong lane marker on the left and followed it into the median barrier, and did nothing beyond simply warning the driver of the impending collision, even though the maneuver to avoid the barrier would have been well within its normal capabilities to station-keep within a lane.

In any case, an automated car would require ANNs for sensor processing, control law algorithms for controlling the speed and steering, and some sort of fuzzy logic/expert system to run the system. And, none of these systems are interchangeable, they do fundamentally different things, but they all tend to get lumped into the AI category.

TTFN (ta ta for now)
I can do absolutely anything. I'm an expert!

https://www.youtube.com/watch?v=BKorP55Aqvg

faq731-376 forum1529 Entire Forum list

http://www.eng-tips.com/forumlist.cfm

 

[rotw]

Ok thanks for the information, which I read carefully. So that's good.

I put the original problem differently. If I build a numerical / computational model of a physical problem.
I could compare the model to experimental data to see if it is matching. Supposedly I would have validated the model.
But is this really a validation ? How do I know that there are no areas of operation or combination of variables that would lead to erroneous results? Gather thousands of experiment data and invest in many test benches? still real world applications could result in set of condition which has not been screened. What if I approach the physical problem with a simplified equation that I can solve and that could provide "limit cases" or bounds to my problem (e.g. pressure would always be limited to 1 bar, temperature cannot physically be outside 250 - 300 K range etc. - just arbitrary example of course). So If I prove that the numerical model results are within these limits I would have established on mathematical ground (or analytically if you prefer) a confidence of the model.

This a different subject than AI, but I wanted to use just to illustrate a concept. So now back to artificial intelligence, when a system is designed / deployed - do we have the framework (and actually do we do that) to ensure that the response of the AI system shall not make at any moment an excursion totally out of control / not designed for, so that ultimately the system response (or resulting action) remains always contained within certain upper/lower limits pre-established with certainty.

 

[GregLocock]

Perhaps a good example is the ESC in a car. The ESC takes the driver inputs and models what the desired outcome is. It then compares it with the cars actual behavior, and then models the effect of applying the brakes to each wheel, and tries to fit the assumed driver request to the actual vehicle response+brakes. It does this iteratively.

At a very crude level, it assumes that if the steering wheel is turned further to the left, the driver wants the cars path to veer more leftwards. In a non ESC car the driver may be loading the right hand wheels up preparatory to a flick. The ESC doesn't understand that. It doesn't need to for normal drivers. Driving a car quickly with ESC on is more of a case of figuring out what inputs the ESC understands and that don't trigger the default "I don't know what is going on, light braking on all wheels and no throttle" that seems to occur.

Cheers

Greg Locock


New here? Try reading these, they might help FAQ731-376

http://eng-tips.com/market.cfm?

 

[GregLocock]

Our replies overlapped. I'm a slow typist.

"when a system is designed / deployed - do we have the framework (and actually do we do that) to ensure that the response of the AI system shall not make at any moment an excursion totally out of control / not designed for, so that ultimately the system response (or resulting action) remains always contained within certain upper/lower limits pre-established with certainty. "

Yes and no. The control system takes the AI input and then applies its own limits. So if the AI requests -4000g braking on one wheel, one way or another the control system will interpret that as maximum effective braking.

Cheers

Greg Locock


New here? Try reading these, they might help FAQ731-376

http://eng-tips.com/market.cfm?

 

[MikeHalloran]

The ESC on my '13 Navigator has a selectable setting for Trailer Sway Control. I leave it on because I sometimes pull a trailer that could weigh more than the Navigator does.

It has triggered once, while the trailer was actually misbehaving rather badly.

At that time, the trailer was equipped with the $800 POS equalizing hitch that the trailer dealer's 'expert' selected, and a single friction damper. The damper manufacturer actually suggested two for a trailer like mine, but the dealer did not choose to install two. They're crap anyway, always squeaking and groaning, and you have to disconnect them to back up.

Anyway, the ESC got the trailer under control, but the ESC response was disturbingly violent; it braked one of the rear wheels really hard. ... not something I could do anyway, but it clearly made the difference.

That was the umpteenth incident of the trailer getting into a yaw oscillation that became a roll oscillation in response to a passing tractor trailer. It also wiggled badly when a Fiat 500 passed; no need to look in the mirror, I could feel everything. Yes, a few times the roll was hard enough to put two wheels (out of four) off the ground.

That's when I decided to buy the $2400 ProPride Projected Pivot Point hitch, which uses a four bar linkage to force yaw rotation to be centered near the tow vehicle's axle, not its bumper.
To civilians, I just say it's a damn miracle, worth every penny. Last summer, with the new hitch, I had a whopping bad blowout on a trailer tire, and didn't even know it until a passing trucker waved at me to stop. Removed the old hitch and the sway damper, but I'm reluctant to sell or give them to anyone else because they behave so badly. If it were my decision Propride 3P or Hensley Arrow or Centerline hitches would be mandatory for bumper pull trailers.

Anyway, the ESC and the TSC work well enough, for what is, relative to Autopilot, a damn simple job. But they only have to worry about the dynamics of one (sometimes articulated) vehicle, and they don't have to concern themselves with anything outside of the vehicle.

I personally have programmed microcontrollers to express behaviors that confounded some pretty smart test equipment. The behaviors were fundamentally simple, but included some self-learning. Modern ECUs and ESC systems are way more advanced, but I'd guess the autopilots will need another five+ years of testing before they're really acceptable, even barring legislative interference/ machinations.







Mike Halloran
Pembroke Pines, FL, USA

 

[GregLocock]

Braking a single wheel is SOP for an ESC, the other alternative is for it to grab the steering wheel, which is rather more off-putting.

Cheers

Greg Locock


New here? Try reading these, they might help FAQ731-376

http://eng-tips.com/market.cfm?

 

[IRstuff]

"But is this really a validation ? How do I know that there are no areas of operation or combination of variables that would lead to erroneous results? Gather thousands of experiment data and invest in many test benches? still real world applications could result in set of condition which has not been screened. What if I approach the physical problem with a simplified equation that I can solve and that could provide "limit cases" or bounds to my problem (e.g. pressure would always be limited to 1 bar, temperature cannot physically be outside 250 - 300 K range etc. - just arbitrary example of course). So If I prove that the numerical model results are within these limits I would have established on mathematical ground (or analytically if you prefer) a confidence of the model."

You don't know. Sometimes the programmer doesn't even know. Testing is essentially a codified "art" i.e., we have procedures that we use to test code, but it's nearly impossible for any programmer to exhaustively test a complex program. Such was the case with the Pentium FDIV bug, wherein a handful of erroneous microcode lines passed normal testing, and only someone with time on their hands or someone "lucky" would find the anomalies. Your example presumes that the problem is simple enough for a simple equation, which is never the case. Moreover, the real world includes noise, glitches, failures, etc., all of which need to be handled. We lost the ability to exhaustively test even simple digital chips more than 2 decades ago, because the test time would exceed the available lifetime of the chip itself.

Moreover, as I wrote earlier, the part of an AV that allows it to drive around in a city is not a clean algorithm at all. It's fuzzy, because the real world is fuzzy. And, as Tesla has demonstrated at least twice, such systems, even after millions of road miles, can still be caught with its digital pants down. But, as with your desire for a simple equation, the driving AI needs to be relatively simple, to keep the processing load down, as well as to make it understandable for the programmer. The more exceptions, special cases, etc., that are added to an AV driving program, the more exponentially complex it becomes, and things potentially start to interact and interfere with each other.

TTFN (ta ta for now)
I can do absolutely anything. I'm an expert!

https://www.youtube.com/watch?v=BKorP55Aqvg

faq731-376 forum1529 Entire Forum list

http://www.eng-tips.com/forumlist.cfm

 

[beej67]

Where I am heading to is this, if we use an artificial intelligence system to predict or anticipate a response to a certain condition, can there be a "mathematical proof" that the information returned remains "bounded". In other words, what warranty do we have that any arbitrary system would not behave in a completely unexpected manner in terms of output when the input itself is within the operability limits set for the artificial intelligence system?

It's my understanding, with ANNs anyway, that there's no proof except what's in the pudding. You don't tune an ANN to two results, you tune it to a billion results, as many as you can build in your data set, and then it tunes itself to best possibly match all results. Its predictive power would then be bounded by whether or not the inputs for the prediction were within the same general space as the inputs for the tuning process.

Does that make sense?

On a side note, I'm very surprised that nobody has sat down and done this with college football. It would seem to me to be a very distinct advantage in Vegas.

Hydrology, Drainage Analysis, Flood Studies, and Complex Stormwater Litigation for Atlanta and the South East -

http://www.campbellcivil.com

 

[IRstuff]

For the specific case of ANNs, boundedness is meaningless; one of the reasons that ANNs are used is precisely because boundaries cannot always be mathematically delineated, particularly so if the input conditions are multivalued, so we're talking n-dimensional spaces that have to be segmented. Moreover, the "features" that ANNs use aren't even necessarily what a human might think were relevant, and that's part of the beauty of ANNs. In classical supervised learning, you simply tell the ANN what you want, and what you don't what, and the ANN tries to figure out what features result in producing that desired segmentation in the training set. Self Organizing Maps (SOMs) are actually trained without supervision, i.e., you let the ANN figure out what it thinks should be segmented. Below is from a demo of SOMs on the web; the SOM takes the random scene and tries to "self-organize" it. This only involves 3 dimensions, i.e., R, G, and B, and it's already complex. The boundaries are fuzzy, and there's no mathematical process for making them less fuzzy; it's just the way nature is.

BEFORE

AFTER

TTFN (ta ta for now)
I can do absolutely anything. I'm an expert!

https://www.youtube.com/watch?v=BKorP55Aqvg

faq731-376 forum1529 Entire Forum list

http://www.eng-tips.com/forumlist.cfm

 

[GregLocock]

I just did a MOOC on ML, I found the SOM lessons a bore, but your example is rather nice. Of course one thing with NNs is that not only is their reasoning (late edit for clarity) inscrutable, even designing them seems rather umm, rule of thumby. How many layers? how big is each layer?

Cheers

Greg Locock


New here? Try reading these, they might help FAQ731-376

http://eng-tips.com/market.cfm?

 

[rotw]

Yes and I think if you put too much layers and/or the layer is too big, NN will make sure to match exactly the training set. When it gets difficult to catch all the info, the neural net would take highly non linear / curved paths and find his way to match the training data set. Downside however is that for points which are outside the training set, you could end up with strange or highly curved profile, while possibly a basic (linear?) behavior could be fit for your job. Maybe for quite complex problems, it could make sense to have such power and capability but for the rest, I guess you don't want these complications.

 

[FoxRox]

Someone please set me straight if I am missing something, but this is a discussion about "AI" as used by marketing departments more so than in a philosophical sense. Complex control schemes and predictive software don't really qualify as intelligence, right? When we talk about "learning", we're really talking about a continuous iterative calibration. Last I knew, we were still a long ways from inventing a true artificial intelligence.

 

[IRstuff]

"Last I knew, we were still a long ways from inventing a true artificial intelligence."

From a Turing Test perspective, we are pretty close, in that there are lab rat programs, bearing some resemblance to Siri and Alexa, that make it very difficult to discern whether you are talking to a machine or a person, aside from the sensorial differences.

The OP was asking about AI "algorithms" which is different that AI in general. In general, these are technically "AI" in the sense that they are artificial intelligences equivalent to amoeba-like creatures.


TTFN (ta ta for now)
I can do absolutely anything. I'm an expert!

https://www.youtube.com/watch?v=BKorP55Aqvg

faq731-376 forum1529 Entire Forum list

http://www.eng-tips.com/forumlist.cfm

 

[MikeHalloran]

<tangent>
Every few months, a new LinkedIn manager shows up,
with a pocketful of sunshine promises,
to 'help' us group managers run our LI groups.
... then said LI manager quietly disappears,
to be replaced by another with a different name and icon.
IMHO, most of them don't pass the Turing Test.
I.e., I'm just about convinced that LinkedIn has a very tiny staff of actual people,
who are busy writing algorithms that try to act as if they are actual people,
helping to run a fairly enormous website, that is evolving toward a singularity.

</tangent>

Mike Halloran
Pembroke Pines, FL, USA

 

[rotw]

Nowadays even actual managers would fail the Turing test...

 

[rotw]

By the way, if I may ask a more technical question within the same thread...
When using an ANN multilayer perceptron I am noticing that outside the training data set, the extrapolation capability of the ANN is poor/misleading. Also the nature of the activation function and shape (I use classical sigmoid from 0 to 1 in each layer) make that the curve has to flaten near the tails 0 and 1.
So when this is computed across the complete network, it compounds into a visible and awkward change of convexity of the ANN output profile near and outside the training data set.
So my question is, if I use different activation function of my own - so not widely tried and tested by technical community; basically this would be a non linear function that becomes almost VERTICAL towards the tails as opposite to the official sigmoid that flatten horizontally (reason for this choice is that intuitively it would seem more in line with the physical behavior of my problem i.e. possibly not extrapolating as it should but at least there would be no change of convexity); I would do all the maths to implement it of course, would that work or could it be that using a function which has not been tested/documented, causes erroneous bogus / behavior (i.e. reliability problem)? Bottom line: can we take the latitude to use an activation function of our own?
Otherwise, any other way to avoid the saturation problem of the network according to your experience?

 

[IRstuff]

What exactly do you mean by "outside the training set?" That could either mean that you are using a valid input from a valid class that wasn't trained on, or something that was a class that was not trained on at all. In the latter case, those have to go into the bitbucket, as the ANN didn't get trained on the unknown class, and would legitimately have no method to determine anything.

What you are suggesting essentially defeats the purpose of the training, and implies that you didn't come up with a valid training set to being with. The purpose of the training is to create non-algorithmic boundaries for what classes that you want to classify. If something is outside of those bounds, then is should be a don't care.

TTFN (ta ta for now)
I can do absolutely anything. I'm an expert!

https://www.youtube.com/watch?v=BKorP55Aqvg

faq731-376 forum1529 Entire Forum list

http://www.eng-tips.com/forumlist.cfm