Artificial intelligence algorithms 1

[rotw]

It is my understanding that artificial intelligence is based on various type of algorithms and that there are a variety of technologies currently competing in the industry, examples: artificial neural networks, genetic algorithms, fuzzy logic, etc. and combination thereof.

Now if I consider a basic example, such as artificial neural networks (e.g. classification problems, image recognition), it is known that these systems are subject to a learning phase (calibration to a certain set of data), once calibrated it can be exploited in industrial applications. So if an intelligent system is trained on a set of data it is expected that the system would extract a certain pattern or signal/information within that set data set. When applied outside of the training data set, subject to a certain limits, the system would return some sort of exploitable information.

Now I would like to take a very simplistic situation.

If we know the response of a system to A and B according to a certain pattern (say we have a linear and continuous transfer function), then at for a certain condition C (somewhere between A and B), we know that the corresponding response would be bounded and it shall be within system responses to A and B. If the behavior is very slightly non linear, we would expect a non linear response in accordance (not a change of order of magnitude for example), so forth and so on.

Where I am heading to is this, if we use an artificial intelligence system to predict or anticipate a response to a certain condition, can there be a "mathematical proof" that the information returned remains "bounded". In other words, what warranty do we have that any arbitrary system would not behave in a completely unexpected manner in terms of output when the input itself is within the operability limits set for the artificial intelligence system?

This is an excuse to ask a more broad question (and sorry if its very generic), nowadays we hear a lot about artificial intelligence systems which are quite present in our daily life, devices etc., so is there a mathematical foundation that underlies the design and deployment of these systems or is this all "trusted" based only on empirical validation ?

For the experts, please forgive my ignorance on the subject. This is more to trigger a discussion and learn, thanks.
I also posted in this section of the forum considering it is a future technology trend, but Ok please feel free to correct if this is inadequate.

 

[rotw]

IRstuff,

Say the input range used for training corresponds to the interval [A,B] (i.e. the training set).

Problem no. 1: In prediction mode, there can be input data outside [A,B] for example: [A-5% , B+5%] and I can't help it, except state validity range...(but having in practice 7 neurons in the input layer it is a lot of combinatoric).
For instance, in that interval portion ]B, B+5%] the ANN is extrapolating. It is very poor at that.
Actually the ANN takes a fixed default value but anyway that is accessory to the point I want to make.

Problem no.2: Here the ANN is presented with data still within [A,B] in prediction mode. Considering the type of activation function being used (I use logistic sigmoid) the input interval [A, B] maps into [0,1]. Because the ANN saturates before the tails, the output profile changes convexity ; so outside the interval [A+x%, B-x%], there is an awkward shape of the output profile vs. the expected physical behavior. Thus my question to adapt the activation function and use one of my own...

 

[IRstuff]

An ANN is a long way from the flexibility of a human brain at "fixing" things like this. If something is outside of the boundaries set by the training, yes, data outside of those bounds are going to suffer.

It seems to me that if you know that this will happen you can simply limit the data at the input, rather than mucking with the sigmoid curve.

TTFN (ta ta for now)
I can do absolutely anything. I'm an expert!

https://www.youtube.com/watch?v=BKorP55Aqvg

faq731-376 forum1529 Entire Forum list

http://www.eng-tips.com/forumlist.cfm

 

[GregLocock]

You are asking the ML algorithm to extrapolate. That is absurd.

Cheers

Greg Locock


New here? Try reading these, they might help FAQ731-376

http://eng-tips.com/market.cfm?

 

[rotw]

For your info certain special neural nets could "extrapolate" but its a damn heavy job to do it. The state of the art shows that by equipping the hidden layer with computational units (e.g cos(x), sin(x) activation functions) in addition to the classic sigmoid this "super class" of neural net could learn the mathematical equation underlying the response and extrapolate. So its not "absurd". It is not the purpose of ANN to work outside validity range /training set interval and we all know that yet it can be done but with additional mathematical efforts: You could educate yourself on the subject and google : equation learner neural nets so called EQL.
I was looking for something LESS LESS complicated than that and I could not go that far with my limited knowledge anyway combined with poor maths skills. I wanted to cover possibly an excursion outside the validity range -> At least avoid a change of convexity and mitigate the problem of neural net saturation. I thought I did explain this or not?

 

[rotw]

IRstuff
Quote
It seems to me that if you know that this will happen you can simply limit the data at the input, rather than mucking with the sigmoid curve.
Unquote
You are saying not to do it, but can you state say why?

 

[IRstuff]

Just the law of unintended consequences. In general, systems do not like having super-high gain, as that often leads to instability.

TTFN (ta ta for now)
I can do absolutely anything. I'm an expert!

https://www.youtube.com/watch?v=BKorP55Aqvg

faq731-376 forum1529 Entire Forum list

http://www.eng-tips.com/forumlist.cfm

 

[rotw]

Ok thats a good point to keep in mind. Thanks.