How to challenge technical solutions in an effective way

[rotw]

Hi,

I posted in the forum under this category considering this topic is still about getting better and improving.

I want to ask you to share some methods, tricks that you use in order to challenge the engineering work you make. This apply in general and to any kind of engineering duty. I dont want to limit it to a discipline in particular or a specific matter so that I can broaden the range of advices and orientations that I could receive here (means civil, mechanical, geotechnical, etc all disciplines are welcome).

Suppose you have reached a final status for an answer to a technical problem, you normally would have sufficient confidence in that answer to deem it acceptable. Eventually you strive to check it, against your experience and expertise, use your self critisism and take advice from specialist etc before to deem it final. But lets take this one step further.

The point here is that there are blind spots in your concept or answer that you dont catch by definition. It is epistemic. Means you bring on board error or weaknesses into the solution. These may impact or may not impact tangibly the rest. For example, it may simply turn out later (say you got a feedback from the field) that this point and that approach could have been thought in a better way.

So there is natural cognitive limit that prevents you from seeing ALL the blind spots, otherwise you would have considered them of course. I think this is particularly relevant when the system you analysis is of increased complex.
I beleive that one of the reason that increases the blind spots is that you are the owner of the solution and therefore you tend to accept it by instinct, so you have to pay extra efforts if you want to challenge it.

So how to submit a solution you deemed acceptable to a serie of tests. How to think outside of the box and submit your solution to a serie of stress, say random unexpected scenarios. Let say we have sense for only 2D vision and we need to see the object in 3D.
You may start then from a totally different standpoint, without any apparent connection to the subject. Then you work out from this standpoint untill you create a new situation that will bring a light to the subject and reveal the 3D contours and the blind spots.

I would appreciate if you can share any methodology you use in your daily job to submit your solutions and answers to tests and challenges. Let me make it clear: I am not refering to a process that happen during the elaboration of the solution, but a process where once the solution is fixed, you switch from "designer role" to "tester role" and keep a searation wall betweem the two.

Thanks for any ideas or experience you can share.

 

[MikeHalloran]

The US FDA requires "regression testing" for software and I think for other stuff.

What that basically means is that you are required to maintain a database of problems that you have encountered in the past, including the result of root cause investigations and a record of the specific circumstances that triggered the problem. ... AND you are required to test every new piece of software against every problem that you have ever encountered before. If you release product with the same problem twice, you will be explaining why it happened until the day you die.

Too many manufacturers in nonregulated industries fail to do something similar, so they have cyclically recurring problems they have had before. There is really no defensible excuse for that happening, but it does. It is especially a plague in outfits with high turnover, obviously because nobody gets a chance to become an old-timer, much less take the time to record whatever they have learned along the way.

In your generalized situation, that means listening, intently, to the old timers in the office and in the shop, and encouraging them to tell you about their history with the company, and making a record of what they have said, and of reverse- engineering old products before you design new ones, so as to avoid repeated mistakes.

The best you can hope for is to only make new mistakes, like no one has ever seen before.





Mike Halloran
Pembroke Pines, FL, USA

 

[rotw]

I found this interesting article

http://www.uvm.edu/~cems/explore/holistic.pdf

There is reference to the Golden Gate Bridge. It is said that while the bridge was a marvel that has exceeded its design specs, nobody foreseen that it will be the place where nearly two suicides per month, on average, will occur. Authors are stressing the need for what they call "holistic engineering" approach.
I think this is also quite relevant to the topic.

 

[IRstuff]

You may consider the example of TRIZ, which is a system developed by Genrich Altshuller as a codification of all possible approaches to inventing something. All the different physics approaches are identified. All means of implementation tradeoffs are identified. For any given problem in TRIZ, you have essentially a checklist of things to consider as possible solutions for your problem.

see:

http://www.opensourcetriz.com/

Likewise, I expect that you can do the same for any subdiscipline of engineering, in general, or mechanical engineering specifically. Obviously, as in TRIZ, this requires discipline to explore every item on the checklist in an open-minded fashion and not dismiss things out of hand.

TTFN
faq731-376

Need help writing a question or understanding a reply? forum1529

 

[rotw]

Thanks Mike and IRstuff.

Interesting points.

 

[KENAT]

Not clear if by tester you actually mean conducting tests or if you mean objectively reviewing the design when you think it's complete.

Depending how much time & effort you're willing to put into it and how much you like to 'design by checklist' you could do comprehensive FME(C)A and DFx (where x can be manufacture, assembly, service, test...) and similar.

In terms of what testing to do, first is obviously to test performance line items explicitly listed in the requirement - plus any implicit ones relating to regulatory/code compliance etc. Then potentially items from the FMECA that couldn't be designed out.

Past data of whatever form can be very useful as Mike says. Definitely on any data on warranty repairs of high volume spare parts or just word of mouth from the guys in the repair shop...

A design review done properly with colleagues can be beneficial, getting the balance between breadth and depth isn't always easy though.

Think about it from the point of view of every one that touches it: planning/purchasing who order the parts, the guy that machines the parts, the assembler that puts it together, the person that unleashes it on the customer, the end user, the service guy...

Posting guidelines faq731-376

http://eng-tips.com/market.cfm?

(probably not aimed specifically at you)
What is Engineering anyway: faq1088-1484​

 

[rotw]

KENAT,

Perhaps testing is the weakest form of the process I refer to, anyway I agree with your point.
So basically when you find the fix to a problem, you start a new process that is to prove that there are conditions not foreseen for which the solution does not work at all, does not work effectively or work adverse.

Testing - in my view - must prove that the equipment, product or any deliverable will perform in real conditions like expected on paper.
The process I refer to should prove that what was (traditionnaly) assumed to be the real conditions was an optimistic perception of reality. That optmistic perception cannot be revealed because of cognitive limitation of analytical reasoning, but can be screened by some other methodology.

Just a possible idea, a Monte-Carlo simulation, that uses Random variables to catch a combination that could not have been foreseen analytically.

Back to the DDT problem, nobody has foreseen that it has downside effects (more than upside), not because of lack of integrity of the inventor, but because it is part of the cognitive blindness. It took too long, decades, (and again a random laboratory experience) to assess the poisonous/destructive impact of pesticides. Possible reason is because the efforts were focused on problem solving, and not understanding what the reality was all about.

But such kind of approach, would be antagonist with making business, as it could require more scientific efforts, probably more expensive will be the solutions to implement, and leading ultimately to poor competivity.

Sometimes, the reality is stretched and it is done intentionnaly. Take the example of Plant decommissioning. Some industries segment use facilities which are completly cost prohibitive for a decommissioning. Often that reality appears later and the cost of decommissioning is simply not foreseable and sometime even not feasible by any existing technology.

Ultimately, it could turn out from all this, means the intention and the commitment to broaden the range of the possibilities, would lead to discovering some new technologies. It might be the case that we are locked in paradygms and we even not know (example: energy, pharmacy, oil exploration, agriculture, access to water, access to food) that is on a global scale. But my point I have to admit is on a more individual level, that is to say, how to challenges our daily work to identify blind spots.

 

[hendersdc]

Peer design reviews prior to release usually catch something. At the very least it forces you to explain the details of the design to someone else and walk though the requirments and use cases. It can bring to light something you missed or bad assumptions you have made. Sometimes you just need a fresh set of eyes from someone that has no stake in the design.

Doug

 

[IRstuff]

TRIZ is an example of the formalism it would take to do things well, and repeatably. Altschuller spent a nontrivial amount of time to collate the data he ultimately used to create TRIZ.

Likewise, for DDT, an extensive analysis would need to be done to even have started to assess its potential effects on insects, animals, and humans. Not mention the fact that when it was used as an insecticide in 1939, we knew nothing about the genetic underpinnings of all creatures, and even less so about long-term effects of poisons.

TTFN
faq731-376

Need help writing a question or understanding a reply? forum1529