Finding optimum

Many problems in life can be described as "Optimization" problems.  That includes many problems in the engineering, sciences and computational fields.  It should come as no surprise that the geeks (raising my hand, here) have given this "optimization" problem a great deal of thought.

The Optimization problem is a search problem.  Generally, information has a cost and management wants to maximize goals while minimizing cost and time spent.

Complicating the problem is that fact that there are "permissible" regions and "not permissible" regions.  For example, if you were trying to optimize fuel economy of a vehicle, a zero-to-sixty time of more than 8.0 seconds might be "not permissible" due to lack of consumer acceptance.  Interesting problems will have many, intersecting "not permissible" constraints.

Optimization geeks often use the analogy of a blind man finding the highest elevation in a golf course.  The blind man will use his cane to determine the slope of the ground and then he will walk in the steepest, upward direction for a few steps.  Then he will use his cane to determine if the slope changed and then walk in the (new) steepest direction.  He will plant a flag when his searching with a cane tells him that all directions lead downhill.

Gold star denotes highest point.  Permissible region is the area inside the red triangle.

Where the landscape is simple, the highest elevation often occurs where two or more constraints come together.  Optimization programs, consequently, will often start their search by checking all the constraint vertices.  An example of this in automobiles is that the option with the lightest weight, the smallest engine and the most transmission speeds is likely to have the best fuel economy.

Another example would be a lost child.  In the presence of a northwest wind, they would likely drift downwind until thay came to a stop at either of the two circles due to the intersection of two, physical barriers.  A fellow could save a lot of time if he had the resources to check out the red circles.

Because of the frequency that optimum solutions are found in the vertices, it is absolutely critical that the boundaries between the permissible and not-permissible regions be defined with great care.

Sometimes life is a bit more complicated and the optimum solution (the highest point) does not occur at a vertices but along the length of a constraint.  This possibility can be checked out with a handful of "depth soundings" along the length of each constraint.  An example of this would be that the optimum spouse might be physically attractive but not the absolute, maximum prettiest girl.

Sometimes the topography and constraints are very complicated and the optimum solution is in the interior of the "permissible" region.  This is a challenging problem because our blind man will converge to local optima rather than the true optimum depending on where he starts.  For instance, if our blind man were dropped into the closest sand trap he would converge to the base of the peninsula that protrudes into the sand trap from the top of the photo where-as the optimum is the star.

This problem is addressed by running the optimization routine many times, the difference being that our blind man starts at several different locations.  For example, you might find the absolute, optimum spouse at a dance, a laundromat, a church, a college classroom, a bar or an Appleseed shoot.  The diligent searcher should exhaust all venues and compare results.

An even harder optimum to find is if the tops of the trees are "permissible".  The chances of our blind man stumbling into a tree is very remote because the tree trunk is not very large.  Yet the top of the tree is the highest elevation inside the red polygon.

Those "trees in the golf course" often seem to be the result of synergistic behaviors of otherwise, predictable inputs.  There are a lot of ways to combine organic materials and nitrogen salts.  Only a few of them result in gunpowder.  The trees are often found by the two-guys-in-a-garage because they have the freedom to look in unlikely places.