I haven’t written.

That much is obvious.

What is not obvious is the fact that I’ll continue to not write. At least for the next little while.

And why not? Did I grow bored of blogging? Bored of writing? Did I finally find myself a girlfriend and have thus come to the realization that there’s more to life than dismal loneliness and my own misanthropic existence?

Haha. No.

So what is it? What disaster in my life has reduced my blog to a ghoulish cemetery of half-hearted posts and comments?

Perhaps I could enthrall you all with tales of my 12-hour work days and sleepless nights. Or I could tell you about how I’ve been trying to tackle such and such extraordinary mathematical problem and how I’ve returned to my bed each and every night, soaked in the sour sweat of dejection and failure.

But I won’t. I won’t because I can sum up the bane of my existence in three, remarkably simple pictures.

This is an example of the waves that flow from the stern of a ship in water, calculated using numerical simulations.

This is a formula that’s supposed to predict the height of said waves — formulated using loads of technical gobleygook.

This is how they match up.

What’s the problem? Most likely, I messed up somewhere in the pages and pages which contains the derivation of the formula. Maybe a ‘2′ needs to be a ‘1′ somewhere. I’m sure it’s something totally inane.

Whatever it is, I can feel it laughing at me. Mocking my pathetic attempts to smother it out of existence.

So, erm…I’m afraid my life is on hold until I get this crap sorted.

Oh, by the way, the answers to the August Musical Challenge were:

(1) “All Is Violent, All Is Bright” by God is an Astronaut

(2) “Scully Doesn’t Know” by Humming Urban Stereo

(3) “One and One” by Robert Miles

(4) “Voyager” by Daft Punk

(5) “Pavane pour une infante défunte” by William Orbit

Story of my Life

Recently, I went to the cinema to watch Disney Pixar’s newest movie, WALL-E. A bleak, post-apocalyptic tour-de-force, the movie depicts the gentle romance between two robots of the future: WALL-E, the not-so-bright and not-so-attractive ‘guy’ with the big heart and sweet personality, and EVE, the sleek, sexy, totally out-of-his-league babe.

The story goes like this: A hundred years into the future, Earth — over-polluted and overtaken by garbage — can no longer sustain life. So we flee to outer space, leaving the planet’s cleanup in the mechanical pincers of an army of stout, capable robots.

Seven hundred, entirely uneventful years pass and now, pillars of compacted trash line the city skies like towering skyscrapers. One day, WALL-E — now the sole surviving creature of his kind — meets EVE, a visitor from outer space with a mysterious mission.

However, Pixar designed these robots so that they’re — well, they’re human. We see them as human. We see them communicate, we see them think, act, understand, love. And we accept this. By the end of the movie, we’ve accepted WALL-E and EVE as equals and we may even shed a tear here and there for our newfound friends.

But what exactly is WALL-E? Is he pure fantasy and fiction?

Or is he — is Artificial Intelligence — simply the way of the future?

Alan Turing’s Vision

This prophecy, published in 1950 by English mathematician Alan Turing was a bold statement indeed. Remember, in that day and age, computers weren’t sleek, glossy, or available in a variety of neat colours; no, they where clunky, they weighed nearly 30 tons, and they took gaggles of people to operate.

Turing, however, saw past all that. He envisioned a day when digital computers programmed with rules and facts would possess the intelligence of man.

This boldness and guiding confidence was exactly what researchers needed and thus was borne the field of artificial intelligence (AI). In the 1950s and 1960s, the field would see enormous growth and popularity. It became the hot topic of students, researchers, writers, and even the movies.

In the 1960s, for example, when Stanley Kubrick directed his 2001: A Space Odyssey, starring HAL, the omniscient and omnipotent robot, he had taken care to directly consult MIT Professor and AI expert Marvin Minsky, who assured him that yes, by the end of the 20th century, robots like HAL would not only live among us, but they would exceed us in many capacities.

It no longer became a question of if machines would become intelligent, but when.

A Philosophical Fork in the Toaster

However in 1973, Berkeley philosophy professor, Hubert Dreyfus published his book, “What Computers Can’t Do”, in which he proposed the exact opposite of what was on everyone’s mind: Machines, he reasoned — as they were progressing now — would never, ever, reach the same intellectual capacities as a human.

There is a passage in Dreyfus’ book in which he recounts the results of a meeting among the top minds in computer science; here, his (early) report of A.I. was deemed to be “sinister”, “dishonest”, “hilariously funny”, and an “incredible misrepresentation of history”.

But of course, researchers in the A.I. community would be incensed. They would be, in fact, deeply, unapologetically pissed off.

After all, they’d just spent the last two decades of their lives telling the world what computers could and would do…only to have their fundamental beliefs and dreams attacked by — of all people — a philosophy scholar?

Hubert Dreyfus Criticises

The core of Dreyfus’ critique was about rules. See, a conventional machine is programmed to accept an input and apply a set of rules to produce an output. The idea is that any intellectual activity, whether it be adding numbers, playing chess, translating languages, or disposing of garbage, could be mimicked using a set of rules.

Dreyfus, however, argued that rules — by themselves — did not contain the necessary information for their application. Suppose we were to design a robot to process the following phrase:

What does “it” refer to, the puppy or the window?

But of course, even a child could tell you that it refers to the puppy. But how does a computer know? Does the computer know that puppies are furry, cute, and love to be hugged and touched by children? Can the computer understand that Mary probably doesn’t want a silly windowpane?

What if instead the phrase was:

Now, it refers to the window. But does the computer know that children enjoy pressing their noses against windows? Does the computer know that the puppy is out of Mary’s reach, separated by a layer of glass?

Not only does understanding the nature of the word “it” in these sentences require such obvious facts about dogs and windows, but it also requires a certain human element. It requires us to empathize with how Mary may feel. It requires us to understand the physical nature of Mary’s body and how she interacts with her surroundings.

Previously, many A.I. researchers believed that programming an understanding of language could be done syntactically – that is, by appealing only to the rules of grammar and dictionary definitions. But Dreyfus (and linguists such as Noam Chomsky) pointed out that the issue was much, much more complex. So much of what we do and say depends on context.

And they were right. A.I. researchers would begin having difficulty producing machines with the common-sense understanding of a mere four-year old. There were simply too many rules — too many rules and each rule leading to more and more rules so that even the most basic statements and stories could simply not be understood without appealing to millions of common-sense facts.

So…Is WALL-E Dead?

But what does this all mean for poor WALL-E? Did Hubert Dreyfus destroy the dream of ever producing a WALL-E? Is true Artificial Intelligence unlikely to ever happen?

No, no, and no!

Dreyfus never intended his original critique to be a crushing blow to Artificial Intelligence. The dream continues to live on, but today, researchers are older and wizen by his words. The field is no longer as naïve and wide-eyed as it was half-a-century ago.

For example, one possible avenue for modern AI research is provided by our own brains: Instead of programming a computer to abide by the traditional step-by-step rules approach, we model it like the neurons in the human brain where the results of the program depend on the ‘strengths’ of each particular neuron.

This radically different method of computing not only combines the work of psychologists and cognitive scientists in understanding how the human mind works, but also biologists and neuroscientists who study the physical brain, and finally, mathematicians and computer scientists, who work to develop the models for artificial neural networks.

If Artificial Intelligence is to succeed – if WALL-E is to ever exist – we know now that it is going to take the work of all of us — of mathematicians, computer scientists, cognitive scientists, philosophers, and psychologists. The dream of imbuing a machine with an intellect – if it is ever to happen – will be the crowning achievement of not any one discipline, but of humankind as a whole.

Seventh Grade Blues

the one

When I was in the seventh grade, one of the girls told me I looked like Keannu Reeves.

Was this some awfully cruel, sadistic joke little girls liked to play on unsuspecting boys?

When I was in the seventh grade, one of the girls told me I looked like Keannu Reeves. No, seriously.

I was hanging upside-down on the jungle gym, minding my own business, and she just walked over and blurted it out. Then she giggled like a moron and ran away. Girls can be so mean.

This became the highlight of my school year (my academic career, even), but you see, I was torn. On one hand, how could anyone confuse Keannu (black shades, gothic trench coat, infinite awesomeness) with me (pubescent, angst-ridden, gawky)? Was this all some awfully cruel, sadistic joke little girls liked to play on unsuspecting boys?

But on the other hand, maybe — maybe she was on to something. Maybe somewhere — somehow, behind all that bad acne and ruffled hair, I really did look like Neo. After all, who was I to disagree?

Today, however, I no longer have to wonder because, according to the latest advances in facial recognition, she was right.

Who in the Land is Fairest of All?

MyHeritage is an internet-based company that offers you the chance to see which celebrity you most resemble. Remember how in Snow White, the queen has a magical mirror which provides her with uninhibited flattery? This is the same, but like, tons better.

After a free signup, you upload a large-ish jpeg of your mug, then let the software crank away. Here were my results: Brad Pitt (71%), Keannu Reeves (63%), Luke Perry (63%), Matt Daemon (63%), and Jordana Brewster (60%).

Brad Pitt? Really? Matt Daemon? Really? Who wouldathunk? But y’know, as I gaze into the mirror…well…yes, I see it now. Definitely. We’re practically brothers!

How does it all work? Is this actual science or just deceptive flattery? To understand how facial recognition works, we’re going to have to delve into the mathematics behind the algorithm.

Recognizing Faces

Suppose we were given someone’s picture. How might we go about identifying that person from a large database of faces?

One way we can go about it is by identifying the characteristics of the subject – perhaps the person has small lips, or a pointed chin, or distinct eyes. From here, we then study the database, going from picture to picture, each time isolating the features of the faces and checking for a match.

But while this might work, it’s also a lot of work; algorithms would need to be defined to analyse each desired feature and a large number of faces to mix and match and could potentially take eons to compute.

A more efficient way to proceed would be to examine these faces as a statistical whole rather than as the sum of its parts. This is similar to the difference between identifying a city by its landmarks and identifying the same city by the density of its roads, the clusters and heights of its buildings, its downtown areas and rural areas, and so on.

A Picture is Worth a Thousand Digits

Snap! But what are pictures, really?

As stored in a computer, a picture is nothing more than a great big grid of dots (or pixels). If the picture is greyscale, each pixel is associated with a number from 0 to 255 representing its brightness, from pitch black (0) to pure white (255).

Now in the abstract theory of Linear Algebra, these grids of pixels are called ‘vectors’. You’ve probably encountered vectors before in Physics class and in fact, these ‘face vectors’ are quite similar.

Like vectors representing force or motion, these new ‘face vectors’ have a `magnitude’ (an overall brightness), as well as a ‘direction’ — the only difference is that they inhabit some higher-dimensional Face Space, rather than the two or three-dimensional physical world we live in.

Thus, faces found in the database are nothing more than vectors that, like other mathematical quantities, can be added, subtracted, multiplied, and generally manipulated as they roam about in the Face Space.

What’s Your Eigenface?

However, Face Spaces are complicated affairs — they’re high dimensional boxes stuffed with a large number of faces, each face containing thousands of pixels.

It would thus be foolish to try and compare each face pixel by pixel; instead we look to construct a small group of pictures representing the general facial patterns of the database. This small but crucial group is called the Eigenface Basis.

Think of how, when we analyse the motion of a ball flying through the air, we break the motion into its horizontal and vertical components. These horizontal and vertical components provide a fundamental basis for which all motion can be broken down into.

Similarly, once the eigenface basis is found using Linear Algebra, each face in the database can then be expressed using certain percentages of each eigenface. For example, we may say that a picture is composed of 10% of the first eigenface, 25% of the second, 4% of the third, and so on.

The beauty of this treatment is that even in a large database, each unique face can be expressed very simply using its eigenface decomposition. We no longer have to express each face using thousands of pixels; now, like a simple recipe in which the eigenfaces are the key ingredients, the entire database can be reconstructed as it was before.

A Problem of Distance

Now imagine each face in the database, represented in terms of its eigenface percentages, akin to coordinates lying in some higher-dimensional plane. Our test subject (which may or may not lie in the database) is then projected onto this plane by expressing it in terms of the eigenface components.

But now, the problem of recognising the subject becomes as simple as finding the shortest distance (or closest match) between our subject and each face in the database, a problem which is aided enormously by the fact each face is represented by only a handful of eigenface components.

The Future and You

But really, just how accurate are these eigenface algorithms?

In optimal conditions (with good lighting, a representative database, front-facing pictures, etc.), a simple eigenface routine might produce accurate readings of up to 90%.

Unfortunately, real life is never that simple, and in reality, one must contend with other ‘noisy’ factors. Factors like variance in pose (person facing at an angle), obstructions (sunglasses or other people), resolution and lighting, and so on. Despite this, however, the science of facial recognition has steadily improved to the point where today, it is becoming a standard for many military, security, and commercial applications.

Yeah, yeah. But now that you know how facial recognition works, go and try it on yourself.

What celebrity do you look like?

A Review

exponential asymptotics

Exponential Asymptotics then, is the bridge between God’s picture-perfect, but wholly unrealizable representations, and the Devil’s ambiguous, but entirely necessary evils.

Last time, we left off talking about asymptotic approximations. Nature — we explained — is often too complicated to describe exactly, and so we turn to these approximate methods for help. Unfortunately, such approximations are divergent and mathematically badly-behaved, even though they provide excellent results.

Now, we will see that there are indeed consequences to using these devilish and divergent approximations.

Dancing with the Devil

Maybe the best thing we can do is look at a concrete example.

For this, we’re going to imagine the flow of water with speed U over a step which is relatively well submerged. To solve this problem, we propose the following asymptotic expansion for the surface of the water:

In this case, the perturbation parameter, ε corresponds to a measure of the inertial and gravitational forces. The idealized state, or first approximation corresponds to the case that ε = 0, or the case for which the force of gravity is infinite.

What that means is that our first order asymptotic approximation tells us the free surface should look like this:

That’s right. Completely flat.

Sure, this does make some sense: Imagine the step as being infinitely deep. Then in this ideal case, free surface wouldn’t feel the bump at all. Hence it would lie completely flat.

Unfortunately, this isn’t quite what happens.

If we use a computer and calculate the solution, it gives us something like this:

Somewhere on that free surface, something is happening.

The asymptotic approximation we calculated does a wonderful job initially, but then out of nowhere, a light flickers on and Pop!.

Waves appear where there were none.

Into the Mist

Maybe the first person who documented the first “WTF?!” moment upon encountering this strange phenomenon was none other than George Gabriel Stokes, one of the greats of British applied maths. He described the phenomenon as a new term, entering into view from behind a mist:

How do we clear the mist?

The argument goes like this: First, we have an approximation that works on the far left. Now, somewhere, along the surface, a switch is turned on, and an exponential (or a sinusoidal, if you will) is turned on.

Thus, there must be something that’s going on the complexified free surface.

Whoa there, Missy! What the heck does that mean?

By complexified free surface, we’re referring to extending the water surface (a real number) to the complex (or `imaginary’) numbers. In other words, if x measures the distance along the water surface (say, 1 meter, 2 meters, etc.), then we allow x to be an imaginary number (say, 1 + i or 2 - 2i)

Physically (and philosophically), it’s pretty nonsensical. We picture it like this:

So we’ve extended the surface by adding an extra dimension.

This, while baffling to outsiders, is a well known trick by mathematicians. Maybe there’s an easier way to explain it.

The Imaginary Route to School

Let’s suppose that Billy walks from his house (x = -10) to school (x = 10) every day.

Monday, however, he wakes up and hears on the radio that due to faulty engineering and a freak division-by-zero accident, the bridge (x = 0) on the way to school has collapsed. What can Billy (determined and diligent student that he is) do?

The answer is that Billy needs to find another way to school. He needs to complexify his usual route, enter into another dimension, and find a new path.

Billy’s adventure is the essence behind the mathematician’s method: Often, we will have some sort of singularity between two points. To deal with it, we seek a new path. It turns out that useful paths can be taken in the ‘imaginary’ plane. So while this is definitely unphysical and unintuitive, take a lesson from Billy.

If he can warp his mind to change his usual route, so can you.

Singularities and Stokes Lines

Let’s return to our complexified water surface.

Well, it turns out that on this extended surface, there are singularities: points where our asymptotic approximation (remember that?) goes terribly awry. This is the reason why the asymptotic expansions diverge — because of these singularities.

With a lot of work and advanced maths, it can be proven that there exists lines (called Stokes Lines) emanating from these singularities, across which a small exponential term is switched on.

Like into the mist, remember?

And so following the above picture, you can see that the asymptotic approximation (the flat surface) works well at the start. But then it hits the edge of the Stokes Line, and at that point a teensy exponential is turned on.

So actually, we should have,

Exponential Asymptotics is the name we give to the branch of mathematics which studies these small exponentials which seem to fly under the radar of ordinary methods. It’s these hidden exponentials which emerge from the divergent tails of asymptotic expansions.

From Heaven to Hell

It’s time to wrap up.

In each previous part, I ended by asking a crucial question. They were:

Part I: Sometimes, as mere mortals with a capacity for only understanding the finite, we’re forced to cut corners and make approximations. Most of the time, it won’t matter. When will it matter?

Part II: In order to describe nature’s oceans and lakes, mathematicians have had to use divergent (badly-behaved) series approximations. Although these give superb results, we are nonetheless representing perfect phenomenons using a far-from perfect representations. “Inventions of the devil”, said Niels Abel. But this is a practical necessity. How else are we to describe God’s infinity with our finite minds? The real question is, what’s being lost in translation?

In this part, we showed that while divergence is bad, there’s no need to disregard it. By using certain mathematical tools, we can unravel the divergence, and reveal what details were lost using the approximations.

Exponential Asymptotics then, is the bridge between God’s picture-perfect, but wholly unrealizable representations, and the Devil’s ambiguous, but entirely necessary evils.

This bridge is where I stand and this bridge is what I study.

A Review

abel’s hell

“Divergent series are the invention of the devil and it is shameful to base on them any demonstration whatsoever.”

niels abel

Last time, we left off talking about Pi.

We talked about how one can imagine Mother Nature using Pi to all its infinite glory, but for mere mortals like you and I, we have to truncate the number — to its tenth decimal place, hundredth decimal place, or even millionth decimal place.

But however we do it, we still miss infinitely many digits (infinity minus a finite number is still infinity, don’t-cha-know).

This is simply an example of our inability to describe the Universe to its fullest.

For the most part, that’s perfectly fine. But in what circumstance does our inability to capture the true nature of a phenomenon become a problem?

A Hopeless Challenge

Let’s talk oceans.

There are few things more majestic than a ship in water. Captain Ahab knew it. Jack and Rose knew it (well, at least until the end). So should you.

Just imagine it. The sparkling blue waters. The expanse of never-ending ocean. The beautiful wedge-shaped pattern that flows behind every ship, like a streaming cape.

And near the ship: turbulent white-water splashes, jets, and ripples, forming in a million different places with a million different patterns, often too quick and too elaborate for the eye to capture.

The mathematical equations which govern the flow of water in an ocean were known as early back as the 18th century. But for the next century and a bit, scientists were absolutely confounded by the equations.

“What in the world,” they asked together, “Can we do with that?”

The equations, as they stand, are much too complex for a direct analysis. Even today, there is a million dollar prize for anyone who can answer even the most basic questions about these equations.

So with no possibility of exact solutions in sight, the mathematicians and physicists turned to developing methods for approximating solutions.

Asymptotic Approximations

One method, in particular, traces its roots back to the time of Henri Poincare (1854-1912) who developed it in order to solve a rather terrifying problem in celestial mechanics.

The idea is to break up our solution into more manageable chunks. The first chunk describes the system — the ocean, in this case — in some idealized state. To this, we add the second chunk, which includes some kind of perturbation to this initial state. To this, we add a third and even smaller perturbation. And on to infinity.

For example, suppose we were to express the height of the water waves in this fashion,

Here, the greek epsilon, ε represents a small perturbation. It can represent, for example, the Froude number, or the surface tension, or some other small effect.

These sums or series are called asymptotic approximations because they get better and better in the asymptotic limit that ε tends to zero. That’s the idealized state.

The amazing thing with these approximations is that they tend to be very good. So good, in fact, that calculating only one or two chunks provides an excellent approximation to reality in most cases.

This was great. People were happy. We’d at least established some kind of systematic way of approaching these intractable formulae and, with a moderate amount of sweat and tears, they seemed to give excellent results.

The Physicists were happy.

So were the Applied Mathematicians.

The Pure Maths chaps, on the other hand, were incensed.

The Puries and their Partypooper Tendencies

The problem, the Puries were quick to point out, is that these asymptotic approximations are usually divergent.

What that means is that the approximations get better and better as we include more and more terms…but only up to a point. After that point, all hell breaks loose, and the whole thing blows up (to infinity).

Thus in effect, we have taken a system of water waves, perfectly well behaved and all very pretty, and approximated it using an infinite series of terms which, when added up, gives us completely nonsense.

The Pure Mathematicians, in all their need for rules and regulations, just weren’t comfortable toying with a concept that was so ill-defined.

But everybody else was all, “Fuggetaboutit. Take a look at all these purty graphs, yo.”

Should We Worry?

By now you’ve hopefully grappled with issue.

Nature in all its magnificence is woefully complicated.

In many cases, it’s so horrendously complicated that its mathematical description needs to be approximated. One such method of approximation involves using a divergent series.

Why do we use them? We use them because they work so nicely.

So the question becomes, in using these divergent approximations, are we ignoring anything useful? In particular, what is the nature of this divergence? Why does our approximation work so well initially, but behave so badly later on?

What’s the price we pay for dancing with the devil?