“Why Define bullshit”, you might ask?    Why not?  It’s probably 90% of what you can read on the Internet.   It’s probably a large chunk of what you hear from your upper management at work.  It’s probably 90% of political advertisements, and it’d probably be 90% of commercial advertising if there weren’t any “truth in advertising” laws.   [Those laws are why drug companies give you a feel-good video that implies you will be floating through fields of flowers after a dose of Fluoxidyne, while a speed reader lists side-effects and says "take this only on the recommendation of your physician." -- No law regulates the images and implications in the ads, just the words and the formal claims.   Absent those laws, most companies would probably tell you wondrous bullshit about their products.]

So, it’s entirely reasonable for a philosopher to worry about what separates bullshit from truth and lies.   Clearly, bullshit isn’t the same thing as telling the truth, and it’s not quite the same thing as lying, either.   At 67 tiny pages, it’s probably the smallest philosophy book you’ll ever read.   [That may be because he stopped writing before he started producing bullshit.]

Many theories of speech production see phonetic implementation of underlying phonological structures as an automatic process governed by universal and sometimes language-specific constraints. Such assumption for example underlies the phonological account of…

I used to  grudgingly sign my name to such sentences back when I was a linguist.  But now I make my money in other ways and I have no reason to pay attention to the Linguistic community.   So, let’s take a look at the text and see what’s right and what’s nonsense.

“Many theories of speech production … phonetic implementation of … phonological…”   The trouble is that “many” is the wrong word because there are no Linguistic theories of speech production.  Linguists, when being careful, tend to call their things “paradigms” or similar terms, in recognition of the fact that they don’t normally predict what is going to happen.  [Often, they are intended as a way of looking at the data or organizing it.]

• In this context, a theory is something that predicts your phonetic implementation (i.e. how you say something, in terms of sounds or tongue positions) from some sort of phonology.   When one says the word “theory” in science, one means some sort of prediction that is clear enough that one could prove it to be false.    [This is standard Karl Popper stuff.  You make up a theory and then you test it by comparing it's prediction to reality.  If it disagrees with a good experiment, the theory is wrong, and you change it or throw it away.  If it agrees with the experiment, it lives to be tested another day.   If it passes many tests, wow, maybe it's actually correct!]  Now, the problem is that there are no linguistic theories in this area, because no one has written anything that makes clear predictions that one can test.  If you don’t believe me, take a look at the Google Scholar search for “phonetic implementation phonological theory predict” and try to find a paper that makes a detailed prediction of which speech sounds are produced when, or any kind of detail about where the tongue goes.  You won’t.
• You may find a few weak predictions along the lines of “the tongue will be raised…” but those don’t count for much because they have a 50% chance of being right by blind chance.  Another reason they are weak is because the “prediction” is usually derived just by trying to say something and feeling where the tongue goes; in science, that’s normally called “reporting the result of an experiment” rather than “predicting from a theory”.
• To an extent, such theories exist in the form of speech synthesis and/or speech recognition systems.  While linguists don’t treat those systems as theories, nevertheless they make detailed predictions of how a given text should sound, even if it is a text that no one has ever said before.  However, while these systems are much more predictive than a typical linguistic paper, they are still not entirely satisfactory because they are all based on recorded speech.   That means, they are not predicting as much as one might hope: their output is just a reproduction of what some human said, but sliced up and glued back together.  Their strongest claims to theory-hood are the places where they cut, and the details of what speech segments they glue back together.   They say that “if we cut speech here and here, then we can glue those bits and we’ll get a whole new word.”

“…underlying phonological structure…”    The word “underlying” is beautiful, because it subtly claims that phonology is the reality, and the tongue motions and sounds we hear are mere secondary phenomena.  But, that word has no right to be used: phonology could just as well be a convenient way of dividing up the sounds that we make, rather than something deeper and more important.   It might be just an artificial way of chopping the world up into convenient boxes.

• Is phonology underlying?  I.e. is it something that is more real and more stable than speech sounds?  That’s what “underlies” means in this context: for example electrons underlie the phenomena of electricity.  That means that electrons (and a few equations) are able to explain anything you might want to know about electricity.
• If phonology were really “underlying” in that sense, the evidence for it [hopefully experiments!] would be triumphantly taught in undergraduate courses.  These would be basic experiments that would support the entire field of Linguistics.  But, the evidence that phonology underlies behaviour isn’t taught, because (to the extent it exists at all) it is ambiguous.
• If phonology were really proven to be “underlying”, people who wrote the phrase in their academic papers would reference it.  [Unless it were so well and broadly known (like Newtonian Mechanics) that it was taught in undergraduate courses.]    Let’s search for it.   As of today, I get the following hits:
  * I. Y. Liberman, doi: 10.1177/074193258500600604 .  No reference.
  * I. Y. Liberman, book.  No reference
  * Bert Vaux, doi:10.1162/002438998553833.  No reference.
  * A. Lahiri, http://dx.doi.org/10.1016/0010-0277(91)90008-R.  No reference.
  * et cetera, through 50 references at least.
  Everyone writes as if their reader knows what “underlying phonological structure” is, but no one points to evidence.  Of course, they’ve all read each other’s papers, and they’ve gotten used to reading that phrase.
• In the real world, phonology is always derived by observing speech.  One cannot know what words someone is saying unless one listens to the speech and then deduces what the phonology ought to have been.    This procedure is a better match to “phonology is a convenient way of categorizing speech” than to “phonology underlies and explains speech”.
• Given the lack of proof that phonology underlies anything, it might be inappropriate to use this phrase.

Then after that rough start, the abstract improves.    The author is careful to say “theories see” instead of “theories show” and then later “Such assumption…”   The author does indeed know that this stuff is just assumed, rather than known.  Because so many of the basic assumptions of linguistics are not solidly established, it’s very hard to write carefully and still make an abstract sound impressively linguistic.

[ Note: the image is copyright http://jerryabuan.zenfolio.com/.  I'm grateful for it.]

Actually, it looks pretty good: it proves its point fairly clearly.   Ordinarily, I’d say that there ought to be a bigger control group, but I think it’s reasonable to believe that the domestic teddy bear reproduces reliably under laboratory conditions.   So having one control bear is a bit weak, but probably OK, especially given the magnitude of the effects we see when Pure Evil is introduced.

• There’s plenty of room for a brief mention of some of the other literature on Pure Evil.
• The data at 1000ppm should be supported by some evidence other than a note.    Perhaps a photo of the cage bars after the teddy bear chewed through them?
• There is nothing presented to make the reader believe that the purple stuff in the 700 ppm offspring is from a different species.   Quite possibly, it is merely mutated teddy bear tissue.
• There should be notes that the experiment and the treatment of the experimental subjects was approved by an ethics committee.
• Normally, one would expect to see an acknowledgement for the source of the experimental funding.
• If the two postdocs who were killed in the experiment contributed to the research, then they should be co-authors.   Of course, if they had just been standing there, authorship would not be appropriate (and I’m not sure whether an acknowledgement would be in good taste or not…).
• It would have been good to have taken tissue samples of the 1000 ppm parent after it was euthanized.  Possibly some light could have been thrown on the biochemical mechanisms of ocular luminescence.   But, perhaps that’s the subject of a separate paper.
• It would be good to have a laboratory analysis of the Pure Evil.    What if it weren’t pure?   After all, materials recovered from exploded toasters are often contaminated with bread crumbs, melted plastic, or plaster dust.

The main issues which are on the table concern the nature of sounds. Sounds enter the content of auditory perception. But what are they? Are sounds individuals? Are they events? Are they properties of sounding objects? If they are events, what type of event are they? What is the relation between sounds and sounding objects? Temporal and causal features of sounds will be important in deciding these and related questions. However, it turns out that a fruitful way to organize these issues deals with the spatial properties of sounds.

Indeed, the various philosophical pronouncements about the nature of sounds can be rather neatly classified according to the spatial status each of them assigns to sounds. Where are sounds? Are they anywhere?

None of these questions make much sense in the real world.  As a physicist and speech researcher, I know what goes on in the world shortly before we perceive a sound:

1. Something vibrates.
2. It causes the air near it to vibrate.
3. The vibration travels through the air as a pressure wave.  The wave spreads out in all directions.
   
4. The pressure wave is modified by all objects it passes.   Some things reflect parts of the wave, some things absorb parts.
5. The wave gets to your ear, and it makes your ear drum vibrate.  That causes the oval window of the cochlea to vibrate, which makes the fluid inside vibrate.
6. Hair cells are triggered by the vibrations and send nerve impulses up the auditory nerve to the auditory cortex.
7. It gets complicated.

Sounds (whatever the philosophers think they are) are the end-result of a complex process that is spread over space and time.  For instance, as I hear a car drive by, my perception is caused by:

1. Rubber tires unsticking from the rough road surface and bouncing back into shape.   These sounds are, of course, made by four tires simultaneously.
2. The sound waves bounce around under the car, bounce off the road, and perhaps other cars near by.
3. The sound wave diffracts over the top of the noise barrier at the side of the highway, and some of it bends down towards my window.
4. It sneaks through the window somehow, leaking through gaps, vibrating the glass, and mostly bouncing back.
5. Then it gets to my ear and various perceptual things happen.

So, in making that sound, there are kilograms of rubber, a ton of fast-moving car, maybe 300kg of air, two windowpanes, and my head.  It’s a complicated process that involves a lot of stuff.  Notably, if any of that stuff changes, the sound will change slightly.  Change the car, the windowpanes, the noise barrier, anything.

So we can think about the philosophical questions they ask:

• Are sounds individuals? Depends on exactly what they mean, but the answer isn’t likely to be interesting.  If they mean “individual” = “isolated thing”, the sound of the car tires isn’t very isolated from other sounds.  Certainly the sounds from the four tires aren’t very isolated from each other.   If they mean “individual” as “unique, never duplicated”, then maybe, if we could somehow split the noise we hear into separate sounds, then no two of them would be exactly alike.  (If one can measure precisely enough.)   But, the process of splitting is pretty artificial.
• Are they events? By “event”, they seem to mean something that has a fairly definite location and time.  That description is spread over many cubic meters of space, and it is only localized in time if you artificially chop the sound into individual moments.
• Are they properties of sounding objects? What you hear is a property of the sounding object, the environment, and you.  It’s not just a property of the sounding object.
• What is the relation between sounds and sounding objects? Part of it is called acoustics.  Most of the time, acoustics gives you a complicated mathematical relationship between the sounding object and the motion of your ears, and it is not something that is easily expressible in words.  Trying to express it in terms of words (unless one wishes to be approximate) is a mistake.  This is not a question for philosophers any more — the physicists took it over in the mid 1800s.
• Where are sounds?  Are they anywhere? These are simply silly questions.  Various parts of the process that produces the sound occur in different places.  There is no single place.  It’s very much like asking “Where is Julius Caesar now?”  He’s not in any one region any more: his atoms are scattered and blow with the breezes.

And, then the article sets off to answer these questions.  Not surprisingly, it doesn’t get much of anywhere.   I’d be embarrassed to have written that article; it was written from the narrow perspective of people who play games with words in order to score points against other philosophers.  It’s not really there to help you understand anything.   Or, if it was, they forgot to read all the acoustics literature that’s accumulated in the last century or two.

F[n] = F[n-1] + F[n-2]       (See A, B, C, D, E, F, G.)

1.000000000

-0.6180339887498949

0.3819660112501051                  ( 1.000 + -0.618… = 0.381… )

-0.2360679774997898               ( -0.618… + 0.381… = -0.236… )

0.1458980337503153                 ( 0.381… + -0.236… = 0.145… )

-0.09016994374947451             ( -0.236… + 0.145… = -0.090… )

…..

Why does it get smaller instead of larger?

But there’s one place where it falls down: Chapter 5, “Uncompetitive in Everything”.  This is an analysis of international trade, and Heath attempts to show that globalization does not force your country to become more competitive.  He makes the claim that you cannot be uncompetitive in everything.

Heath uses an example of two bakeries: A “Rich Side” bakery that is especially good at making bagels, but relatively inefficient at making tarts, and a “Poor Side” bakery that is good at tarts and poor at bagels.   The bakeries are in different countries, so they do not share a common currency.  He goes on to show that — even if the labor costs are much higher on the rich side — both sides are better off if they trade bagels for tarts.

The essential argument is this:  suppose the rich side makes bagels for £1 and tarts for £2.   Suppose the poor side makes bagels for ¥2 and tarts for ¥1.  [I'm not saying Brits are rich and Japanese are poor here: £ and ¥ are just two convenient, different currency symbols that people will recognize.]  Now, someone on the poor side has two ways to get a bagel.   One way is that they can make it locally, for ¥2.   The other way is that they can make a tart (for ¥1) and sell it to someone on the rich side.  If international trade is rare, they’ll get a price near £2, and they can then go over to the rich side and buy two bagels with that amount of money.      So, with international trade, they could end up with two bagels for half the price.     And, of course, the same logic applies to both sides.     Trade benefits both countries.  This isn’t a new idea.  It comes from David Ricardo, around 1817.  [1, 2, 3]

The arguments presented are absolutely correct, as far as they go.  But the model presented is just too simple to capture a couple of important effects in the real world.   Two goods is not enough.  The trouble is, there need to be at least two other goods in the model:

1. Goods that you cannot produce at all in your country.    Foreign travel is one example of these.   If you live in Alberta (or Yorkshire), you cannot produce a trip to Paris no matter how hard you try.  You can only do it in France.   Another example is stuff that requires huge capital or intellectual investments, like semiconductor fabrication plants.   Building anything close to state-of-the-art chips takes multi-billion dollar investments and lots of specially educated people, and most nations will not have the capability within their borders.
2. Goods that you really don’t want to sell, on moral/ethical grounds.     Such as the slave trade, sex tourism, one’s kidney, or the right for other nations to dump their waste on your ground.   And, there are a whole bunch of milder variants of #2 that one would also prefer to avoid:  for instance, “Ship us your recycling, we’ll sort it and send it back.” — honest, but dirty jobs.

Now, the trouble is, there is a certain insatiable demand for #1.  Everyone wants to do a little travel, and the political elite will arrange to do some, by fair means or foul.   Or, the elite will send their kids to expensive private schools in England.   And, you can bet that whomever has money and power will have new PCs and iPads or whatever nearby.   This demand is not very elastic, and will never go quite to zero no matter what the exchange rate is.

Therefore, if conventional trade becomes sufficiently uncompetitive, the outflow of money for #1 will end up being balanced against category #2.    Therefore, it does matter (to a degree) how competitive you are.   You don’t want your industries to be less valuable to outsiders than things in category #2.

[Oh, and there are other ways in which Ricardo's analysis doesn't quite represent reality.   It assumes that the bagels produced on the rich side and the poor side are completely indistinguishable (which might easily be false).  It also assumes that trade is free, the costs of trade are small, that prices are reasonably stable so that the risk of the price changing badly between the time you decide to bake tarts and the time you finally end up buying bagels.   All these simple models are [of course] simplifications of the real world.]

Dear Michel,

I hope you are enjoying your summer. The sun is bright, the weather warm and the beach as good place as any to learn how to code in Java. Six years hence, I am the last person to begrudge you this sunny start to research that will take about three years longer than you think it will. So enjoy.

But let me pass on a few pieces of advice that can all harken back to an old military saying – “Amateurs talk strategy; professionals talk logistics.” The big ideas are important, but in the end the hardest parts of what you will do is refining those big ideas into testable hypotheses that you then can actually test. Once you do that, it turns out you will have proof that you are up to this DPhil and have something worth submitting. Until then, you are an amateur.

Don’t get me wrong. Right now, you are bursting with ideas for how to answer the most interesting questions at the frontiers of computational linguistics. And that is a very good thing. Those ideas will keep going through the cruel days it will take you to fully understand how to compute an Eigenvalue Decomposition or a G-Function. Enjoy them. Let them turn over in your mind as you have that second beer.

That said, please, for my sake, write them down –And not on the window in your room. You are going to move more than once over the next six years and the landlord just might come in when you are not home and clean off the mess. To be sure, the best of what you come up with will come back to you at some point, at least as far as I remember. But still, keep track of your ideas, your research and your sources. Maybe start a journal with dates and short, legible explanations. Six years from now when you are editing your bibliography in BibTeX (it’s a programming language used for copyediting, don’t worry about it now), you will thank me. It turns out that (Brown 2000) is kind of ambiguous as citations go and hunting them down all over again made me very mad at you for being so lazy.

Another thing to keep in mind is that you should not underestimate your capacity needs. Too much time and energy will be spent trying to fit what you want to do in the resources you have at you deposal. It is easier (and if you value your time cheaper) to just get more resources. Right now, you are coding on that trusty laptop you bought before you started school. To be sure, it will serve you surprisingly well. You will find that it will be up to the data processing needed for your MPhil thesis, but you will have to put it on its side on the floor with a desk fan at its back to prevent it from overheating.

Ultimately, you will build a small city of towers. ‘Towers?’, your thinking, ‘Isn’t that a little excessive?’ No. It’s not. You will get a number of them over the next few years. One will explode – loudly – and fill your apartment with the smell of burnt metal. Ultimately you will have about four sometimes five, even six on the bad days, running at any given time.

Which leads me to a very important point; do not underestimate your ability to confuse yourself. Four to six computers running at the same time is surprisingly complex to manage. The obvious answer will be trying to figure out a way of networking them together and threading your code. Don’t waste your time. This is exactly what I mean about not confusing yourself. Your thesis is in computational linguistics. You use computer science; you don’t study it. If you want to inject an unholy amount of complexity and confusion into the next six years of your life, network your computers.

Or you could do what you will inevitably do anyway. Just copy the essential bits of your data five or six times and network them via your sneakers, flash drives and GMail. Simple, dirty, cheap, but it works.

If you are worried that doing this means that you are not sufficiently challenging yourself, don’t be. Even this will continue to surprise you with opportunities to confuse yourself. At 330AM, when you have searched every line of code to find where you are dividing by zero because it seems infinites have apparently recruited all of the numbers in your results into their Borg, you will realize you are actually looking at code you wrote three months ago on a different machine. You can formalize this into a convenient equation: Self = Confusion/0.

This is how you will get to Sesame Street. Yes, sunny days, chasing the clouds away Sesame Street. High-resolution pictures of everyone in the neighborhood are freely available on Google Images and as desktop wallpaper, they will give a face to each of the computers you will have churning away over the next few years. That way when you finally get your act together and begin a proper coding log, you can amuse yourself with entries like, “Bert exploded after the power source apparently overheated, hard drive recovered.”

The hilarity never ends and some people, like your future wife, will argue that it never started. To be sure this turning your computers into Sesame Street characters is evidence of infantile regression and perhaps some light madness. But if there is one thing that is true about writing a DPhil, particularly one that demands extensive computational work, it is that it is a long, lonely process. More than once you will ask yourself why you are doing this at all, especially after you take that job in a few years and the thesis fills each and every one of your nights, weekends and days off.

But you will find as you are looking at Ernie at 215AM on a Saturday night with ‘Rubber Ducky’ stuck in your head as you again try to optimize and debug your code that you have actually come a long way. You have gotten efficient at coding. You have actually learned statistics and a fair bit of upper level linear algebra. Math is no longer voodoo to you, which is an accomplishment in itself. Indeed, when the lawyers at work ask you about your thesis and you begin to explain it, they will squint at you with the suspicion they would use if they suspected you were concealing a prior felony conviction. (Oh, and by the way, don’t tell them the Sesame Street thing. That makes the squinting much harder.)

The last piece of advice is to be grateful for all of those (real people) around you. When your advisor tells you that a certain experiment or data are crap, believe him and move on, he’s right. Take your future wife on plenty of dates. She will be surprisingly understanding of your need to spend all of your free time on Sesame Street, the SATA cables and menagerie of components with which you decorate the apartment and the occasional chorus of expletives from the other room that wake her in the middle of the night. And be grateful for your future self, who was kind enough to write you this letter. He has just saved you a lot of time and aggravation.

Your friend,

Michel

My daughter applied to quite a number of universities this year, and I am feeling like an ill-informed consumer.  Not so much because I haven’t informed myself, but because the data I see is either of dubious relevance or follows a dubious methodology.

Most of these educational rankings are built of three kinds of factors:

1. Money.   E.g. how much money comes in.
2. Counting people.   E.g. student/teacher ratios.
3. Counting research results.  E.g. published research papers and Nobel prizes.
4. Opinions.  E.g. what people think about it.
5. Admission statistics. E.g. what fraction do they accept and what fraction take up the offer.
   

The trouble is that none of these factors are particularly satisfactory::

1. Money doesn’t directly educate people.
2. Smaller classes ought to be better, but when I read the research literature [e.g. 1, 2, ] it doesn’t seem to make a big difference.  And, those studies are mostly done on primary schools, not universities, so who knows how relevant they are?   Universities also have the complication of a wide variety of class sizes, from single-person tutorials (e.g. Oxford) up to huge lectures (e.g. nearly everywhere).  Thus, you can have two universities with the same student/teacher ratio: one could have all mid-size classes and the other could have a mix of small classes and large lectures.
3. I am a firm believer that education should be done by people who have enough knowledge of the field to admit all the things we don’t know.  Researchers are good at that; people who do nothing but teach must surely be tempted to compromise and to teach what the students want to hear.  In my experience, students want short, simple, complete answers and textbooks often supply such answers even if the truth is messier (for example, see here).  BUT, research isn’t the same thing as teaching, so it can’t be more than a correlation.
4. Everyone has opinions, but what are they based on?  Ideally, you’d like to ask a lot of people who went undergraduate at X, graduate school at Y, and taught at Z, and then put their opinions together.  Nobody can really compare more than about three universities, not if you want an informed choice.  But what is usually done?  They ask deans to rate all 500 universities in the survey.  The Dean probably knows something, first-hand, about five, and he/she has good second-hand rumors from people at another dozen, perhaps.  Where do the rest of the ratings come from?  Thin air, or perhaps from reading last-year’s survey.
5. The acceptance rate just depends on how many people apply.   So, that’s just the opinion of parents.  The fraction of people who take up the offer is just the opinion of students who haven’t been to university yet, so that’s not particularly helpful.

This was all made more obvious by moving from the US to the UK a few years ago.  Now, in 2011, I have pretty strong opinions about a lot of UK universities, many of which I had never even heard of when I lived in the States.  And, I had totally missed some pretty good places, like Warwick or Imperial College.  I did know that Durham was pretty good, a place that few other people in the US had heard about, but I only had a high opinion of it because I worked with a good grad-student who came from Durham.  Without that one chance encounter, Durham would have been a blank.

Now, I ask myself, “Where do my opinions about UK universities come from?”  Not from personal experience!  I’ve visited Cambridge once (Physics department), Edinburgh once (Speech technology and Linguistics), University College London twice (Nanotechnology and Linguistics), Birmingham University once (Computer Science), and that’s about it.  Hardly a uniform sample, and hardly a big one.  I’ve met a lot of professors from other UK universities at conferences and a fair number of students who have come to Oxford, but those are almost all speech scientists or linguists.  My only basis for an overall evaluation of universities (rather than their language and speech departments) are what I read on the web or in newspapers.

But you also pick these things up by talking to people.  Everyone else has opinions and expresses them with that shrug of the shoulders or that tone of voice.  But, do they have a better basis for their opinions than I do?  I doubt it.

So, where did we apply, and why?  (But don’t take this bit seriously.)

• Cambridge, probably so she wouldn’t have to live at home.  (NB: You can’t apply to both Cambridge and Oxford.)
• Durham, because it’s a good place (at least based on one grad student) and it’s not in London.
• MIT, probably because MIT students put a police car up on the Great Dome, years ago.  Also, a “weird, energetic” culture and a couple of good lectures during a campus visit.  And, both of her parents went there, years ago.
• Stanford.  Palm trees, blue skies, an atmosphere of academics mixed with luxury, and a really enthusiastic tour guide.  Good lectures.
• Johns Hopkins, despite the fact that a student there told us “I like it here, but if you can get into X, I’d go there instead.”  The April fool’s prank certainly helped: it showed the administration had a sense of humor.   Good stuff on display in the physics lobby, but a negative was they wouldn’t let us to to a lecture.
• Not Harvard, despite it’s high rank.  I think it was because the tour guides and campus visit were remarkably uninformative.  Too careful and cautious.  We visited some classes: an excellent history lecture and a mediocre statistics lecture.
• Princeton.  Perhaps because of Hoagie Haven.  They would have let us into a lecture, but we didn’t realize it at first.
• Duke.  Good place, near the family, good visit.
• …and a few of other places.

My daughter requires me to say that there were other, more rational reasons in addition to those above.  For instance, Durham (like Cambridge) offers Natural Sciences instead of Physics, which is attractive because she doesn’t want to focus down too tightly, too soon.  But some of the decision really does come down to the visit: the tour guides, the lectures, the stuff on the walls, and the way the students walk.

The number of hits Google gives is just an estimate, a fact that Google themselves emphasises. For instance, you might have noticed that Google only gives three significant digits in their results (see Matt Cutts’ comment here [2]). There is more, as Greg pointed out: see an entry by Prof Jean Véronis here [3] and another by Mark Liberman here [4].  These distortions are rather sophisticated and might not affect your search queries.

Incidentally, you might wonder why I’ve been using Google’s results for an academic paper. Primarily because there is simply no practical alternative. Second, I’m taking some precautions to help ensure that (to the extent that the counts are distorted) all my results are affected equally. Since I’m looking at the ratios between certain search queries, any distortion that is common to both queries will not affect the results.

Second, I made  sure that I saw the number of hits deflate. Third, I assume that all my  search queries were equally distorted, which again makes the results more viable.

But, despite the usefulness of the Google counts, there are limitations that might cause trouble. One oddity occurs in big searches: One can only access a tiny fraction of the results Google returns (e.g., when googling “Volkswagen”, one will get some 800-900 million results, but one will only be able to click through a couple of hundreds of them).

Another problem is this: Well into my research I’ve noticed that the  results seems to be inflated: Certain search queries will return thousands of hits at first, but as you browse through them, the number collapses. Try the following search parameters and see for yourself:

“{menace OR menaces OR menaced} {me OR you OR him OR her OR us OR  them} to” -secrecy – silence

(Note the double inverted commas and the negatives.) At first, Google gives some 40 000 hits for this query, but as you page through, the number collapses to nothing, really.

To my research, this was a serious blow, as I relied on the viability of  Google’s number. I’ve got in touch with Alex Chitu of the blog Google Operating System [5] and he explained to me that the reason for this distortion lies in the syntactic complexity of the search queries: They use quotation marks, logical operators, and exclusions; which again makes Google’s estimates get coarser.

This might explain the extreme inflation (by a multiplier of about 200), but even simple search queries give results that are inflated to a certain degree. Try to google ‘Eierschalensollbruchstelle’ (from Eierschalensollbruchstellenverursacher, German for egg puncher; a small  kitchen utensil to punch a hole into an egg, so it doesn’t crack when  boiling) and you’ll get some 500 or 600 results. As you click through them, it deflates to well under 200 results. Or try the proper name ‘  “Adam Roseneck” ’ (it’s important to put it into inverted commas): it gives 500 – 600 hits, which deflate to well under 50.

Now, the point with ‘Eierschalensollbruchstelle’ and ‘Adam Roseneck’ is that they are not incredibly complex queries in terms of their syntactic structure, but there is still some significant inflation going on, viz. by the multiplier 4 and 10, respectively.

Further, when comparing my results from 2008 to results you’d get today, I’ve noticed that current Google results are completely off (maybe this  points to a possible change of Google’s search algorithm?). Yahoo’s 2011 results, however, remained nearly the same compared to 2008 (despite a slight, overall increase; which you’d expect from a rapidly growing internet).

All these issues call Google’s estimate into doubt. As Google say, their “results estimates are just that — estimates” (again here [2]).  It is for you to judge how reliable these estimates are.