Google Labs Aptitude Test (GLAT)

Joe Kraus has a new typepad about entrepreneurship, and one of his first posts is about Hiring -- No False Positives, a great subject. It's important to remember that there's no one who's great in every environment, so great candidates need to be interviewed as much for cultural fit as they do for skillset. Joe writes:

Any hiring process should focus on never letting in a bad fit. Even if that means accidentally rejecting a lot of people that would be good fits. Said another way, it optimizes for no false positives, even at the expense of false negatives.

The Google hiring process is notoriously long and complicated. Internally, it's kind of a Liberum veto thing - a single no-vote of the hiring committee means you're not in. Why? Because they put the principle of 'no false positives' to work. They assume that there is a huge talent pool of great people and that they can afford to pass on people that would be great fits in order to make sure they never let someone through who doesn't fit.

Trust me. It's so hard to do. Especially in a startup where you've got much more to do than you have people to do it. But, slip up, even once and it's trouble fast.

I totally agree with Joe on this, and have been burned when I didn't listen to this advice. I would rather be understaffed, with everyone productive, than hire someone who does nothing (zero productivity) or -- worse -- takes away from the productivity of others (negative productivity).

So I was flipping through Dr. Dobb's Journal for October 2004 that came in the mail this week, and discovered after page 28 the Google Labs Aptitude Test, test code WR-426F: "How much aptitude do you have for the sort of mind-bending engineering problems encountered each day at Google Labs? Take the GLAT and find out. Simply answer all questions to the best of your abilities (cheaters will answer to the karma police) [and send it to: Google Labs Jobs, 1600 Amphitheatre Parkway, Mountain View, VA 94043-1351]. Score high enough and we'll be in touch. Good luck."

A few comments before I list the test.

• Philosophically, I have a slight problem with their desire for people not to cheat. One set of skills is the raw talent to answer the questions oneself, derived from first principles and creativity, and I guess this test is meant to probe that. On the other hand, another set of skills that is also useful is having the perseverance, the searching skills (uh, hello, this is the friggin' Google Labs Aptitude Test, and the sheer willpower to find the answers by any means necessary shows a lot about a candidate). I'd like to think that the best candidate is someone who's smart and creative enough to answer some questions, and who's tenacious enough to glean the answers to some other questions, and who's ethical enough to tell me when they went for additional help. I believe the answers to most of life's (and a company's) questions do not come from within; they come from constructively collaborating with others; that is why a team is often greater than the sum of its parts.

• That said, I think this test is awesome. I love the idea, and I enjoyed noodling on some of the problems just for the fun of it. Kudos to Google for taking the first 10-digit prime found in consecutive digits of e billboard up a notch. By the way, finding the answer to that question is not brain surgery, but it sure does make one feel smart for solving it because for a moment you think to yourself, "yeah, I'm smart enough to work at Google", before going back to whatever it was you were doing before you fell down that rathole.

• Wouldn't it be cool if there were also a Google Labs Attitude Test? :)

If, after reading the following questions, you decide you're Google Caliber, by all means apply for a Google job, because heaven knows that with only 2200 employees they'll need to do more recruiting before they can be "the biggest and most important company in the world in 5-8 years".

Google Labs Aptitude Test:

1. Solve this cryptic equation, realizing of course that values for M and E could be interchanged. No leading zeroes are allowed.

   WWWDOT - GOOGLE = DOTCOM

2. Write a haiku describing possible methods for predicting search traffic seasonality.

3. What's the next line?

   1
   1 1
   2 1
   1 2 1 1
   1 1 1 2 2 1

4. You are in a maze of twisty little passages, all alike. There is a dusty laptop here with a weak wireless connection. There are dull, lifeless gnomes strolling about. What dost thou do?

   A) Wander aimlessly, bumping into obstacles until you are eaten by a grue.
   B) Use the laptop as a digging device to tunnel to the next level.
   C) Play MPoRPG until the battery dies along with your hopes.
   D) Use the computer to map the nodes of the maze and discover an exit path.
   E) Email your resume to Google, tell the lead gnome you quit and find yourself in a whole different world

   (editor's note: I always thought the acronym was MMoRPG, not MPoRPG?)

5. What's broken with Unix?

   (editor's note: I would love to hear Allan's answer to this question...)

   How would you fix it?

   (editor's note: Allan's pontification parable rave about this question is far better than an actual answer to this question...)

6. On your first day at Google, you discover that your cubicle mate wrote the textbook you used as a primary resource in your first year of graduate school. Do you:

   A) Fawn obsequiously and ask if you can have an autograph.
   B) Sit perfectly still and use only soft keystrokes to avoid disturbing her concentration
   C) Leave her daily offerings of granola and English toffee from the food bins.
   D) Quote your favorite formula from the textbook and explain how it's now your mantra.
   E) Show her how example 17b could have been solved with 34 fewer lines of code.

7. Which of the following expresses Google's over-arching philosophy?

   A) "I'm feeling lucky"
   B) "Don't be evil"
   C) "Oh, I already fixed that"
   D) "You should never be more than 50 feet from food"
   E) All of the above

8. How many different ways can you color an icosahedron with one of three colors on each face?

   (editor's note: "An icosahedron is not-necessarily a regular 20-faced polyhedron! Examples include the regular icosahedron, Jessen's orthogonal icosahedron, rhombic icosahedron, 18-sided prism, 19-sided pyramid, or 10-sided dipyramid..." Jiminy crickets, am I a geek...)

   What colors would you choose?

9. This space is intentionally blank. Please fill it with something that improves upon emptiness.
   
   

10. On an infinite, two-dimensional, rectangular lattice of 1-ohm resistors, what is the resistance between two nodes that are a knight's move away?

11. It's 2pm on a sunny Sunday afternoon in the Bay Area. You're minutes from the Pacific Ocean, redwood forest hiking trails and world class cultural attractions. What do you do?

    (editor's note: I should stop interrupting this test with editor's notes, but I cannot help but ask myself if there's any possible answer to this question better than do some laundry....)

12. In your opinion, what is the most beautiful math equation ever derived?

13. Which of the following is NOT an actual interest group formed by Google employees?

    A) Women's basketball
    B) Buffy fans
    C) Cricketeers
    D) Nobel winners
    E) Wine club

14. What will be the next great improvement in search technology?

    (editor's note: It's gotta be Fisher, right? Gotta get me a pocket Fisher, man...)

15. What is the optimal size of a project team, above which additional members do not contribute productivity equivalent to the percentage increase in the staff size?

    A) 1
    B) 3
    C) 5
    D) 11
    E) 24

16. Given a triangle ABC, how would you use only a compass and straight edge to find a point P such that triangles ABP, ACP, and BCP have equal perimeters? (Assume that ABC is constructed so that a solution does exist.)

17. Consider a function which, for a given whole number n, returns the number of ones required when writing out all numbers between 0 and n. For example, f(13) = 6. Notice that f(1) = 1. What is the next largest n such that f(n) = n?

18. What's the coolest hack you've ever written?

19. 'Tis known in refined company, that choosing K things out of N can be done in ways as many as choosing N minus K from N: I pick K, you the remaining.

    Find though a cooler bijection, where you show a knack uncanny, of making your choices contain all K of mine. Oh, for pedantry: let K be no more than half N.

20. What number comes next in the sequence: 10, 9, 60, 90, 70, 66, ?

    A) 96
    B) 10 to the 100th power
    C) Either of the above
    D) None of the above

21. In 29 words or fewer, describe what you would strive to accomplish if you worked at Google Labs.

    (final editor's note: I still have 23 words left over after scribbling "don't be evil; do some laundry"... any suggestions?)
    

Note that cruftbox has a scan of the GLAT in stylish green-and-white jpg's. (Spoiler alert: some of the test's questions are answered in the comments of that post.)

I wonder how they'd feel if I submitted the form completely written in collage-style, cutting and pasting passages from newspapers and magazines. Or if I answered all the questions in crayon.

By the way, you can find at least two answers on Dion's blog. And I do agree with Mark Szpakowski's answer to the most beautiful equation ever derived:

Most beautiful equation ever derived? It's gotta be:

(e**(i*pi)) + 1 = 0

Brings together e (the Google [IPO offering] valuation :-), pi, i, 1 and 0, not to mention equality, addition and multiplication/exponentiation. What more can you ask for?

I try to grok this in fullness.

Euler, you magnificent bastard, all I can say is, Amen, brother. And remember, kids, don't drink and derive...

Comments

[Elizabeth]

I'm never going to be employed again.

[Adam]

Doesn't the IRS owe you money? Clearly you know their job better than them, so clearly they should hire you...

[Elizabeth]

Whoa...you DO read me.

I only know that they owe me money because TurboTax told me so. So for the most part I'm just putting my faith (and numbers) in Intuit. Who will win - TT vs. IRS? I guess we'll see.

[Adam]

Of course I read you.

Your question inspired me to look up TurboTax's How to Survive an IRS Audit, which links to IRS Publication 1: Your Rights as a Taxpayer.

Which didn't answer your question at all ("do I trust Intuit or the IRS?").

The Web is a marvellous place for finding things. Unless you want to find the answer to a specific question, in which case, you takes your chances...

[Timboy]

Agreed about the dire consequences of false positives. But there's something both creepy and (I'm guessing) counterproductive about doing the screening with the same kind of test all the time on all the people you hire.

If you screen everyone with wonky puzzle questions, and insist that anyone who seemingly does badly on them is dinged, then you're going to get a company full of people who like to solve wonky puzzles, and nobody else. This might sound like heaven (and I like puzzles like that myself), but are you sure that you don't need a little bit more diversity in your ecosystem than that?

[DF]

I have to admit that 'no false positives' kinda bugs me. I mean, yeah, you do want no false positives. But *one* dissenting interview?

*shrug*

Well, that explains why--while doing highly relevant research, and being reccomended internally by four different current employees--I got dropped like a hot potato from the interview process.

It seems an odd sort of optimization, unless you really assume that good people are pretty much interchangable.

About cheating: my favorite final exam in grad school was in a Mathematical Logic course --- five problems, take-home for one week, collaboration encouraged. The only rules were: credit everyone you collaborated with, and turn in your own write-up (on the theory that if you can hold it in your mind for long enough to write it up, then you understand it).

The professor's rationale for all this was: mathematics is a fundamentally social activity. (A minority view, admittedly.)

[Adam]

Timboy wrote: "...then you're going to get a company full of people who like to solve wonky puzzles, and nobody else. This might sound like heaven (and I like puzzles like that myself), but are you sure that you don't need a little bit more diversity in your ecosystem than that?"

And if your monoculture is the result of inbreeding, you just might be a redneck Googleite.

Danyelf wrote: "It seems an odd sort of optimization, unless you really assume that good people are pretty much interchangable."

As Tim O'Reilly said, "no matter how big you are, all the smart people don't work for you."

Anonymous wrote: "The only rules were: credit everyone you collaborated with, and turn in your own write-up... Mathematics is a fundamentally social activity."

That's so cool. I wish more teachers possessed that kind of enlightenment.

And, for fun, let's check out the Gbrowser rumors: "Google has done some high profile hiring in the recent days. 4 people who worked on Internet Explorer (one being Adam Bosworth), another one from Java lead developers (Joshua Bloch) and another guy (Joe Beda) who was working on future Microsoft technologies like Avalon and Longhorn. It is also notable that Google now holds ownership of the domain name GBrowser.com leading to more speculation of a browser from Google’s stable." As notable as a Googlewhack. Scratch that. As notable as a book on googlewhacking...

[jeff]

Unless you went to MIT, Stanford, or a 1st tier engineering school with a MS, Ph.D, Google won't even consider your resume submission. That's submission, not consideration to even glance at your resume. Even if it's only for a mid-level software position! Frankly, with that kind of 'great thou art' attitude, who wants to work for such a pretentious arrogant company?

[Adam]

Who wants to work for such a company?

At least 2400 people, when last I counted... :)

[Tanko]

Problem with silly tests like this is Google will have a bunch of book smart people, but very few creative people that are needed to come up with ideas from no-where. I'm sick of egotistical programmers that think the whole world revolves around technical smarts. There are a lot of different kinds of smarts in the world. It would be nice if a company of Google's size would invest in diversity.

When you're in a fast moving business, it's good to have people that rock the boat with new off-the-wall ideas, not people self-selected to think exactly like you.

2 cents from someone that would never get hired at Google.

[Adam]

Every single person I know at Google is both book smart and creative. It's true that I only know 24 of the 2400 folks at Google, but I'd like to think they're a representative slice of the population.

[jeff]

I"m just a straight shooter with an ultimate dream of doing absolutely nothing. :)

[Hann Channing]

Has not anybody realized that Google "test" is just another marketing questionnare? They are segmenting their target markets! for Planck sake!

[Adam]

Of course it's just marketing. And beautiful marketing at that! By the way, now you can officially get a beautiful green-and-white copy of the GLAT from Google.

Jeff, your dreams of doing nothing remind me of the movie Office Space:

Peter Gibbons : What would you do if you had a million dollars?
Lawrence : I'll tell you what I'd do, man, two chicks at the same time, man.
Peter Gibbons : That's it? If you had a million dollars, you'd do two chicks at the same time?
Lawrence : Damn straight. I always wanted to do that, man. And I think if I had a million dollars I could hook that up, cause chicks dig a dude with money.
Peter Gibbons : Well, not all chicks.
Lawrence : Well the kind of chicks that'd double up on a dude like me do.
Peter Gibbons : Good point.
Lawrence : Well what about you now? what would you do?
Peter Gibbons : Besides two chicks at the same time?
Lawrence : Well yeah.
Peter Gibbons : Nothing.
Lawrence : Nothing, huh?
Peter Gibbons : I'd relax, I would sit on my ass all day, I would do nothing.
Lawrence : Well you don't need a million dollars to do nothing, man. Just take a look at my cousin, he's broke, don't do shit.

[jeff]

http://www.ebaumsworld.com/officespace.html

[lynn]

dear editor,

re: question 11 - by 2:00 pm the washing should be on the line, possibly even dry!

[Adam]

Wow, thanks Lynn!

(As an aside, I got very excited when I discovered there was a Laundry Web Service until I realized that it wasn't quite a Web Service, but rather a Laundry Service with the name Web. We techies are so easily led astray... :)

And Jeff, that Office Space soundboard is hilarious...

[Kannappan]

Hey Am unable to find answers to any of the logical questions...Dumb Me...

[dan]

The scary thing about question 6 is why does a company with that much respect for
intellect cram its workers in a cubicle?
(Worse yet, two to a cube.)

[Adam]

Unfortunately, in Silicon Valley cubes are the norm, Dan. :(

Kannappan, which logical question would you like the answer to? Perhaps Google answers can help... :)

By the way, there's a great reference to this post by Peter Coffee at eWeek, which I'll quote here:

As CommerceNet Fellow Adam Rifkin has observed, Google had better hope for a vigorous response, because the company will need a lot of smart and hard-working people to live up to Web designer Jason Kottke's April prediction that the company will be "the biggest and most important company in the world in 5-8 years." That stems from a view of Google, not as mere search engine, but as a versatile services platform that's backed by huge amounts of exceptionally cost-effective computation: Kottke, in turn, points to Topix.net founder/CEO Rich Skrenta's April characterization of Google as "the world's biggest computer and most advanced operating system."

[George Bush]

Google sucks..this is the worst search engine ever created.With no interface to visualzie the creativity...
Sucks! sucks ! sucks

[Adam]

A vote for Bush is a vote against Google? ;)

[Howard Robbins]

GLAT Problem: Given a triangle ABC, how would you use only a compass and straight edge to find a point P such that triangles ABP, ACP, and BCP have equal perimeters? (Assume that ABC is constructed so that a solution does exist.)

ANSWER: Draw a circle about point A with radius BC, a circle about point B with radius AC, and a circle about point C with radius AB. Then the stated problem is almost equivalent to the classic problem of constructing a circle that is tangent to three given circles. I say "almost" because not all solutions of the three-circle problem are acceptable solutions of the GLAT problem. Solutions of the 3-circles problem can be found as follows: Increase or decrease the radii of all 3 circles an increment D, which is chosen to make two of the circles become tangent to each other. Then an inversion [a geometric transformation that replaces radii by their reciprocals] about the point of tangency converts 2 of the 3 circles into a pair of parallel lines, and converts the 3rd circle into a new circle. Find a circle that is tangent to both of these lines, and is also tangent to the transformed version of the 3rd circle. [This is trivially easy, but gives multiple solutions.] Then inverting its 3 points of tangency gives 3 points on a circle that solves the 3-circle problem (after adjusting its radius by increment D.) If its tangencies are all external, or all internal, to the 3 original circles, it also solves the GLAT problem: its center is point P.

[Adam]

Howard, that's beautiful. If I worked at Google, I would hire you on the spot. Alas, I do not work at Google...