{"id":12,"date":"2018-10-05T11:46:51","date_gmt":"2018-10-05T11:46:51","guid":{"rendered":"http:\/\/mikecosgrave.com\/annotation\/?page_id=12"},"modified":"2018-10-15T12:33:55","modified_gmt":"2018-10-15T12:33:55","slug":"ai","status":"publish","type":"page","link":"https:\/\/mikecosgrave.com\/annotation\/ai\/","title":{"rendered":"AI"},"content":{"rendered":"<div class=\"container font-size5 content-width3\">\n<div class=\"header reader-header reader-show-element\"><a class=\"domain reader-domain\" href=\"https:\/\/www.huffingtonpost.com\/wait-but-why\/the-ai-revolution-the-road-to-superintelligence_b_6648480.html?guccounter=1\">huffingtonpost.com<\/a><\/p>\n<div class=\"domain-border\"><\/div>\n<h1 class=\"reader-title\">The AI Revolution: The Road to Superintelligence<\/h1>\n<div class=\"credits reader-credits\"><\/div>\n<div class=\"meta-data\">\n<div class=\"reader-estimated-time\">41-52 minutes<\/div>\n<\/div>\n<\/div>\n<hr \/>\n<div class=\"content\">\n<div class=\"moz-reader-content line-height4 reader-show-element\">\n<div id=\"readability-page-1\" class=\"page\">\n<div id=\"entry-text\" data-rapid-parsed=\"sec\">\n<p><em><big>Artificial superintelligence is coming, probably whether we like it or not, and probably within our lifetimes. If many of the experts are correct, this will either be our greatest dream or our worst nightmare. Here\u2019s why.<\/big><\/em><\/p>\n<p><em>\u201cWe are on the edge of change comparable to the rise of human life on Earth.\u201d<br \/>\n\u2014Vernor Vinge<\/em><\/p>\n<p>What does it feel like to stand here?<\/p>\n<p>It seems like a pretty intense place to be standing \u2014 but then you have to remember something about what it\u2019s like to stand on a time graph: You can\u2019t see what\u2019s to your right. So here\u2019s how it actually feels to stand there:<\/p>\n<p>Which probably feels pretty normal.<\/p>\n<p><strong><em>The Far Future: Coming Soon<\/em><\/strong><\/p>\n<p>Imagine taking a time machine back to 1750, a time when the world was in a permanent power outage, long-distance communication meant either yelling loudly or firing a cannon in the air, and all transportation ran on hay. When you get there, you retrieve a dude, bring him to 2015, and then walk him around and watch him react to everything. It\u2019s impossible for us to understand what it would be like for him to see shiny capsules racing by on a highway, talk to people who had been on the other side of the ocean earlier in the day, watch sports that were being played 1,000 miles away, hear a musical performance that happened 50 years ago, and play with my magical wizard rectangle, which he could use to capture a real-life image or record a living moment, generate a map with a paranormal moving blue dot that shows him where he is, look at someone\u2019s face and chat with them even though they\u2019re on the other side of the country, and worlds of other inconceivable sorcery. This is all before you show him the Internet or explain things like the International Space Station, the Large Hadron Collider, nuclear weapons, or general relativity.<\/p>\n<p>This experience for him wouldn\u2019t be surprising or shocking or even mind-blowing; those words aren\u2019t big enough. He might actually die.<\/p>\n<p>But here\u2019s the interesting thing: If he then went back to 1750 and got envious that we\u2019d gotten to see his reaction and decided he wanted to try the same thing, he\u2019d take the time machine and go back the same distance, get someone from around the year 1500, bring him to 1750, and show him everything. And the 1500 guy would be shocked by a lot of things \u2014 but he wouldn\u2019t die. It would be <em>far <\/em>less of an insane experience for him, because while 1500 and 1750 were very different, they were <em>much less<\/em> different from each other than 1750 and 2015 are. The 1500 guy would learn some mind-bending shit about space and physics, he\u2019d be impressed by how committed to that new imperialism fad Europe turned out to be, and he\u2019d have to do some major revisions of his world-map conception. But watching everyday life \u2014 transportation, communication, etc. \u2014 go by in 1750 definitely wouldn\u2019t make him die.<\/p>\n<p>No, in order for the 1750 guy to have as much fun as we had with him, he\u2019d have to go much further back \u2014 maybe all the way back to about 12,000 B.C., before the First Agricultural Revolution gave rise to the first cities and to the concept of civilization. If someone from a purely hunter-gatherer world \u2014 from a time when humans were, more or less, just another animal species \u2014 saw the vast human empires of 1750, with their towering churches, their ocean-crossing ships, their concept of being \u201cinside,\u201d and their enormous mountain of collective, accumulated human knowledge and discovery, he\u2019d likely die.<\/p>\n<p>And then what if, after dying, <em>he<\/em> got envious and wanted to do the same thing? If he went back 12,000 years to 24,000 B.C. and got a guy and brought him to 12,000 B.C., he\u2019d show the guy everything, and the guy would be like, \u201cOK, what\u2019s your point? Who cares?\u201d For the 12,000 B.C. guy to have the same fun, he\u2019d have to go back over 100,000 years and get someone he could show fire and language to for the first time.<\/p>\n<p>In order for someone to be transported into the future and die from the level of shock they\u2019d experience, they\u2019d have to go enough years ahead that a \u201cdie level of progress,\u201d or a Die Progress Unit (DPU), has been achieved. So a DPU took over 100,000 years in hunter-gatherer times, but at the post-Agricultural Revolution rate it only took about 12,000 years. The post-Industrial Revolution world has moved so quickly that a 1750 person only needs to go forward a couple hundred years for a DPU to have happened.<\/p>\n<p>This pattern \u2014 human progress moving increasingly quickly as time goes on \u2014 is what futurist Ray Kurzweil calls human history\u2019s law of accelerating returns. This happens because more advanced societies have the ability to progress at a faster <em>rate<\/em> than less advanced societies \u2014 <em>because<\/em> they\u2019re more advanced. Nineteenth-century humanity knew more and had better technology than 15th-century humanity, so it\u2019s no surprise that humanity made far more advances in the 19th century than in the 15th century; 15th-century humanity was no match for 19th-century humanity.<\/p>\n<p>This works on smaller scales too. The movie <em>Back to the Future<\/em> came out in 1985, and \u201cthe past\u201d took place in 1955. In the movie, when Michael J. Fox went back to 1955, he was caught off-guard by the newness of TVs, the prices of soda, the lack of love for shrill electric guitar, and the variation in slang. It was a different world, yes, but if the movie were made today and the past took place in 1985, the movie could have had <em>much <\/em>more fun with <em>much <\/em>bigger differences. The character would be in a time before personal computers, Internet, or cellphones; today\u2019s Marty McFly, a teenager born in the late \u201890s, would be much more out of place in 1985 than the movie\u2019s Marty McFly was in 1955.<\/p>\n<p>This is for the same reason we just discussed: the law of accelerating returns. The average rate of advancement between 1985 and 2015 was higher than the rate between 1955 and 1985, because the former was a more advanced world, so much more change happened in the most recent 30 years than in the prior 30.<\/p>\n<p>So advances are getting bigger and bigger and happening more and more quickly. This suggests some pretty intense things about our future, right?<\/p>\n<p>Kurzweil <a href=\"http:\/\/books.google.com\/books\/about\/The_Singularity_Is_Near.html?id=9FtnppNpsT4C\" target=\"_hplink\" rel=\"nofollow\" data-ylk=\"subsec:paragraph;cpos:19;elm:context_link;itc:0\" data-rapid-parsed=\"slk\" data-rapid_p=\"3\" data-v9y=\"0\">suggests<\/a> that the progress of the entire 20th century would have been achieved in only 20 years at the rate of advancement in the year 2000; in other words, by 2000, the rate of progress was five times faster than the <em>average <\/em>rate of progress during the 20th century. He believes another 20th century\u2019s worth of progress happened between 2000 and 2014, and that <em>another <\/em>20th century\u2019s worth of progress will happen by 2021, in only seven years. He believes that a couple of decades later, a 20th century\u2019s worth of progress will happen multiple times in the same year, and even later, in less than one month. All in all, Kurzweil believes that because of the law of accelerating returns, the 21st century will achieve 1,000 times the progress of the 20th century.<\/p>\n<p>If Kurzweil and others who agree with him are correct, then we may be as blown away by 2030 as our 1750 guy was by 2015 (<em>i.e.<\/em>, the next DPU might only take a couple of decades), and the world in 2050 might be <em>so <\/em>vastly different from today\u2019s world that we would barely recognize it.<\/p>\n<p>This isn\u2019t science fiction. It\u2019s what many scientists smarter and more knowledgeable than you or I firmly believe \u2014 and if you look at history, it\u2019s what we should logically predict.<\/p>\n<p>So then why, when you hear me say something like, \u201cThe world 35 years from now might be totally unrecognizable,\u201d are you thinking, \u201cCool, but <em>naaah<\/em>\u201c? Three reasons we\u2019re skeptical of outlandish forecasts of the future:<\/p>\n<p><em>1) When it comes to history, we think in straight lines.<\/em> When we imagine the progress of the next 30 years, we look back to the progress of the previous 30 as an indicator of how much will likely happen. When we think about the extent to which the world will change in the 21st century, we just take the 20th-century progress and add it to the year 2000. This was the same mistake our 1750 guy made when he got someone from 1500 and expected to blow his mind as much as his own had been blown by going the same distance ahead. It\u2019s most intuitive for us to think <em>linearly<\/em>, when we should be thinking <em>exponentially<\/em>. If someone is being cleverer about it, they might predict the advances of the next 30 years not by looking at the previous 30 years but by taking the <em>current <\/em>rate of progress and judging based on that. They\u2019d be more accurate but still way off. In order to think about the future correctly, you need to imagine things moving at a <em>much faster rate <\/em>than they\u2019re moving now.<\/p>\n<p><em>2) The trajectory of very recent history often tells a distorted story. <\/em>First, even a steep exponential curve seems linear when you only look at a tiny slice of it, the same way a huge circle looks almost like a straight line if you look at a little segment of it up close. Second, exponential growth isn\u2019t totally smooth and uniform. Kurzweil <a href=\"http:\/\/books.google.com\/books\/about\/The_Singularity_Is_Near.html?id=9FtnppNpsT4C\" target=\"_hplink\" rel=\"nofollow\" data-ylk=\"subsec:paragraph;cpos:25;elm:context_link;itc:0\" data-rapid-parsed=\"slk\" data-rapid_p=\"5\" data-v9y=\"0\">explains<\/a> that progress happens in \u201cS-curves\u201d:<\/p>\n<p>An \u201cS\u201d is created by the wave of progress when a new paradigm sweeps the world. The curve goes through three phases:<\/p>\n<div data-rapid-cpos=\"28\" data-rapid-subsec=\"paragraph\" data-rapid-parsed=\"subsec\">\n<ol>\n<li>Slow growth (the early phase of exponential growth)<\/li>\n<\/ol>\n<\/div>\n<ul>\n<li>Rapid growth (the late, explosive phase of exponential growth)<\/li>\n<li>A leveling off as the particular paradigm matures<\/li>\n<\/ul>\n<p>If you look only at very recent history, the part of the S-curve you\u2019re on at the moment can obscure your perception of how quickly things are advancing. The chunk of time between 1995 and 2007 saw the explosion of the Internet; the introduction of Microsoft, Google, and Facebook into the public consciousness; the birth of social networking; and the introduction of cellphones and then smartphones. That was Phase 2, the growth-spurt part of the S. But 2008 to 2015 has been less groundbreaking, at least on the technological front. Someone thinking about the future today might examine the last few years to gauge the current rate of advancement, but that\u2019s missing the bigger picture. In fact, a new, huge Phase 2 growth spurt might be brewing right now.<\/p>\n<p><em>3) Our own experience makes us stubborn old men about the future. <\/em>We base our ideas about the world on our personal experience, and that experience has ingrained the rate of growth of the recent past in our heads as \u201cthe way things happen.\u201d We\u2019re also limited by our imagination, which takes our experience and uses it to conjure future predictions \u2014 but often, what we know simply doesn\u2019t give us the tools to think accurately about the future.<sup>1<\/sup> When we hear a prediction about the future that contradicts our experience-based notion of how things work, our instinct is that the prediction must be naive. If I tell you, later in this post, that you may live to be 150, or 250, or <em>not die at all<\/em>, your instinct will be, \u201cThat\u2019s stupid. If there\u2019s one thing I know from history, it\u2019s that everybody dies.\u201d And yes, no one in the past has not died. But no one flew airplanes before airplanes were invented either.<\/p>\n<p>So while \u201c<em>naaah<\/em>\u201c might feel right as you read this post, it\u2019s probably actually wrong. The fact is that if we\u2019re being truly logical and expecting historical patterns to continue, we should conclude that much, much, <em>much <\/em>more should change in the coming decades than we intuitively expect. Logic also suggests that if the most advanced species on a planet keeps making larger and larger leaps forward at an ever-faster rate, at some point, they\u2019ll make a leap so great that it completely alters life as they know it and the perception they have of what it means to be a human \u2014 kind of like how evolution kept making great leaps toward intelligence until finally it made such a large leap to the human being that it completely altered what it means for any creature to live on planet Earth. And if you spend some time reading about what\u2019s going on today in science and technology, you start to see a lot of signs quietly hinting that life as we know it cannot withstand the leap that\u2019s coming next.<\/p>\n<p><strong><em>The Road to Superintelligence<\/em><\/strong><\/p>\n<p><strong>What Is AI?<\/strong><\/p>\n<p>If you\u2019re like me, you used to think artificial intelligence was a silly sci-fi concept, but lately you\u2019ve been hearing it mentioned by serious people, and you don\u2019t quite get it.<\/p>\n<p>There are three reasons a lot of people are confused about the term \u201cAI\u201d:<\/p>\n<p><em>1) We associate AI with movies.<\/em> <em>Star Wars<\/em>. <em>Terminator<\/em>. <em>2001: A Space Odyssey<\/em>. Even <em>The Jetsons<\/em>. Those are fiction, as are the robot characters, so it makes AI sound a little fictional to us.<\/p>\n<p><em>2) \u201cAI\u201d is a broad topic.<\/em> It ranges from your phone\u2019s calculator to self-driving cars to something in the future that might change the world dramatically. \u201cAI\u201d refers to all of these things, which is confusing.<\/p>\n<p><em>3) We use AI all the time in our daily lives, but we often don\u2019t realize it\u2019s AI.<\/em> John McCarthy, who coined the term \u201cartificial intelligence\u201d in 1956, <a href=\"http:\/\/cacm.acm.org\/magazines\/2012\/1\/144824-artificial-intelligence-past-and-future\/fulltext\" target=\"_hplink\" rel=\"nofollow\" data-ylk=\"subsec:paragraph;cpos:41;elm:context_link;itc:0\" data-rapid-parsed=\"slk\" data-rapid_p=\"8\" data-v9y=\"0\">complained<\/a>, \u201cAs soon as it works, no one calls it AI anymore.\u201d Because of this phenomenon, AI often sounds more like a mythical future prediction than a reality. At the same time, it makes it sound like a pop concept from the past that never came to fruition. Ray Kurzweil says he hears people say that AI withered in the 1980s, which he <a href=\"http:\/\/books.google.com\/books\/about\/The_Singularity_Is_Near.html?id=9FtnppNpsT4C\" target=\"_hplink\" rel=\"nofollow\" data-ylk=\"subsec:paragraph;cpos:41;elm:context_link;itc:0\" data-rapid-parsed=\"slk\" data-rapid_p=\"9\" data-v9y=\"0\">compares<\/a> to \u201cinsisting that the Internet died in the dot-com bust of the early 2000s.\u201d<\/p>\n<p>So let\u2019s clear things up. First, stop thinking of <em>robots<\/em>. A robot is a <em>container <\/em>for AI \u2014 sometimes mimicking the human form, sometimes not \u2014 but the AI itself is the computer <em>inside <\/em>the robot. AI is the brain, and the robot is its body \u2014 if it even has a body. For example, the software and data behind Siri is AI, the woman\u2019s voice we hear is a personification of that AI, and there\u2019s no robot involved at all.<\/p>\n<p>Secondly, you\u2019ve probably heard the term \u201csingularity\u201d or \u201ctechnological singularity.\u201d This term has been used in math to describe an asymptote-like situation where normal rules no longer apply. It\u2019s been used in physics to describe a phenomenon like an infinitely small, dense black hole or the point we were all squished into right before the Big Bang. Again, situations where the usual rules don\u2019t apply. In 1993, Vernor Vinge wrote a <a href=\"https:\/\/www-rohan.sdsu.edu\/faculty\/vinge\/misc\/singularity.html\" target=\"_blank\" rel=\"nofollow noopener\" data-ylk=\"subsec:paragraph;cpos:43;elm:context_link;itc:0\" data-rapid-parsed=\"slk\" data-rapid_p=\"10\" data-v9y=\"0\">famous essay<\/a> in which he applied the term to the moment in the future when our technology\u2019s intelligence exceeds our own \u2014 a moment for him when life as we know it will be forever changed and normal rules will no longer apply. Ray Kurzweil then muddled things a bit by defining the singularity as the time when the law of accelerating returns has reached such an extreme pace that technological progress is happening at a seemingly infinite pace, and after which we\u2019ll be living in a whole new world. I found that many of today\u2019s AI thinkers have stopped using the term, and it\u2019s confusing anyway, so I won\u2019t use it much here (even though we\u2019ll be focusing on that <em>idea<\/em> throughout).<\/p>\n<p>Finally, while there are many different types or forms of AI, since \u201cAI\u201d is a broad concept, the critical categories we need to think about are based on an AI\u2019s <em>caliber<\/em>. There are three major AI caliber categories:<\/p>\n<p><em>1) Artificial narrow intelligence (ANI):<\/em> Sometimes referred to as \u201cweak AI,\u201d artificial narrow intelligence is AI that specializes in <em>one<\/em> area. There\u2019s AI that can beat the world chess champion in chess, but that\u2019s the only thing it does. Ask it to figure out a better way to store data on a hard drive and it\u2019ll look at you blankly.<\/p>\n<p><em>2) Artificial general intelligence (AGI):<\/em> Sometimes referred to as \u201cstrong AI,\u201d or \u201chuman-level AI,\u201d \u201cartificial general intelligence\u201d refers to a computer that is as smart as a human <em>across the board<\/em> \u2014 a machine that can perform any intellectual task that a human being can. Creating an AGI is a <em>much <\/em>harder task than creating an ANI, and we\u2019ve yet to do it. Professor Linda Gottfredson describes intelligence as \u201ca very general mental capability that, among other things, involves the ability to reason, plan, solve problems, think abstractly, comprehend complex ideas, learn quickly, and learn from experience.\u201d An AGI would be able to do all those things as easily as you can.<\/p>\n<p><em>3) Artificial superintelligence (ASI):<\/em> Oxford philosopher and leading AI thinker Nick Bostrom <a href=\"http:\/\/www.nickbostrom.com\/superintelligence.html\" target=\"_blank\" rel=\"nofollow noopener\" data-ylk=\"subsec:paragraph;cpos:47;elm:context_link;itc:0\" data-rapid-parsed=\"slk\" data-rapid_p=\"11\" data-v9y=\"0\">defines<\/a> \u201csuperintelligence\u201d as \u201can intellect that is much smarter than the best human brains in practically every field, including scientific creativity, general wisdom and social skills.\u201d Artificial superintelligence ranges from a computer that\u2019s just a little smarter than a human to one that\u2019s trillions of times smarter \u2014 across the board. ASI is the reason that the topic of AI is such a spicy meatball, and the reason that the words \u201cimmortality\u201d and \u201cextinction\u201d will both appear in these posts multiple times.<\/p>\n<p>As of now, humans have conquered the lowest caliber of AI \u2014 ANI \u2014 in many ways, and it\u2019s everywhere. The AI revolution is the road from ANI through AGI to ASI \u2014 a road that we may or may not survive but that, either way, will change everything.<\/p>\n<p>Let\u2019s take a close look at what the leading thinkers in the field believe this road looks like and why this revolution might happen way sooner than you might think.<\/p>\n<p><strong>Where We Are Currently: A World Running on ANI<\/strong><\/p>\n<p>Artificial narrow intelligence is machine intelligence that equals or exceeds human intelligence or efficiency at a <em>specific <\/em>thing. A few examples:<\/p>\n<div data-rapid-cpos=\"52\" data-rapid-subsec=\"paragraph\" data-rapid-parsed=\"subsec\">\n<ul>\n<li>Cars are full of ANI systems, from the computer that figures out when the anti-lock brakes should kick in to the computer that tunes the parameters of the fuel-injection systems. Google\u2019s <a href=\"https:\/\/www.youtube.com\/channel\/UCCLyNDhxwpqNe3UeEmGHl8g\" target=\"_blank\" rel=\"nofollow noopener\" data-ylk=\"subsec:paragraph;cpos:52;elm:context_link;itc:0\" data-rapid-parsed=\"slk\" data-rapid_p=\"12\" data-v9y=\"0\">self-driving car<\/a>, which is being tested now, will contain robust ANI systems that allow it to perceive and react to the world around it.<\/li>\n<\/ul>\n<\/div>\n<ul>\n<li>Your phone is a little ANI factory. When you navigate using your map app, receive tailored music recommendations from Pandora, check tomorrow\u2019s weather, talk to Siri, or engage in dozens of other everyday activities, you\u2019re using ANI.<\/li>\n<li>Your email spam filter is a classic type of ANI. It starts off loaded with intelligence about how to figure out what\u2019s spam and what\u2019s not, and then it learns and tailors its intelligence to you as it gets experience with your particular preferences. The Nest thermostat does the same thing as it starts to figure out your typical routine and act accordingly.<\/li>\n<li>You know the whole creepy thing that goes on when you search for a product on Amazon and then you see that as a \u201crecommended for you\u201d product on a <em>different <\/em>site, or when Facebook somehow knows whom it makes sense for you to add as a friend? That\u2019s a network of ANI systems working together to inform each other about who you are and what you like and then using that information to decide what to show you. The same goes for Amazon\u2019s \u201cPeople who bought this also bought&#8230;\u201d thing: That\u2019s an ANI whose job it is to gather info from the behavior of millions of customers and synthesize that info to cleverly upsell you so you\u2019ll buy more things.<\/li>\n<li>Google Translate is another classic ANI, impressively good at one narrow task. Voice recognition is another, and there are a bunch of apps that use those two ANIs as a tag team, allowing you to speak a sentence in one language and have the phone spit out the same sentence in another.<\/li>\n<li>When your plane lands, it\u2019s not a human that decides which gate it should go to, just like it\u2019s not a human that determined the price of your ticket.<\/li>\n<li>The world\u2019s best checkers, chess, Scrabble, backgammon, and Othello players are now all ANI systems.<\/li>\n<li>Google Search is one large ANI brain with incredibly sophisticated methods for ranking pages and figuring out what to show you in particular. The same goes for Facebook\u2019s Newsfeed.<\/li>\n<\/ul>\n<p>And those are just in the consumer world. Sophisticated ANI systems are widely used in sectors and industries like military, manufacturing, and finance (algorithmic high-frequency AI traders account for <a href=\"http:\/\/books.google.com\/books\/about\/Superintelligence.html?id=7_H8AwAAQBAJ\" target=\"_hplink\" rel=\"nofollow\" data-ylk=\"subsec:paragraph;cpos:66;elm:context_link;itc:0\" data-rapid-parsed=\"slk\" data-rapid_p=\"13\" data-v9y=\"0\">more than half<\/a> of equity shares traded on U.S. markets), and in expert systems like those that help doctors make diagnoses, and, most famously, in IBM\u2019s <a href=\"http:\/\/www.ibm.com\/smarterplanet\/us\/en\/ibmwatson\/\" target=\"_blank\" rel=\"nofollow noopener\" data-ylk=\"subsec:paragraph;cpos:66;elm:context_link;itc:0\" data-rapid-parsed=\"slk\" data-rapid_p=\"14\" data-v9y=\"0\">Watson<\/a>, who contained enough facts and understood coy Trebek-speak well enough to soundly beat the most prolific <em>Jeopardy<\/em> champions.<\/p>\n<p>ANI systems as they are now aren\u2019t especially scary. At worst, a glitchy or badly programmed ANI can cause an isolated catastrophe like knocking out a power grid, causing a harmful nuclear-power-plant malfunction, or triggering a financial-markets disaster (like the <a href=\"http:\/\/www.ritholtz.com\/blog\/wp-content\/uploads\/2010\/10\/flash-crash-dow-popup.png\" target=\"_blank\" rel=\"nofollow noopener\" data-ylk=\"subsec:paragraph;cpos:67;elm:context_link;itc:0\" data-rapid-parsed=\"slk\" data-rapid_p=\"15\" data-v9y=\"0\">2010 Flash Crash<\/a>, when an ANI reacted the wrong way to an unexpected situation and caused the stock market to briefly plummet, taking $1 trillion of market value with it, only part of which was recovered when the mistake was corrected).<\/p>\n<p>But while ANI doesn\u2019t have the capability to cause an <em>existential threat<\/em>, we should see this increasingly large and complex ecosystem of relatively harmless ANI as a precursor of the world-altering hurricane that\u2019s on the way. Each new ANI innovation quietly adds another brick onto the road to AGI and ASI. Or as Aaron Saenz <a href=\"http:\/\/singularityhub.com\/2010\/08\/10\/we-live-in-a-jungle-of-artificial-intelligence-that-will-spawn-sentience\/\" target=\"_blank\" rel=\"nofollow noopener\" data-ylk=\"subsec:paragraph;cpos:68;elm:context_link;itc:0\" data-rapid-parsed=\"slk\" data-rapid_p=\"16\" data-v9y=\"0\">sees it<\/a>, our world\u2019s ANI systems \u201care like the amino acids in the early Earth\u2019s primordial ooze\u201d \u2014 the inanimate stuff of life that, one unexpected day, woke up.<\/p>\n<p><em><strong>The Road From ANI to AGI<\/strong><\/em><\/p>\n<p><strong>Why It\u2019s So Hard<\/strong><\/p>\n<p>Nothing will make you appreciate human intelligence like learning about how unbelievably challenging it is to create a computer as smart as we are. Building skyscrapers, putting humans in space, figuring out the details of how the Big Bang went down \u2014 all are far easier than understanding our own brain or how to make something as cool as it. As of now, the human brain is the most complex object in the known universe.<\/p>\n<p>What\u2019s interesting is that the hard parts of trying to build an AGI (a computer as smart as humans in <em>general<\/em>, not just at one narrow specialty) are not intuitively what you\u2019d think they are. Build a computer that can multiply two 10-digit numbers in a split second? Incredibly easy. Build one that can look at a dog and answer whether it\u2019s a dog or a cat? Spectacularly difficult. Make AI that can beat any human in chess? Done. Make one that can read a paragraph from a 6-year-old\u2019s picture book and not just recognize the words but understand the <em>meaning<\/em> of them? Google is currently spending <a href=\"http:\/\/www.wired.com\/2014\/01\/google-buying-way-making-brain-irrelevant\/\" target=\"_blank\" rel=\"nofollow noopener\" data-ylk=\"subsec:paragraph;cpos:72;elm:context_link;itc:0\" data-rapid-parsed=\"slk\" data-rapid_p=\"17\" data-v9y=\"0\">billions<\/a> of dollars trying to do it. Hard things like calculus, financial-market strategy, and language translation are mind-numbingly easy for a computer, while easy things like vision, motion, movement, and perception are insanely hard for it. Or, as computer scientist Donald Knuth <a href=\"http:\/\/books.google.com\/books\/about\/The_Quest_for_Artificial_Intelligence.html?id=EQhfPgAACAAJ\" target=\"_hplink\" rel=\"nofollow\" data-ylk=\"subsec:paragraph;cpos:72;elm:context_link;itc:0\" data-rapid-parsed=\"slk\" data-rapid_p=\"18\" data-v9y=\"0\">puts it<\/a>, \u201cAI has by now succeeded in doing essentially everything that requires \u2018thinking\u2019 but has failed to do most of what people and animals do \u2018without thinking.\u2019\u201d<\/p>\n<p>What you quickly realize when you think about this is that those things that seem easy to us are actually unbelievably complicated, and they only seem easy because those skills have been optimized in us (and most animals) by hundreds of million years of animal evolution. When you reach your hand up toward an object, the muscles, tendons, and bones in your shoulder, elbow, and wrist instantly perform a long series of physics operations, in conjunction with your eyes, to allow you to move your hand in a straight line through three dimensions. It seems effortless to you because you have perfected software in your brain for doing it. The same idea goes for why it\u2019s not that malware is dumb for not being able to figure out the slanty-word-recognition test when you sign up for a new account on a site; it\u2019s that your brain is super-impressive for being <em>able <\/em>to.<\/p>\n<div id=\"inline-newsletter_placeholder\">\n<div data-rapid-subsec=\"newsletter-box\" data-rapid-parsed=\"subsec\">\n<div>\n<div>\n<div>\n<p>Subscribe to The Morning Email.<\/p>\n<p>Wake up to the day&#8217;s most important news.<\/p>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<p>On the other hand, multiplying big numbers or playing chess are new activities for biological creatures, and we haven\u2019t had any time to evolve a proficiency at them, so a computer doesn\u2019t need to work too hard to beat us. Think about it. Which would you rather do: Build a program that could multiply big numbers, or build one that could understand the essence of a \u201cB\u201d well enough that it you could show it a \u201cB\u201d in any one of thousands of unpredictable fonts or handwriting and it could instantly know it was a \u201cB\u201d?<\/p>\n<p>One fun example: When you look at the image below, you and a computer both can figure out that it\u2019s a rectangle with two distinct shades, alternating.<\/p>\n<p>Tied so far. But if you pick up the black and reveal the whole image&#8230;<\/p>\n<p>&#8230;you have no problem giving a full description of the various opaque and translucent cylinders, slats, and 3D corners, but the computer would fail miserably. It would describe what it sees \u2014 a variety of two-dimensional shapes in several different shades \u2014 which is actually what\u2019s there. Your brain is <a href=\"http:\/\/books.google.com\/books\/about\/How_the_Mind_Works.html?id=5cXKQUh6bVQC\" target=\"_hplink\" rel=\"nofollow\" data-ylk=\"subsec:paragraph;cpos:79;elm:context_link;itc:0\" data-rapid-parsed=\"slk\" data-rapid_p=\"26\" data-v9y=\"0\">doing a ton of fancy shit<\/a> to interpret the implied depth, shade mixing, and room lighting the picture is trying to portray. And looking at the picture below, a computer sees a two-dimensional white, black, and gray collage, while you easily see what it really is: a photo of an entirely black, 3D rock.<\/p>\n<p><small>Credit: Matthew Lloyd<\/small><\/p>\n<p>And everything we just mentioned is still only taking in stagnant information and processing it. To be human-level intelligent, a computer would have to understand things like the difference between subtle facial expressions; the distinction between being pleased, relieved, content, satisfied, and glad; and why <em>Braveheart <\/em>was great but <em>The Patriot<\/em> was terrible.<\/p>\n<p>Daunting.<\/p>\n<p>So how do we get there?<\/p>\n<p><strong>First Key to Creating AGI: Increasing Computational Power<\/strong><\/p>\n<p>One thing that definitely needs to happen for AGI to be a possibility is an increase in the power of computer hardware. If an AI system is going to be as intelligent as the brain, it\u2019ll need to equal the brain\u2019s raw computing capacity.<\/p>\n<p>One way to express this capacity is in the total calculations per second (CPS) the brain could manage, and you could come to this number by figuring out the maximum CPS of each structure in the brain and then adding them all together.<\/p>\n<p>Ray Kurzweil came up with a shortcut by taking someone\u2019s professional estimate for the CPS of one structure and that structure\u2019s weight compared with that of the whole brain and then multiplying proportionally to get an estimate for the total. It sounds a little iffy, but he did this a bunch of times with various professional estimates of different regions, and the total always arrived in the same ballpark: around 10<sup>16<\/sup>, or 10 quadrillion CPS.<\/p>\n<p>Currently, the world\u2019s fastest supercomputer, China\u2019s <a href=\"http:\/\/www.reuters.com\/article\/2014\/11\/17\/us-china-supercomputer-idUSKCN0J11VV20141117\" target=\"_blank\" rel=\"nofollow noopener\" data-ylk=\"subsec:paragraph;cpos:89;elm:context_link;itc:0\" data-rapid-parsed=\"slk\" data-rapid_p=\"28\" data-v9y=\"0\">Tianhe-2<\/a>, has actually beaten that number, clocking in at about 34 quadrillion CPS. But Tianhe-2 is also a dick, taking up 720 square meters of space, using 24 megawatts of power (the brain runs on just <a href=\"http:\/\/www.popsci.com\/technology\/article\/2009-11\/neuron-computer-chips-could-overcome-power-limitations-digital\" target=\"_blank\" rel=\"nofollow noopener\" data-ylk=\"subsec:paragraph;cpos:89;elm:context_link;itc:0\" data-rapid-parsed=\"slk\" data-rapid_p=\"29\" data-v9y=\"0\">20 watts<\/a>), and costing $390 million to build. That\u2019s not especially applicable to wide usage, or even most commercial or industrial usage, yet.<\/p>\n<p>Kurzweil suggests that we think about the state of computers by looking at how many CPS you can buy for $1,000. When that number reaches human level \u2014 10 quadrillion CPS \u2014 then that\u2019ll mean AGI could become a very real part of life.<\/p>\n<p><a href=\"http:\/\/www.mooreslaw.org\/\" target=\"_blank\" rel=\"nofollow noopener\" data-ylk=\"subsec:paragraph;cpos:91;elm:context_link;itc:0\" data-rapid-parsed=\"slk\" data-rapid_p=\"30\" data-v9y=\"0\">Moore\u2019s law<\/a> is a historically reliable rule that states that the world\u2019s maximum computing power doubles approximately every two years, meaning computer hardware advancement, like general human advancement through history, grows exponentially. Looking at how this relates to Kurzweil\u2019s CPS-per-$1,000 metric, we\u2019re currently at about 10 trillion CPS per $1,000, right on pace with <a href=\"http:\/\/books.google.com\/books\/about\/The_Singularity_Is_Near.html?id=9FtnppNpsT4C\" target=\"_hplink\" rel=\"nofollow\" data-ylk=\"subsec:paragraph;cpos:91;elm:context_link;itc:0\" data-rapid-parsed=\"slk\" data-rapid_p=\"31\" data-v9y=\"0\">this graph<\/a>\u2018s predicted trajectory:<\/p>\n<p>So the world\u2019s $1,000 computers are now beating the mouse brain, and they\u2019re at about a thousandth of human level. This doesn\u2019t sound like much until you remember that we were at about a trillionth of human level in 1985, a billionth in 1995, and a millionth in 2005. Being at a thousandth in 2015 puts us right on pace to get to an affordable computer that rivals the power of the brain by 2025.<\/p>\n<p>So on the hardware side, the raw power needed for AGI is technically available now, in China, and we\u2019ll be ready for affordable, widespread AGI-caliber hardware within 10 years. But raw computational power alone doesn\u2019t make a computer generally intelligent The next question is: How do we bring human-level intelligence to all that power?<\/p>\n<p><strong>Second Key to Creating AGI: Making It Smart<\/strong><\/p>\n<p>This is the icky part. The truth is that no one really knows how to make it smart; we\u2019re still debating how to make a computer human-level intelligent and capable of knowing what a dog and a weirdly written \u201cB\u201d and a mediocre movie is. But there are a bunch of far-fetched strategies out there, and at some point one of them will work. Here are the three most common strategies I came across.<\/p>\n<p><em>1) Plagiarize the brain:<\/em> This is like scientists toiling over how that kid who sits next to them in class is so smart and keeps doing so well on the tests, and even though they keep studying diligently, they can\u2019t do nearly as well as that kid, and then they finally decide, \u201cOK, fuck it. I\u2019m just gonna copy that kid\u2019s answers.\u201d It makes sense: We\u2019re stumped trying to build a super-complex computer, and there happens to be a perfect prototype for one in each of our heads.<\/p>\n<p>The science world is working hard on reverse engineering the brain to figure out how evolution made such a rad thing; <a href=\"http:\/\/www.wired.com\/2010\/08\/reverse-engineering-brain-kurzweil\/\" target=\"_blank\" rel=\"nofollow noopener\" data-ylk=\"subsec:paragraph;cpos:98;elm:context_link;itc:0\" data-rapid-parsed=\"slk\" data-rapid_p=\"33\" data-v9y=\"0\">optimistic estimates<\/a> say we can do this by 2030. Once we do that, we\u2019ll know all the secrets of how the brain runs so powerfully and efficiently, and we can draw inspiration from it and steal its innovations. One example of computer architecture that mimics the brain is the artificial neural network. It starts out as a network of transistor \u201cneurons,\u201d connected to each other with inputs and outputs, and it knows nothing \u2014 like an infant brain. The way it \u201clearns\u201d is that it tries to do a task \u2014 say, handwriting recognition \u2014 and at first its neural firings and subsequent guesses at deciphering each letter will be completely random. But when it\u2019s told it got something right, the transistor connections in the firing pathways that happened to create that answer are strengthened; when it\u2019s told it was wrong, those pathways\u2019 connections are weakened. After a lot of this trial and feedback, the network has, by itself, formed smart neural pathways, and the machine has become optimized for the task. The brain learns a bit like this but in a more sophisticated way, and as we continue to study the brain, we\u2019re discovering ingenious new ways to take advantage of neural circuitry.<\/p>\n<p>More extreme plagiarism involves a strategy called \u201cwhole-brain emulation,\u201d where the goal is to slice a real brain into thin layers, scan each one, use software to assemble an accurate 3D model, and then implement the model on a powerful computer. We\u2019d then have a computer officially capable of everything the brain is capable of; it would just need to learn and gather information. If engineers get <em>really <\/em>good, they\u2019d be able to emulate a real brain with such exact accuracy that the brain\u2019s full personality and memory would be intact once the brain architecture has been uploaded to a computer. If the brain belonged to Jim right before he passed away, the computer would now wake up as Jim (<a href=\"http:\/\/waitbutwhy.com\/2014\/12\/what-makes-you-you.html\" target=\"_blank\" rel=\"nofollow noopener\" data-ylk=\"subsec:paragraph;cpos:99;elm:context_link;itc:0\" data-rapid-parsed=\"slk\" data-rapid_p=\"34\" data-v9y=\"0\">?<\/a>), which would be a robust human-level AGI, and we could now work on turning Jim into an unimaginably smart ASI, which he\u2019d probably be really excited about.<\/p>\n<p>How far are we from achieving whole-brain emulation? Well, so far we\u2019ve not yet <a href=\"http:\/\/www.smithsonianmag.com\/smart-news\/weve-put-worms-mind-lego-robot-body-180953399\/?no-ist\" target=\"_blank\" rel=\"nofollow noopener\" data-ylk=\"subsec:paragraph;cpos:100;elm:context_link;itc:0\" data-rapid-parsed=\"slk\" data-rapid_p=\"35\" data-v9y=\"0\">just recently<\/a> been able to emulate a 1-millimeter-long flatworm brain, which consists of just 302 total neurons. The human brain contains 100 billion. If that makes it seem like a hopeless project, remember the power of exponential progress: Now that we\u2019ve conquered the tiny worm brain, an ant brain might happen before too long, followed by a mouse brain, and suddenly this will seem much more plausible.<\/p>\n<p><em>2) Try to make evolution do what it did before but for us this time:<\/em> So if we decide the smart kid\u2019s test is too hard to copy, we can try to copy the way he <em>studies<\/em> for the tests instead.<\/p>\n<p>Here\u2019s something we know: Building a computer as powerful as the brain <em>is <\/em>possible; our own brain\u2019s evolution is proof. And if the brain is just too complex for us to emulate, we could try to emulate <em>evolution <\/em>instead. The fact is that even if we can emulate a brain, that might be like trying to build an airplane by copying a bird\u2019s wing-flapping motions. Often machines are best designed using a fresh, machine-oriented approach, not by mimicking biology exactly.<\/p>\n<p>So how can we simulate evolution to build an AGI? The method, called \u201cgenetic algorithms,\u201d would work something like this: There would be a performance-and-evaluation process that would happen again and again (the same way biological creatures \u201cperform\u201d by living life and are \u201cevaluated\u201d by whether they manage to reproduce or not). A group of computers would try to do tasks, and the most successful ones would be <em>bred <\/em>with each other by having half of each of their programming merged together into a new computer. The less successful ones would be eliminated. Over many, many iterations, this process, mimicking natural selection, would produce better and better computers. The challenge would be creating an automated evaluation and breeding cycle so that this evolution process could run on its own.<\/p>\n<p>The downside of copying evolution is that evolution likes to take a billion years to do things, and we want to do this in a few decades.<\/p>\n<p>But we have a lot of advantages over evolution. First, evolution has no foresight and works randomly \u2014 it produces more unhelpful mutations than helpful ones \u2014 but we would control the process so that it would only be driven by beneficial glitches and targeted tweaks. Secondly, evolution doesn\u2019t <em>aim <\/em>for anything, including intelligence; sometimes an environment might even select <em>against <\/em>higher intelligence (since it uses a lot of energy). We, on the other hand, could specifically direct this evolutionary process toward increasing intelligence. Third, to select for intelligence, evolution has to innovate in a bunch of other ways to facilitate intelligence \u2014 like revamping the ways cells produce energy \u2014 when we can remove those extra burdens and use things like electricity. It\u2019s no doubt we\u2019d be much, much faster than evolution, but it\u2019s still not clear whether we\u2019ll be able to improve upon evolution <em>enough<\/em> to make this a viable strategy.<\/p>\n<p><em>3) Make this whole thing the computer\u2019s problem, not ours.<\/em> This is when scientists get desperate and try to program the test to take itself. But it might be the most promising method we have.<\/p>\n<p>The idea is that we\u2019d build a computer whose two major skills would be doing research on AI and coding changes into itself, allowing it to not only learn but improve its own <em>architecture<\/em>. We\u2019d teach computers to be computer scientists so that they could bootstrap their own development. And that would be their main job: figuring out how to make <em>themselves<\/em> smarter. More on this later.<\/p>\n<p><strong>All of This Could Happen Soon<\/strong><\/p>\n<p>Rapid advancements in hardware and innovative experimentation with software are happening simultaneously, and AGI could creep up on us quickly and unexpectedly for two main reasons:<\/p>\n<p>1) Exponential growth is intense, and what seems like a snail\u2019s pace of advancement can quickly race upwards. This GIF illustrates this concept nicely:<\/p>\n<p>2) When it comes to software, progress can seem slow, but then one epiphany can instantly change the rate of advancement (kind of like the way science, during the time when humans thought the universe was geocentric, was having difficulty calculating how the universe worked, but then the discovery that it was heliocentric suddenly made everything <em>much <\/em>easier). Or, when it comes to something like a computer that improves itself, we might seem far away but actually be just one tweak of the system away from having it become 1,000 times more effective and zooming upward to human-level intelligence.<\/p>\n<p><strong><em>The Road From AGI to ASI<\/em><\/strong><\/p>\n<p>At some point we\u2019ll have achieved AGI \u2014 computers with human-level general intelligence. Just a bunch of people and computers living together in equality.<\/p>\n<p>Oh actually not at all.<\/p>\n<p>The thing is that an AGI with an identical level of intelligence and computational capacity as a human would still have significant advantages over humans. Like:<\/p>\n<p><strong>Hardware<\/strong><\/p>\n<div data-rapid-cpos=\"119\" data-rapid-subsec=\"paragraph\" data-rapid-parsed=\"subsec\">\n<ul>\n<li><em>Speed:<\/em> The brain\u2019s neurons max out at around 200 hertz, while today\u2019s microprocessors (which are much slower than they will be when we reach AGI) run at 2 gigahertz, or 10 million times faster than our neurons. And the brain\u2019s internal communications, which can move at about 120 meters per second, are horribly outmatched by a computer\u2019s ability to communicate optically at the speed of light.<\/li>\n<\/ul>\n<\/div>\n<ul>\n<li><em>Size and storage:<\/em> The brain is locked into its size by the shape of our skulls, and it couldn\u2019t get much bigger anyway, or the 120-meters-per-second internal communications would take too long to get from one brain structure to another. Computers can expand to any physical size, allowing far more hardware to be put to work, a much larger working memory (RAM), and a long-term memory (hard-drive storage) that has both far greater capacity and far greater precision than our own.<\/li>\n<li><em>Reliability and durability:<\/em> It\u2019s not only the memories of a computer that would be more precise. Computer transistors are more accurate than biological neurons, and they\u2019re less likely to deteriorate (and can be repaired or replaced if they do). Human brains also get fatigued easily, while computers can run nonstop, at peak performance, 24\/7.<\/li>\n<\/ul>\n<p><strong>Software<\/strong><\/p>\n<div data-rapid-cpos=\"124\" data-rapid-subsec=\"paragraph\" data-rapid-parsed=\"subsec\">\n<ul>\n<li><em>Editability, upgradability, and a wider breadth of possibility:<\/em> Unlike the human brain, computer software can receive updates and fixes and can be easily experimented on. The upgrades could also span to areas where human brains are weak. Human vision software is superbly advanced, while its complex engineering capability is pretty low-grade. Computers could match the human on vision software but could <em>also <\/em>become equally optimized in engineering and any other area.<\/li>\n<\/ul>\n<\/div>\n<div data-rapid-cpos=\"125\" data-rapid-subsec=\"paragraph\" data-rapid-parsed=\"subsec\">\n<ul>\n<li><em>Collective capability:<\/em> Humans crush all other species at building a vast collective intelligence. Beginning with the development of language and the forming of large, dense communities, advancing through the inventions of writing and printing, and now intensified through tools like the Internet, humanity\u2019s collective intelligence is one of the major reasons we\u2019ve been able to get so far ahead of all other species. And computers <a href=\"http:\/\/books.google.com\/books\/about\/Superintelligence.html?id=7_H8AwAAQBAJ\" target=\"_hplink\" rel=\"nofollow\" data-ylk=\"subsec:paragraph;cpos:125;elm:context_link;itc:0\" data-rapid-parsed=\"slk\" data-rapid_p=\"38\" data-v9y=\"0\">will be way better at it than we are<\/a>. A worldwide network of AI running a particular program could regularly sync with itself so that anything any one computer learned would be instantly uploaded to all other computers. The group could also take on one goal as a unit, because there wouldn\u2019t necessarily be dissenting opinions and motivations and self-interest like we have within the human population.<\/li>\n<\/ul>\n<\/div>\n<p>AI, which will likely get to AGI by being programmed to self-improve, wouldn\u2019t see human-level intelligence as some important milestone \u2014 it\u2019s only a relevant marker from our point of view \u2014 and wouldn\u2019t have any reason to stop at our level. And given the advantages over us that even human-intelligence-equivalent AGI would have, it\u2019s pretty obvious that it would only hit human intelligence for a brief instant before racing onwards to the realm of superior-to-human intelligence.<\/p>\n<p>This may shock the shit out of us when it happens. The reason is that from <em>our <\/em>perspective, A) while the intelligence of different kinds of animals varies, the main characteristic we\u2019re aware of about any animal\u2019s intelligence is that it\u2019s far lower than ours, and B) we view the smartest humans as <em>way<\/em> smarter than the dumbest humans. Kind of like this:<\/p>\n<p>So as AI zooms upward in intelligence toward us, we\u2019ll see it as simply becoming smarter, <em>for an animal<\/em>. Then, when it hits the lowest capacity of humanity \u2014 Nick Bostrom uses the term \u201cthe village idiot\u201d \u2014 we\u2019ll be like, \u201cOh, wow, it\u2019s like a dumb human. Cute!\u201d The only thing is that in the grand spectrum of intelligence, <em>all <\/em>humans, from the village idiot to Einstein, are within a very small range, so <em>just <\/em>after hitting village-idiot level and being declared an AGI, it\u2019ll suddenly be smarter than Einstein and we won\u2019t know what hit us:<\/p>\n<p>And what happens after that?<\/p>\n<p><strong>An Intelligence Explosion<\/strong><\/p>\n<p>I hope you enjoyed normal time, because this is when this topic gets abnormal and scary, and it\u2019s going to stay that way from here forward. I want to pause here to remind you that every single thing I\u2019m going to say is real \u2014 real science and real forecasts of the future from a large array of the most respected thinkers and scientists. Just keep remembering that.<\/p>\n<p>Anyway, as I said above, most of our current models for getting to AGI involve the AI getting there by self-improvement. And once it gets to AGI, even systems that formed and grew through methods that didn\u2019t involve self-improvement would now be smart enough to begin self-improving if they wanted to.<sup>2<\/sup><\/p>\n<p>And here\u2019s where we get to an intense concept: recursive self-improvement. It works like this: An AI system at a certain level \u2014 say, village idiot \u2014 is programmed with the goal of improving its own intelligence. Once it does, it\u2019s <em>smarter<\/em> \u2014 maybe at this point it\u2019s at Einstein\u2019s level \u2014 so now, with an Einstein-level intellect, when it works to improve its intelligence, it has an easier time and it can make bigger leaps. These leaps make it <em>much <\/em>smarter than any human, allowing it to make even <em>bigger<\/em> leaps. As the leaps grow larger and happen more rapidly, the AGI soars upwards in intelligence and soon reaches the superintelligent level of an ASI system. This is called an intelligence explosion, and it\u2019s the ultimate example of the law of accelerating returns.<\/p>\n<p>There is some debate about how soon AI will reach human-level general intelligence; the median year on a survey of hundreds of scientists about when they believed we\u2019d be more likely than not to have reached AGI was <a href=\"http:\/\/books.google.com\/books\/about\/Superintelligence.html?id=7_H8AwAAQBAJ\" target=\"_hplink\" rel=\"nofollow\" data-ylk=\"subsec:paragraph;cpos:136;elm:context_link;itc:0\" data-rapid-parsed=\"slk\" data-rapid_p=\"41\" data-v9y=\"0\">2040<\/a>. That\u2019s only 25 years from now, which doesn\u2019t sound that huge until you consider that many of the thinkers in this field think it\u2019s likely that the progression from AGI to ASI happens <em>very <\/em>quickly. Like, this could happen:<\/p>\n<p>It takes decades for the first AI system to reach low-level general intelligence, but it finally happens. A computer is able to understand the world around it as well as a human 4-year-old. Suddenly, within an hour of hitting that milestone, the system pumps out the grand theory of physics that unifies general relativity and quantum mechanics, something no human has been able to definitively do. Ninety minutes after that, the AI has become an ASI, 170,000 times more intelligent than a human.<\/p>\n<p>Superintelligence of that magnitude is not something we can remotely grasp, any more than a bumblebee can wrap its head around Keynesian economics. In our world, \u201csmart\u201d means a 130 IQ, and \u201cstupid\u201d means an 85 IQ. We don\u2019t have a word for an IQ of 12,952.<\/p>\n<p>What we do know is that humans\u2019 utter dominance on this Earth suggests a clear rule: With intelligence comes power. This means an ASI, when we create it, will be the most powerful being in the history of life on Earth, and all living things, including humans, will be entirely at its whim \u2014 and this might happen in the next few decades.<\/p>\n<p>If our meager brains were able to invent wi-fi, then something 100 or 1,000 or 1 billion times smarter than we are should have no problem controlling the positioning of each and every atom in the world in any way it likes, at any time. Everything we consider magic, every power we imagine a supreme God to have, will be as mundane an activity for the ASI as flipping on a light switch is for us. Creating the technology to reverse human aging, curing disease and hunger and even mortality, reprogramming the weather to protect the future of life on Earth \u2014 all suddenly possible. Also possible is the immediate end of all life on Earth. As far as we\u2019re concerned, if an ASI comes into being, there is now an omnipotent god on Earth \u2014 and the all-important question for us is:<\/p>\n<p>Will it be a nice god?<\/p>\n<p>That\u2019s the topic of <a href=\"http:\/\/waitbutwhy.com\/2015\/01\/artificial-intelligence-revolution-2.html\" target=\"_blank\" rel=\"nofollow noopener\" data-ylk=\"subsec:paragraph;cpos:142;elm:context_link;itc:0\" data-rapid-parsed=\"slk\" data-rapid_p=\"42\" data-v9y=\"0\">Part 2<\/a>.<\/p>\n<p><em><strong>Footnotes:<\/strong><\/em><\/p>\n<div data-rapid-cpos=\"144\" data-rapid-subsec=\"paragraph\" data-rapid-parsed=\"subsec\">\n<ol>\n<li>Kurzweil points out that his phone is about a millionth the size of, a millionth the price of, and a thousand times more powerful than his MIT computer was 40 years ago. Good luck trying to figure out where a comparable future advancement in computing would leave us, let alone one far, far more extreme, since the progress grows exponentially.<\/li>\n<\/ol>\n<\/div>\n<ul>\n<li>There\u2019s much more on what it means for a computer to \u201cwant\u201d to do something in Part 2.<\/li>\n<\/ul>\n<div data-rapid-cpos=\"150\" data-rapid-subsec=\"paragraph\" data-rapid-parsed=\"subsec\"><\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<div><\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>huffingtonpost.com The AI Revolution: The Road to Superintelligence 41-52 minutes Artificial superintelligence is coming, probably whether we like it or not, and probably within our lifetimes. If many of the experts are correct, this will either be our greatest dream or our worst nightmare. Here\u2019s why. \u201cWe are on the edge of change comparable to &hellip; <a href=\"https:\/\/mikecosgrave.com\/annotation\/ai\/\" class=\"more-link\">Continue reading <span class=\"screen-reader-text\">AI<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-12","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/mikecosgrave.com\/annotation\/wp-json\/wp\/v2\/pages\/12","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mikecosgrave.com\/annotation\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mikecosgrave.com\/annotation\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mikecosgrave.com\/annotation\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/mikecosgrave.com\/annotation\/wp-json\/wp\/v2\/comments?post=12"}],"version-history":[{"count":2,"href":"https:\/\/mikecosgrave.com\/annotation\/wp-json\/wp\/v2\/pages\/12\/revisions"}],"predecessor-version":[{"id":34,"href":"https:\/\/mikecosgrave.com\/annotation\/wp-json\/wp\/v2\/pages\/12\/revisions\/34"}],"wp:attachment":[{"href":"https:\/\/mikecosgrave.com\/annotation\/wp-json\/wp\/v2\/media?parent=12"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}