[Hplusroadmap] [SL4] Re: Paper: Artificial Intelligence will Kill our Grandchildren

[Dan Bolser]

2008/6/25 Bryan Bishop <kanzure at gmail.com>:
> On Friday 13 June 2008, Anthony Berglas wrote:
>> http://berglas.org/Articles/AIKillGrandchildren/AIKillGrandchildren.h
>>tml
>
> Let's do this.
>
>> There have been many exaggerated claims as to the power of Artificial
>> Intelligence (AI), but there has also been real progress.  Computers
>
> This entirely depends on what progress you are expecting ai to come in
> the form of. This might be due to the fact that many people are taking
> peculiar routes to getting programmable intelligence, taking other
> loopholes and whatever, assuming definitions of intelligence, and so
> on. Difficult issues there, but otherwise if you look more closely
> there might be some real progress in other areas. RSI, maybe.
>
>> can drive cars across rough desert tracks, understand speech, and
>> prove complex mathematical theorems.  It is difficult to predict
>> future progress, but if a computer ever became about as good at
>> programming computers as people are, then it could program a copy of
>
> No, it could just copy bits and bytes, nothing about programming is
> needed for copying from one machine to another.
>
>> itself.  This would lead to an exponential rise in intelligence (now
>
> No, this is not true. The ground problem inherently limits the available
> hardware on which the software can 'exponentially' expand on. Really
> the numbers are going to look like something that keeps hitting the top
> of available hardware capacity, unless they are physically grounded
> with manufacturing processes that allow it to exponentially make the
> machinery to make the machinery to ... <do what it does>.
>
>> often referred to as the Singularity).  And evolution suggests that a
>> sufficiently powerful AI would probably destroy humanity.  This paper
>
> Wha? How does 'evolution' suggest this? Evolution doesn't come to our
> door and actually say this to us, so I fail to see what you're trying
> to say here. Evolution isn't a good friend of ours (and I'm not saying
> it's a bad friend). Merely that you need some elaboration and
> clarification.

Evolution eats its grandparents.