[Hplusroadmap] Fwd: Re: Transhumans - "We're fruity and delicious"

[Bryan Bishop]

----------  Forwarded Message  ----------

Subject: Re: We're fruity and delicious
Date: Wednesday 14 May 2008
From: Bryan Bishop <kanzure at gmail.com>
To: "Michael Anissimov" <michaelanissimov at gmail.com>

On Wednesday 14 May 2008, Michael Anissimov wrote:
> A poorly programmed, recursively self-improving AI could easily
> swallow the entire solar system in an effort to maximize some utility
> function.

So could a person with the right tech too.

> Self improving AI  --> molecular nanotechnology --> capability to
> disassemble every atom on the planet and rearrange it to maximize a
> utility function.

I don't know why only the ai gets to know those secrets.

> This danger could follow you into space quite easily.

revenge? I mean, re: Vinge?

> Read http://www.singinst.org/upload/LOGI//seedAI.html for why an AI
> could go from zero to hero very quickly.

Although I am far from the last one to be eager about recursive 
self-improving processes**, I have to point out that the ai has to have 
a substrate on which to run, i.e. a physical microprocessor, and as it 
learns and grows and becomes smarter (assume it is as intelligent as 
possible on the current architecture), it must create a new 
architecture of sorts so that it may continue that progression. I think 
you agree up to this point. But here's the kicker: any ai will run 
straight into the same problems we have with these situations. The 
simple fact that it needs to learn and read and be grounded, to obtain 
the resources and machinery to *make* the new computational 
architecture for itself. It doesn't just intuit this from the void. And 
running explosive-recursive ai on hardware that costs you money is just 
stupid, unless you're scheming to make money off of every single 
thought or, uh, good idea / byproduct generated by the ai. *That's* 
what you have to be worried about -- money being crept into all of 
this. Otherwise your problems are basically solved by forking.*

Re: the ai will have the same problems as us; it too will have to learn 
how to do manufacturing, how to interface with the world just as we are 
learning. Essentially you'll (not just *it*) will come to:
http://heybryan.org/exp.html
http://heybryan.org/2008-05-09.html for an email-essay.

Because any truly exponential process will need to be grounded in 
something that is self-replicating. Unless you want upper boundaries 
with weird limits like 'financial feasability' or "well, turns out 
these oil wells [for energy] just aren't too deep". Uh, not that I'm 
suggesting ever running ai on oil energy. Bad example. :-)

> I don't get what "fork" means in this context.

Fork in the road, that sort of thing. In programming projects across the 
web, you can download source code and fork it, basically making your 
own branch. I am using this same development model for physical 
manufacturing systems and clanking self-replication (in exp.html).

For a glimpse at what open source provides us with:
http://en.wikipedia.org/wiki/Debian
http://debian.org/
http://ubuntu.org/ And more concisely:

 "Debian is known for strict adherence to the Unix and free software 
philosophies. Debian is also known for its abundance of options — the 
current release includes over twenty-six thousand software packages for 
eleven computer architectures. These architectures range from the 
Intel/AMD 32-bit/64-bit architectures commonly found in personal 
computers to the ARM architecture commonly found in embedded systems 
and the IBM eServer zSeries mainframes. Throughout Debian's lifetime, 
other distributions have taken it as a basis to develop their own, 
including: Ubuntu, MEPIS, Dreamlinux, Damn Small Linux, Xandros, 
Knoppix, Linspire, sidux, Kanotix, and LinEx among others. A 
university's study concluded that Debian's 283 million source code 
lines would cost 10 billion USA Dollars to develop by proprietary 
means."

"Ubuntu's popularity has climbed steadily since its 2004 release. It has 
been the most viewed Linux distribution on Distrowatch.com in 2005,[4] 
2006,[5] In an August 2007 survey of 38,500 visitors on 
DesktopLinux.com, Ubuntu was the most popular distribution with 30.3 
percent of respondents using it.[7] Third party sites have arisen to 
provide Ubuntu packages outside of the Ubuntu organization. Ubuntu was 
awarded the Reader Award for best Linux distribution at the 2005 
LinuxWorld Conference and Expo in London.[107] It has been favorably 
reviewed in online and print publications.[108][109][110] Ubuntu won 
InfoWorld's 2007 Bossie Award for Best Open Source Client OS.[111] Mark 
Shuttleworth indicates that there were at least 8 million Ubuntu users 
at the end of 2006.[112] The large user-base has resulted in a large 
stable of non-Canonical websites. These include general help sites like 
Easy Ubuntu Linux,[113] dedicated weblogs (Ubuntu Gazette),[114] and 
niche sites within the Ubuntu Linux niche itself (Ubuntu Women).[115] 
The year 2007 saw the online publication of the first magazine 
dedicated to Ubuntu, Full Circle.[116]"

In this context, forking would simply mean forking with a 
self-replicating machine, such as your O'Neill habitats or virgling 
system. Yes, ai could catch up to you in the end, good cop bad cop 
except good ai bad ai.

Or really it's "good exponential intelligence, bad exponential 
intelligence". Not so much ai. It's anything that's capable of that 
sort of growth, at that pace, with that sort of interface and grounding 
in reality. Again **.

> Society goes to hell when I give myself a technology that lets me
> kill hundreds of people undetected (microrobotics, for instance),
> then millions of other people get it, then they all use it.  All it
> takes is 1/1000 people to be murderous for this to be a problem.

So you probably hate me for http://biohack.sf.net/ right?

- Bryan

** http://heybryan.org/recursion.html
________________________________________
http://heybryan.org/

-------------------------------------------------------
________________________________________
http://heybryan.org/