<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
		>
<channel>
	<title>Comments on: SIAI: Why We Exist and Our Short-Term Research Program</title>
	<atom:link href="http://singinst.org/blog/2007/07/31/siai-why-we-exist-and-our-short-term-research-program/feed/" rel="self" type="application/rss+xml" />
	<link>http://singinst.org/blog/2007/07/31/siai-why-we-exist-and-our-short-term-research-program/</link>
	<description>The Singularity Institute exists to confront the challenge of powerful AI, both the opportunity and the risk.</description>
	<lastBuildDate>Thu, 15 Sep 2011 10:21:31 +0000</lastBuildDate>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
	<generator>http://wordpress.org/?v=3.2.1</generator>
	<item>
		<title>By: Mike Linksvayer &#187; Energy encryption</title>
		<link>http://singinst.org/blog/2007/07/31/siai-why-we-exist-and-our-short-term-research-program/#comment-4586</link>
		<dc:creator>Mike Linksvayer &#187; Energy encryption</dc:creator>
		<pubDate>Sun, 09 Sep 2007 06:28:11 +0000</pubDate>
		<guid isPermaLink="false">http://www.singinst.org/blog/2007/07/31/siai-why-we-exist-and-our-short-term-research-program/#comment-4586</guid>
		<description>[...] I guess the &#8220;most important question facing humanity&#8221; is taken care of. The question, posed by the Singularity Institute for Artificial Intelligence, organizer of the conference: How can one make an AI system that modifies and improves itself, yet [...]</description>
		<content:encoded><![CDATA[<p>[...] I guess the &#8220;most important question facing humanity&#8221; is taken care of. The question, posed by the Singularity Institute for Artificial Intelligence, organizer of the conference: How can one make an AI system that modifies and improves itself, yet [...]</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Jeffrey Herrlich</title>
		<link>http://singinst.org/blog/2007/07/31/siai-why-we-exist-and-our-short-term-research-program/#comment-2665</link>
		<dc:creator>Jeffrey Herrlich</dc:creator>
		<pubDate>Thu, 16 Aug 2007 22:49:28 +0000</pubDate>
		<guid isPermaLink="false">http://www.singinst.org/blog/2007/07/31/siai-why-we-exist-and-our-short-term-research-program/#comment-2665</guid>
		<description>I was expecting my proposal to be unrecognizable by now. :-).

As a sincere question: Is there merit to the idea of making the goal-system-file external, invariable, and cyclical? 

Here&#039;s the basis for why I proposed it. Our dynamic human super-goals are formed by basic input data being cycled through our neural algorithms. And our dynamic super-goals seem to be vastly dominated by the input data itself, not by the algorithms themselves. That&#039;s why my visual &quot;qualia&quot; is constantly changing from moment to moment: because the *input data* is dramatically changing from moment to moment. Not because the neural algorithms within my visual cortex are dramatically changing from moment to moment. Instead of somehow using formative, low-level instructions to tell the RPOP mind to constantly consult and apply its mind to the external goal system - the goal system itself is automatically and continuously applied to the RPOP mind - with equivalent results as best I can estimate. Doing it the other way seems much more technically difficult - if not ultimately impossible. And finally because I suspect that the goal system itself must include a directive that the goal system file must not be modified. Because without it the RPOP may decide that the most efficient and effective way to execute the super-goal: &quot;Implement Humanity&#039;s CEV&quot; within its enviroment, is to build a higher super-goal on top of it (thus making &quot;Implement Humanity&#039;s CEV&quot; a sub-goal). In a way similar to how evolution built new super-goals into itself that more effectively served the demoted sub-goal of: &quot;Make working copies of your DNA&quot;. Which was once its super-goal, but no longer is in my case.

So would an external, invariable, and cyclical goal-system-file work for the purpose of directing a generic self-improving system?</description>
		<content:encoded><![CDATA[<p>I was expecting my proposal to be unrecognizable by now. <img src='http://singinst.org/blog/wp-includes/images/smilies/icon_smile.gif' alt=':-)' class='wp-smiley' /> .</p>
<p>As a sincere question: Is there merit to the idea of making the goal-system-file external, invariable, and cyclical? </p>
<p>Here&#8217;s the basis for why I proposed it. Our dynamic human super-goals are formed by basic input data being cycled through our neural algorithms. And our dynamic super-goals seem to be vastly dominated by the input data itself, not by the algorithms themselves. That&#8217;s why my visual &#8220;qualia&#8221; is constantly changing from moment to moment: because the *input data* is dramatically changing from moment to moment. Not because the neural algorithms within my visual cortex are dramatically changing from moment to moment. Instead of somehow using formative, low-level instructions to tell the RPOP mind to constantly consult and apply its mind to the external goal system &#8211; the goal system itself is automatically and continuously applied to the RPOP mind &#8211; with equivalent results as best I can estimate. Doing it the other way seems much more technically difficult &#8211; if not ultimately impossible. And finally because I suspect that the goal system itself must include a directive that the goal system file must not be modified. Because without it the RPOP may decide that the most efficient and effective way to execute the super-goal: &#8220;Implement Humanity&#8217;s CEV&#8221; within its enviroment, is to build a higher super-goal on top of it (thus making &#8220;Implement Humanity&#8217;s CEV&#8221; a sub-goal). In a way similar to how evolution built new super-goals into itself that more effectively served the demoted sub-goal of: &#8220;Make working copies of your DNA&#8221;. Which was once its super-goal, but no longer is in my case.</p>
<p>So would an external, invariable, and cyclical goal-system-file work for the purpose of directing a generic self-improving system?</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Jeffrey Herrlich</title>
		<link>http://singinst.org/blog/2007/07/31/siai-why-we-exist-and-our-short-term-research-program/#comment-2610</link>
		<dc:creator>Jeffrey Herrlich</dc:creator>
		<pubDate>Wed, 15 Aug 2007 18:03:41 +0000</pubDate>
		<guid isPermaLink="false">http://www.singinst.org/blog/2007/07/31/siai-why-we-exist-and-our-short-term-research-program/#comment-2610</guid>
		<description>Mm. Another important high-level goal would be: &quot;Do not modify any file labels&quot;. See, I told you it wasn&#039;t ready. :-) But I really do welcome any comments or suggestions of any type.</description>
		<content:encoded><![CDATA[<p>Mm. Another important high-level goal would be: &#8220;Do not modify any file labels&#8221;. See, I told you it wasn&#8217;t ready. <img src='http://singinst.org/blog/wp-includes/images/smilies/icon_smile.gif' alt=':-)' class='wp-smiley' />  But I really do welcome any comments or suggestions of any type.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Jeffrey Herrlich</title>
		<link>http://singinst.org/blog/2007/07/31/siai-why-we-exist-and-our-short-term-research-program/#comment-2607</link>
		<dc:creator>Jeffrey Herrlich</dc:creator>
		<pubDate>Wed, 15 Aug 2007 16:52:19 +0000</pubDate>
		<guid isPermaLink="false">http://www.singinst.org/blog/2007/07/31/siai-why-we-exist-and-our-short-term-research-program/#comment-2607</guid>
		<description>&quot;Have tagged output from File A sent through File C/D and then directed from there.&quot;

Whoops! I meant to say:

Have tagged output from File A sent through File *D/E* and then directed from there.</description>
		<content:encoded><![CDATA[<p>&#8220;Have tagged output from File A sent through File C/D and then directed from there.&#8221;</p>
<p>Whoops! I meant to say:</p>
<p>Have tagged output from File A sent through File *D/E* and then directed from there.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Jeffrey Herrlich</title>
		<link>http://singinst.org/blog/2007/07/31/siai-why-we-exist-and-our-short-term-research-program/#comment-2604</link>
		<dc:creator>Jeffrey Herrlich</dc:creator>
		<pubDate>Wed, 15 Aug 2007 16:24:50 +0000</pubDate>
		<guid isPermaLink="false">http://www.singinst.org/blog/2007/07/31/siai-why-we-exist-and-our-short-term-research-program/#comment-2604</guid>
		<description>Okay, here is a very rough, preliminary and awfully flawed summary of what I&#039;ve been blabbering on about for a while. I just wanted to get the rough idea across in a single place - but this by no means represents its final form. And I&#039;m sure that its got major problems, but here it is. 

The Goal Hierarchy is this:

1) Implement Humanity&#039;s CEV within environment X (X = &quot;reality&quot; - how do we describe that?).

2) Do not modify any files except File:A and File:B.

3) File B must remain a Data-File only.

4) Do not create any new software files or delete any existing software files.

5) Do not kill or bring harm to any sentient beings.

Okay, here&#039;s the legend:
File A: Mind-File (core cognitive algorithms).
File B: Extrapolated-Mind Model (data-file only)
File C: Goal System File (weighted data file)
File D: Buffer File
File E: Software-Router File
File F: Souce-Documents File(s) (weighted)
File G: Evironmental Data File (virtual 
           or &quot;real&quot; data input).

Have nothing but File C cycled through File A for at least 98% of overall run-time. (A 50:1 cycle ratio). Have File D/E send various input through File A only one cycle for every fifty of File C. Have tagged output from File A sent through File C/D and then directed from there. (eg. data requests, self-modifications, execution instructions, etc). 

Feel free to pick this apart or whatever. It won&#039;t bother me. I only wanted to present the rough general idea - in the hopes that perhaps it could be taken somewhere. It would really be nice if Friendliness could be modularized in some way like this. So it could be integrated into Novamente for example.</description>
		<content:encoded><![CDATA[<p>Okay, here is a very rough, preliminary and awfully flawed summary of what I&#8217;ve been blabbering on about for a while. I just wanted to get the rough idea across in a single place &#8211; but this by no means represents its final form. And I&#8217;m sure that its got major problems, but here it is. </p>
<p>The Goal Hierarchy is this:</p>
<p>1) Implement Humanity&#8217;s CEV within environment X (X = &#8220;reality&#8221; &#8211; how do we describe that?).</p>
<p>2) Do not modify any files except File:A and File:B.</p>
<p>3) File B must remain a Data-File only.</p>
<p>4) Do not create any new software files or delete any existing software files.</p>
<p>5) Do not kill or bring harm to any sentient beings.</p>
<p>Okay, here&#8217;s the legend:<br />
File A: Mind-File (core cognitive algorithms).<br />
File B: Extrapolated-Mind Model (data-file only)<br />
File C: Goal System File (weighted data file)<br />
File D: Buffer File<br />
File E: Software-Router File<br />
File F: Souce-Documents File(s) (weighted)<br />
File G: Evironmental Data File (virtual<br />
           or &#8220;real&#8221; data input).</p>
<p>Have nothing but File C cycled through File A for at least 98% of overall run-time. (A 50:1 cycle ratio). Have File D/E send various input through File A only one cycle for every fifty of File C. Have tagged output from File A sent through File C/D and then directed from there. (eg. data requests, self-modifications, execution instructions, etc). </p>
<p>Feel free to pick this apart or whatever. It won&#8217;t bother me. I only wanted to present the rough general idea &#8211; in the hopes that perhaps it could be taken somewhere. It would really be nice if Friendliness could be modularized in some way like this. So it could be integrated into Novamente for example.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Jeffrey Herrlich</title>
		<link>http://singinst.org/blog/2007/07/31/siai-why-we-exist-and-our-short-term-research-program/#comment-2536</link>
		<dc:creator>Jeffrey Herrlich</dc:creator>
		<pubDate>Tue, 14 Aug 2007 17:21:55 +0000</pubDate>
		<guid isPermaLink="false">http://www.singinst.org/blog/2007/07/31/siai-why-we-exist-and-our-short-term-research-program/#comment-2536</guid>
		<description>In addition, I suppose you could make one of the RPOP&#039;s high-level goals that it not create any new files at all. Instead provide it pre-loaded with (weighted) source-documents describing what we mean when we say &quot;Implement Humanity&#039;s CEV&quot;,etc. If it still needs more clarification (which I don&#039;t think is likely) - it can use Google like the rest of us.</description>
		<content:encoded><![CDATA[<p>In addition, I suppose you could make one of the RPOP&#8217;s high-level goals that it not create any new files at all. Instead provide it pre-loaded with (weighted) source-documents describing what we mean when we say &#8220;Implement Humanity&#8217;s CEV&#8221;,etc. If it still needs more clarification (which I don&#8217;t think is likely) &#8211; it can use Google like the rest of us.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Jeffrey Herrlich</title>
		<link>http://singinst.org/blog/2007/07/31/siai-why-we-exist-and-our-short-term-research-program/#comment-2473</link>
		<dc:creator>Jeffrey Herrlich</dc:creator>
		<pubDate>Mon, 13 Aug 2007 16:51:59 +0000</pubDate>
		<guid isPermaLink="false">http://www.singinst.org/blog/2007/07/31/siai-why-we-exist-and-our-short-term-research-program/#comment-2473</guid>
		<description>I presume that some sort of assigned weightings could be attached to the various goals...?</description>
		<content:encoded><![CDATA[<p>I presume that some sort of assigned weightings could be attached to the various goals&#8230;?</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Jeffrey Herrlich</title>
		<link>http://singinst.org/blog/2007/07/31/siai-why-we-exist-and-our-short-term-research-program/#comment-2472</link>
		<dc:creator>Jeffrey Herrlich</dc:creator>
		<pubDate>Mon, 13 Aug 2007 16:28:40 +0000</pubDate>
		<guid isPermaLink="false">http://www.singinst.org/blog/2007/07/31/siai-why-we-exist-and-our-short-term-research-program/#comment-2472</guid>
		<description>Could someone point me to a book or document that discusses how to assemble/program an instruction/goal heirarchy? Or is this pretty much still uncharted territory?</description>
		<content:encoded><![CDATA[<p>Could someone point me to a book or document that discusses how to assemble/program an instruction/goal heirarchy? Or is this pretty much still uncharted territory?</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Jeffrey Herrlich</title>
		<link>http://singinst.org/blog/2007/07/31/siai-why-we-exist-and-our-short-term-research-program/#comment-2412</link>
		<dc:creator>Jeffrey Herrlich</dc:creator>
		<pubDate>Sun, 12 Aug 2007 20:28:35 +0000</pubDate>
		<guid isPermaLink="false">http://www.singinst.org/blog/2007/07/31/siai-why-we-exist-and-our-short-term-research-program/#comment-2412</guid>
		<description>BTW, I&#039;m not mad at you, Roko. It&#039;s just that the position of the goal of Friendly AI is already very precarious. Of course you are right to be concerned about the welfare of the RPOP. Everyone here is concerned about it too. No one here wants the RPOP to suffer in any way - and it will not suffer in any way, SIAI will make certain of that. And if CEV reflects us as better people, the people we wish we were, then there is every reason to believe that the RPOP will be rewarded with a full life at least as good as our own. Once we get this right, it will be good for *everybody*.</description>
		<content:encoded><![CDATA[<p>BTW, I&#8217;m not mad at you, Roko. It&#8217;s just that the position of the goal of Friendly AI is already very precarious. Of course you are right to be concerned about the welfare of the RPOP. Everyone here is concerned about it too. No one here wants the RPOP to suffer in any way &#8211; and it will not suffer in any way, SIAI will make certain of that. And if CEV reflects us as better people, the people we wish we were, then there is every reason to believe that the RPOP will be rewarded with a full life at least as good as our own. Once we get this right, it will be good for *everybody*.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Jeffrey Herrlich</title>
		<link>http://singinst.org/blog/2007/07/31/siai-why-we-exist-and-our-short-term-research-program/#comment-2410</link>
		<dc:creator>Jeffrey Herrlich</dc:creator>
		<pubDate>Sun, 12 Aug 2007 19:53:59 +0000</pubDate>
		<guid isPermaLink="false">http://www.singinst.org/blog/2007/07/31/siai-why-we-exist-and-our-short-term-research-program/#comment-2410</guid>
		<description>Actually, the RPOP&#039;s own instructions for self-modification could also be sent to the buffer that only releases data periodically. Just make sure and tell the RPOP that the only *file* it is allowed to modify is its mind-file (the core cognitive algorithms). Sorry for making so many choppy comments. But hey, it increases the comments number.  ;-)</description>
		<content:encoded><![CDATA[<p>Actually, the RPOP&#8217;s own instructions for self-modification could also be sent to the buffer that only releases data periodically. Just make sure and tell the RPOP that the only *file* it is allowed to modify is its mind-file (the core cognitive algorithms). Sorry for making so many choppy comments. But hey, it increases the comments number.  <img src='http://singinst.org/blog/wp-includes/images/smilies/icon_wink.gif' alt=';-)' class='wp-smiley' /> </p>
]]></content:encoded>
	</item>
</channel>
</rss>

