Singularity Institute

Writing a Comprehensive Singularity FAQ

Key Aim: Write a comprehensive or nearly comprehensive Singularity Frequently Asked Questions (FAQ), describing both SIAI's positions and the positions of many others on crucial Singularity-related issues. Goal is 100+ pages and answering more than 30 questions.

The vision behind the project:

• Long-time SIAI supporters such as Joshua Fox have pointed out that there is no complete, concise explanation for many of the principles behind SIAI's activism.

• Rather, the reasoning behind many of our positions are strewn across a variety of places: Less Wrong, Yudkowsky's papers, the SL4 mailing list archives, papers by Nick Bostrom, and articles on Accelerating Future.

• One particularly noteworthy example would be arguments in favor of a hard takeoff- these appear in very few places, and even key documents such as Levels of Organization in General Intelligence only give the topic 10 or so pages worth of attention. This is disappointing because the evidence in favor of a hard takeoff scenario could fill hundreds of pages.

• Academic papers generally only appeal to academics. Though academic papers on Singularity-related topics are extremely important, popular works are also very important because they have the potential to viscerally reach a much wider audience, which may include academics that never even considered whether they should regard the Singularity as important.

Steps to be taken in order to realize this vision.

Step 1: Create an overview of the questions to be answered. Integrate actual questions asked across different blogs, forums, books, videos, etc.
Deliverable: List of questions.
Target date: January 15.

Step 2: Begin answering the questions one by one, with high-quality answers including multiple paragraphs, the introduction of relevant concepts, hyperlinks, and citations. Not a throwaway FAQ meant to answer questions as quickly as possible, but a reading experience designed to leave the reader much better informed about the Singularity and relevant issues than when they began.
Deliverable: First draft of FAQ.
Target date: March 15.

Step 3: Open the FAQ up to SIAI's Visiting Fellows, staff, and interns for feedback and discussion. It is anticipated that many updates and revisions will be made based on said feedback.
Deliverable: Final draft of FAQ.
Target date: April 15.

Step 4: Publicizing the FAQ. Post on allied blogs and websites, promote on forums and mailing lists. There is significant demand for such a FAQ, evidenced by the many discussions on the Singularity around the Internet every day.
Deliverable: Various.
Target date for initial wave: April 15.

Total budget: $5,000

We estimate that completing this project will require 3.5 months of an SIAI staffer (specifically Media Director Michael Anissimov) working half-time. The estimated cost per month of SIAI staff time is $2,400, so the subtotal comes to $4,200. Assorted time spent by Visiting Fellows on providing feedback is estimated to cost approximately $800, for a total of $5,000.

How this project, if carried out, will reduce existential risk:

• A great number of people are casually interested in the Singularity but not interested in putting in the time necessary to acquire the detailed ideas behind what the Singularity Institute does, which generally requires 6-12 months of in-depth study.

• This "ambient ignorance" of Singularity-related issues lowers the likelihood that any arbitrary researcher will publish on Singularity issues, because of the "silly factor" surrounding the issue. A Singularity FAQ that shows that many basic Singularity ideas have scientific, non-speculative foundations could go far to dispel this silliness.

• A well-promoted web page on Singularity issues can accrue tens or even hundreds of thousands of unique visitors in a span of a few months if promoted right. For instance, "The Benefits of a Successful Singularity" piece at Good.is was viewed by about 80,000 people over a few days due to reaching the front page of Digg.com. Companies routinely pay consultants many tens of thousands of dollars to secure similar outcomes, but SIAI has skilled writers and promoters willing to work for significantly less because of the importance of the cause. By exposing many more people to the fundamental ideas, we can increase the probability of subsequent support for Friendly AI research, which mitigates existential risk.

How benefits will be assessed:

A brief assessment of the grants' effects will be written up afterward, discussing whether similar grants would be a good use of future funds.

The assessment will include:

• The actual time costs that went into the FAQ.

• How many views the resulting FAQ receives.

• The number of incoming links the FAQ receives.

• To what extent informally surveyed newcomers in the target audience report finding the FAQ useful.

• To what extent informally surveyed bloggers and other writers report that the FAQ helps them build on past analysis.