Part 36 (2/2)

12. 12. Ibid. Ibid.

13. 13. Norbert Wiener, Norbert Wiener, Cybernetics: or, Control and Communication in the Animal and the Machine Cybernetics: or, Control and Communication in the Animal and the Machine (Cambridge, Ma.s.s.: MIT Press, 1948). (Cambridge, Ma.s.s.: MIT Press, 1948).

14. 14. ”How Do You Persist When Your Molecules Don't?” ”How Do You Persist When Your Molecules Don't?” Science and Consciousness Review Science and Consciousness Review 1.1 (June 2004), /sacredhuston.html.

17. 17. Jerry A. Fodor, Jerry A. Fodor, RePresentations: Philosophical Essays on the Foundations of Cognitive Science RePresentations: Philosophical Essays on the Foundations of Cognitive Science (Cambridge, Ma.s.s.: MIT Press, 1981). (Cambridge, Ma.s.s.: MIT Press, 1981).

Chapter Eight: The Deeply Intertwined Promise and Peril of GNR 1. 1. Bill McKibben, ”How Much Is Enough? The Environmental Movement as a Pivot Point in Human History,” Harvard Seminar on Environmental Values, October 18, 2000. Bill McKibben, ”How Much Is Enough? The Environmental Movement as a Pivot Point in Human History,” Harvard Seminar on Environmental Values, October 18, 2000.

2. 2. In the 1960s, the U.S. government conducted an experiment in which it asked three recently graduated physics students to build a nuclear weapon using only publicly available information. The result was successful; the three students built one in about three years (/nais/nl/n.nukes.html). Plans for how to build an atomic bomb are available on the Internet and have been published in book form by a national laboratory. In 2002, the British Ministry of Defence released measurements, diagrams, and precise details on bomb building to the Public Record Office, since removed (news.bbc.co.uk/1/hi/uk/1932702.stm). Note that these links do not contain actual plans to build atomic weapons. In the 1960s, the U.S. government conducted an experiment in which it asked three recently graduated physics students to build a nuclear weapon using only publicly available information. The result was successful; the three students built one in about three years (/nais/nl/n.nukes.html). Plans for how to build an atomic bomb are available on the Internet and have been published in book form by a national laboratory. In 2002, the British Ministry of Defence released measurements, diagrams, and precise details on bomb building to the Public Record Office, since removed (news.bbc.co.uk/1/hi/uk/1932702.stm). Note that these links do not contain actual plans to build atomic weapons.

3. 3. ”The John Stossel Special: You Can't Say That!” ABC News, March 23, 2000. ”The John Stossel Special: You Can't Say That!” ABC News, March 23, 2000.

4. 4. There is extensive information on the Web, including military manuals, on how to build bombs, weapons, and explosives. Some of this information is erroneous, but accurate information on these topics continues to be accessible despite efforts to remove it. Congress pa.s.sed an amendment (the Feinstein Amendment, SP 419) to a Defense Department appropriations bill in June 1997, banning the dissemination of instructions on building bombs. See Anne Marie Helmenstine, ”How to Build a Bomb,” February 10, 2003, chemistry.about.com/library/weekly/aa021003a.htm. Information on toxic industrial chemicals is widely available on the Web and in libraries, as are information and tools for cultivating bacteria and viruses and techniques for creating computer viruses and hacking into computers and networks. Note that I do not provide specific examples of such information, since it might be helpful to destructive individuals and groups. I realize that even stating the availability of such information has this potential, but I feel that the benefit of open dialogue about this issue outweighs this concern. Moreover, the availability of this type of information has been widely discussed in the media and other venues. There is extensive information on the Web, including military manuals, on how to build bombs, weapons, and explosives. Some of this information is erroneous, but accurate information on these topics continues to be accessible despite efforts to remove it. Congress pa.s.sed an amendment (the Feinstein Amendment, SP 419) to a Defense Department appropriations bill in June 1997, banning the dissemination of instructions on building bombs. See Anne Marie Helmenstine, ”How to Build a Bomb,” February 10, 2003, chemistry.about.com/library/weekly/aa021003a.htm. Information on toxic industrial chemicals is widely available on the Web and in libraries, as are information and tools for cultivating bacteria and viruses and techniques for creating computer viruses and hacking into computers and networks. Note that I do not provide specific examples of such information, since it might be helpful to destructive individuals and groups. I realize that even stating the availability of such information has this potential, but I feel that the benefit of open dialogue about this issue outweighs this concern. Moreover, the availability of this type of information has been widely discussed in the media and other venues.

5. 5. Ray Kurzweil, Ray Kurzweil, The Age of Intelligent Machines The Age of Intelligent Machines (Cambridge, Ma.s.s.: MIT Press, 1990). (Cambridge, Ma.s.s.: MIT Press, 1990).

6. 6. Ken Alibek, Ken Alibek, Biohazard Biohazard (New York: Random House, 1999). (New York: Random House, 1999).

7. 7. Ray Kurzweil, Ray Kurzweil, The Age of Spiritual Machines The Age of Spiritual Machines (New York: Viking, 1999). (New York: Viking, 1999).

8. 8. Bill Joy, ”Why the Future Doesn't Need Us,” Bill Joy, ”Why the Future Doesn't Need Us,” Wired Wired, April 2000, /wired/archive/8.04/joy.html.

9. 9. Handbooks on gene splicing (such as A. J. Harwood, ed., Handbooks on gene splicing (such as A. J. Harwood, ed., Basic DNA and RNA Protocols Basic DNA and RNA Protocols [Totowa, N.J.: Humana Press, 1996]) along with reagents and kits that enable gene splicing are generally available. Even if access to these materials were limited in the West, there are a large number of Russian companies that could provide equivalent materials. [Totowa, N.J.: Humana Press, 1996]) along with reagents and kits that enable gene splicing are generally available. Even if access to these materials were limited in the West, there are a large number of Russian companies that could provide equivalent materials.

10. 10. For a detailed summary site of the ”Dark Winter” simulation, see ”DARK WINTER: A Bioterrorism Exercise June 2001”: .au/articles/2003/08/0311059849278131.html. See note 129 in chapter 5. See also S. C. Barton, J. Gallaway, and P. Atana.s.sov, ”Enzymatic Biofuel Cells for Implantable and Microscale Devices,” Chemical Reviews Chemical Reviews 104.10 (October 2004): 486786. 104.10 (October 2004): 486786.

14. 14. J. M. Hunt has calculated that there are 1.55 i 10 J. M. Hunt has calculated that there are 1.55 i 1019 kilograms (10 kilograms (1022 grams) of organic carbon on Earth. Based on this figure, and a.s.suming that all ”organic carbon” is contained in the bioma.s.s (note that the bioma.s.s is not clearly defined, so we are taking a conservatively broad approach), we can compute the approximate number of carbon atoms as follows: grams) of organic carbon on Earth. Based on this figure, and a.s.suming that all ”organic carbon” is contained in the bioma.s.s (note that the bioma.s.s is not clearly defined, so we are taking a conservatively broad approach), we can compute the approximate number of carbon atoms as follows: Average atomic weight of carbon (adjusting for isotope ratios) = 12.011.Carbon in the bioma.s.s = 1.55 i 1022 grams / 12.011 = 1.3 i 10 grams / 12.011 = 1.3 i 1021 mols. mols.1.3 i 1011i 6.02 i 1023 (Avogadro's number) = 7.8 i 10 (Avogadro's number) = 7.8 i 1044 carbon atoms. carbon atoms.J. M. Hunt, Petroleum Geochemistry and Geology Petroleum Geochemistry and Geology (San Francisco: W. H. Freeman, 1979). (San Francisco: W. H. Freeman, 1979).

15. 15. Robert A. Freitas Jr., ”The Gray Goo Problem,” March 20, 2001, /KSRM/5.11.htm. and section 6.3.1, ”Molecular a.s.semblers Are Too Dangerous,” pp. 2046, /KSRM/6.3.1.htm; Foresight Inst.i.tute, ”Molecular Nanotechnology Guidelines: Draft Version 3.7,” June 4, 2000, mendations,” Zyvex preprint, April 2000, section 8.4 ”Malicious Ecophagy” and section 6.0 ”Ecophagic Thermal Pollution Limits (ETPL),” mendations,” Zyvex preprint, April 2000, section 8.4 ”Malicious Ecophagy” and section 6.0 ”Ecophagic Thermal Pollution Limits (ETPL),” /Nano/Microbivores.htm, and ”Microbivores: Artificial Mechanical Phagocytes,” Robert A. Freitas Jr., ”Microbivores: Artificial Mechanical Phagocytes Using Digest and Discharge Protocol,” Zyvex preprint, March 2001, /Nano/Microbivores.htm, and ”Microbivores: Artificial Mechanical Phagocytes,” Foresight Update Foresight Update no. 44, March 31, 2001, pp. 1113, /proactionary.htm and /proactionary.htm and mon reasoning biases. Evaluate risk according to available science, not popular perception, and allow for common reasoning biases.

3. 3. Give precedence to ameliorating known and proven threats to human health and environmental quality over acting against hypothetical risks. Give precedence to ameliorating known and proven threats to human health and environmental quality over acting against hypothetical risks.

4. 4. Treat technological risks on the same basis as natural risks; avoid underweighting natural risks and overweighting human-technological risks. Fully account for the benefits of technological advances. Treat technological risks on the same basis as natural risks; avoid underweighting natural risks and overweighting human-technological risks. Fully account for the benefits of technological advances.

5. 5. Estimate the lost opportunities of abandoning a technology, and take into account the costs and risks of subst.i.tuting other credible options, carefully considering widely distributed effects and follow-on effects. Estimate the lost opportunities of abandoning a technology, and take into account the costs and risks of subst.i.tuting other credible options, carefully considering widely distributed effects and follow-on effects.

6. 6. Consider restrictive measures only if the potential impact of an activity has both significant probability and severity. In such cases, if the activity also generates benefits, discount the impacts according to the feasibility of adapting to the adverse effects. If measures to limit technological advance do appear justified, ensure that the extent of those measures is proportionate to the extent of the probable effects. Consider restrictive measures only if the potential impact of an activity has both significant probability and severity. In such cases, if the activity also generates benefits, discount the impacts according to the feasibility of adapting to the adverse effects. If measures to limit technological advance do appear justified, ensure that the extent of those measures is proportionate to the extent of the probable effects.

7. 7. When choosing among measures to restrict technological innovation, prioritize decision criteria as follows: Give priority to risks to human and other intelligent life over risks to other species; give non-lethal threats to human health priority over threats limited to the environment (within reasonable limits); give priority to immediate threats over distant threats; prefer the measure with the highest expectation value by giving priority to more certain over less certain threats, and to irreversible or persistent impacts over transient impacts. When choosing among measures to restrict technological innovation, prioritize decision criteria as follows: Give priority to risks to human and other intelligent life over risks to other species; give non-lethal threats to human health priority over threats limited to the environment (within reasonable limits); give priority to immediate threats over distant threats; prefer the measure with the highest expectation value by giving priority to more certain over less certain threats, and to irreversible or persistent impacts over transient impacts.

25. 25. Martin Rees, Martin Rees, Our Final Hour: A Scientist's Warning: How Terror, Error, and Environmental Disaster Threaten Humankind's Future in This Century-on Earth and Beyond Our Final Hour: A Scientist's Warning: How Terror, Error, and Environmental Disaster Threaten Humankind's Future in This Century-on Earth and Beyond (New York: Basic Books, 2003). (New York: Basic Books, 2003).

26. 26. Scott Shane, Scott Shane, Dismantling Utopia: How Information Ended the Soviet Union Dismantling Utopia: How Information Ended the Soviet Union (Chicago: Ivan R. Dee, 1994); see also the review by James A. Dorn at /community/guide/lihistory/ny-history-hs331a,0,6101197.story.

28. 28. Jim Oeppen and James W. Vaupel, ”Broken Limits to Life Expectancy,” Jim Oeppen and James W. Vaupel, ”Broken Limits to Life Expectancy,” Science Science 296.5570 (May 10,2002): 102931. 296.5570 (May 10,2002): 102931.

29. 29. Steve Bowman and Helit Barel, Weapons of Ma.s.s Destruction: The Terrorist Threat, Congressional Research Service Report for Congress, December 8, 1999, /meme/frame.html?main=/articles/art0172.html. Eliezer S. Yudkowsky, ”Creating Friendly AI 1.0, The a.n.a.lysis and Design of Benevolent Goal Architectures” (2001), The Singularity Inst.i.tute, /meme/frame.html?main=/articles/art0172.html.

31. 31. Ted Kaczynski, ”The Unabomber's Manifesto,” May 14, 2001, lmeme/frame.html?main=/articles/art0182.html. Ted Kaczynski, ”The Unabomber's Manifesto,” May 14, 2001, lmeme/frame.html?main=/articles/art0182.html.

32. 32. Bill McKibben, Bill McKibben, Enough: Staying Human in an Engineered Age Enough: Staying Human in an Engineered Age (New York: Times Books, 2003). (New York: Times Books, 2003).

33. 33. Kaczynski, ”The Unabomber's Manifesto.” Kaczynski, ”The Unabomber's Manifesto.”

34. 34. Foresight Inst.i.tute and IMM, ”Foresight Guidelines on Molecular Nanotechnology,” February 21, 1999, /meme/frame.html?main=/articles/art0557.html; Chris Phoenix and Mike Treder, ”Safe Utilization of Advanced Nanotechnology,” January 28, 2003, /meme/frame.html?main=/articles/art0547.html; Robert A. Freitas Jr., ”The Gray Goo Problem,” KurzweilAI.net, 20 March 2002, /meme/frame.html?main=/articles/art0142.html. Foresight Inst.i.tute and IMM, ”Foresight Guidelines on Molecular Nanotechnology,” February 21, 1999, /meme/frame.html?main=/articles/art0557.html; Chris Phoenix and Mike Treder, ”Safe Utilization of Advanced Nanotechnology,” January 28, 2003, /meme/frame.html?main=/articles/art0547.html; Robert A. Freitas Jr., ”The Gray Goo Problem,” KurzweilAI.net, 20 March 2002, /meme/frame.html?main=/articles/art0142.html.

<script>