Part 11 (2/2)

Determine how the synaptic strengths of all the connections are adjusted during the training of this neural net. This is a key design decision and is the subject of a great deal of research and discussion. There are a number of possible ways to do this: (1) For each recognition trial, increment or decrement each synaptic strength by a (generally small) fixed amount so that the neural net's output more closely matches the correct answer. One way to do this is to try both incrementing and decrementing and see which has the more desirable effect. This can be time-consuming, so other methods exist for making local decisions on whether to increment or decrement each synaptic strength.

(2) Other statistical methods exist for modifying the synaptic strengths after each recognition trial so that the performance of the neural net on that trial more closely matches the correct answer.

Note that neural net training will work even if the answers to the training trials are not all correct. This allows using real-world training data that may have an inherent error rate. One key to the success of a neural netbased recognition system is the amount of data used for training. Usually a very substantial amount is needed to obtain satisfactory results. As with human students, the amount of time that a neural net spends learning its lessons is a key factor in its performance.

VariationsMany variations of the above are feasible. For example:There are different ways of determining the topology. In particular, the interneuronal wiring can be set either randomly or using an evolutionary algorithm.

There are different ways of setting the initial synaptic strengths.

The inputs to the neurons in layeri do not necessarily need to come from the outputs of the neurons in layer do not necessarily need to come from the outputs of the neurons in layeri1. Alternatively, the inputs to the neurons in each layer can come from any lower layer or any layer.

There are different ways to determine the final output.

The method described above results in an ”all or nothing” (1 or 0) firing called a nonlinearity. There are other nonlinear functions that can be used. Commonly a function is used that goes from 0 to 1 in a rapid but more gradual fas.h.i.+on. Also, the outputs can be numbers other than 0 and 1.

The different methods for adjusting the synaptic strengths during training represent key design decisions.

The above schema describes a ”synchronous” neural net, in which each recognition trial proceeds by computing the outputs of each layer, starting with layer0 through layer through layerM. In a true parallel system, in which each neuron is operating independently of the others, the neurons can operate ”asynchronously” (i.e., independently). In an asynchronous approach, each neuron is constantly scanning its inputs and fires whenever the sum of its weighted inputs exceeds its threshold (or whatever its output function specifies).

10. Robert Mannell, ”Acoustic Representations of Speech,” 2008, clas.mq.edu.au/acoustics/frequency/acoustic_speech.xhtml. Robert Mannell, ”Acoustic Representations of Speech,” 2008, clas.mq.edu.au/acoustics/frequency/acoustic_speech.xhtml.11. Here is the basic schema for a genetic (evolutionary) algorithm. Many variations are possible, and the designer of the system needs to provide certain critical parameters and methods, detailed below. Here is the basic schema for a genetic (evolutionary) algorithm. Many variations are possible, and the designer of the system needs to provide certain critical parameters and methods, detailed below.

The Evolutionary AlgorithmCreate N N solution ”creatures.” Each one has: solution ”creatures.” Each one has:A genetic code: a sequence of numbers that characterize a possible solution to the problem. The numbers can represent critical parameters, steps to a solution, rules, etc.

For each generation of evolution, do the following:Do the following for each of the N N solution creatures: solution creatures: Apply this solution creature's solution (as represented by its genetic code) to the problem, or simulated environment. Rate the solution.

Pick the L L solution creatures with the highest ratings to survive into the next generation. solution creatures with the highest ratings to survive into the next generation.

Eliminate the (N L L) nonsurviving solution creatures.

Create (N L L) new solution creatures from the L L surviving solution creatures by: surviving solution creatures by: (1) Making copies of the L L surviving creatures. Introduce small random variations into each copy; or surviving creatures. Introduce small random variations into each copy; or (2) Create additional solution creatures by combining parts of the genetic code (using ”s.e.xual” reproduction, or otherwise combining portions of the chromosomes) from the L L surviving creatures; or surviving creatures; or (3) Do a combination of (1) and (2).

Determine whether or not to continue evolving:Improvement = (highest rating in this generation) (highest rating in the previous generation).

If Improvement < improvement=”” threshold=”” then=”” we're=””>

The solution creature with the highest rating from the last generation of evolution has the best solution. Apply the solution defined by its genetic code to the problem.

Key Design DecisionsIn the simple schema above, the designer needs to determine at the outset:Key parameters: N L Improvement threshold.

What the numbers in the genetic code represent and how the solution is computed from the genetic code.

A method for determining the N N solution creatures in the first generation. In general, these need only be ”reasonable” attempts at a solution. If these first-generation solutions are too far afield, the evolutionary algorithm may have difficulty converging on a good solution. It is often worthwhile to create the initial solution creatures in such a way that they are reasonably diverse. This will help prevent the evolutionary process from just finding a ”locally” optimal solution. solution creatures in the first generation. In general, these need only be ”reasonable” attempts at a solution. If these first-generation solutions are too far afield, the evolutionary algorithm may have difficulty converging on a good solution. It is often worthwhile to create the initial solution creatures in such a way that they are reasonably diverse. This will help prevent the evolutionary process from just finding a ”locally” optimal solution.

How the solutions are rated.

How the surviving solution creatures reproduce.

VariationsMany variations of the above are feasible. For example:There does not need to be a fixed number of surviving solution creatures (L) from each generation. The survival rule(s) can allow for a variable number of survivors.

There does not need to be a fixed number of new solution creatures created in each generation (N L L). The procreation rules can be independent of the size of the population. Procreation can be related to survival, thereby allowing the fittest solution creatures to procreate the most.

The decision as to whether or not to continue evolving can be varied. It can consider more than just the highest-rated solution creature from the most recent generation(s). It can also consider a trend that goes beyond just the last two generations.

12. Dileep George, ”How the Brain Might Work: A Hierarchical and Temporal Model for Learning and Recognition” (PhD dissertation, Stanford University, June 2008). Dileep George, ”How the Brain Might Work: A Hierarchical and Temporal Model for Learning and Recognition” (PhD dissertation, Stanford University, June 2008).13. A. M. Turing, ”Computing Machinery and Intelligence,” A. M. Turing, ”Computing Machinery and Intelligence,” Mind Mind, October 1950.14. Hugh Loebner has a ”Loebner Prize” compet.i.tion that is run each year. The Loebner silver medal will go to a computer that pa.s.ses Turing's original text-only test. The gold medal will go to a computer that can pa.s.s a version of the test that includes audio and video input and output. In my view, the inclusion of audio and video does not actually make the test more challenging. Hugh Loebner has a ”Loebner Prize” compet.i.tion that is run each year. The Loebner silver medal will go to a computer that pa.s.ses Turing's original text-only test. The gold medal will go to a computer that can pa.s.s a version of the test that includes audio and video input and output. In my view, the inclusion of audio and video does not actually make the test more challenging.15. ”Cognitive a.s.sistant That Learns and Organizes,” Artificial Intelligence Center, SRI International, /project/CALO. ”Cognitive a.s.sistant That Learns and Organizes,” Artificial Intelligence Center, SRI International, /project/CALO.16. Dragon Go! Nuance Communications, Inc., /products/dragon-go-in-action/index.htm. Dragon Go! Nuance Communications, Inc., /products/dragon-go-in-action/index.htm.17. ”Overcoming Artificial Stupidity,” ”Overcoming Artificial Stupidity,” WolframAlpha Blog WolframAlpha Blog, April 17, 2012, blog.wolframalpha.com/author/stephenwolfram/.

Chapter 8: The Mind as Computer1. Salomon Bochner, Salomon Bochner, A Biographical Memoir of John von Neumann A Biographical Memoir of John von Neumann (Was.h.i.+ngton, DC: National Academy of Sciences, 1958). (Was.h.i.+ngton, DC: National Academy of Sciences, 1958).2. A. M. Turing, ”On Computable Numbers, with an Application to the Entscheidungsproblem,” A. M. Turing, ”On Computable Numbers, with an Application to the Entscheidungsproblem,” Proceedings of the London Mathematical Society Proceedings of the London Mathematical Society Series 2, vol. 42 (193637): 23065, lab.ox.ac.uk/activities/ieg/e-library/sources/tp2-ie.pdf. A. M. Turing, ”On Computable Numbers, with an Application to the Entscheidungsproblem: A Correction,” Series 2, vol. 42 (193637): 23065, lab.ox.ac.uk/activities/ieg/e-library/sources/tp2-ie.pdf. A. M. Turing, ”On Computable Numbers, with an Application to the Entscheidungsproblem: A Correction,” Proceedings of the London Mathematical Society Proceedings of the London Mathematical Society 43 (1938): 54446. 43 (1938): 54446.3. John von Neumann, ”First Draft of a Report on the EDVAC,” Moore School of Electrical Engineering, University of Pennsylvania, June 30, 1945. John von Neumann, ”A Mathematical Theory of Communication,” John von Neumann, ”First Draft of a Report on the EDVAC,” Moore School of Electrical Engineering, University of Pennsylvania, June 30, 1945. John von Neumann, ”A Mathematical Theory of Communication,” Bell System Technical Journal Bell System Technical Journal, July and October 1948.4. Jeremy Bernstein, Jeremy Bernstein, The a.n.a.lytical Engine: Computers-Past, Present, and Future The a.n.a.lytical Engine: Computers-Past, Present, and Future, rev. ed. (New York: William Morrow & Co., 1981).5. ”j.a.pan's K Computer Tops 10 Petaflop/s to Stay Atop TOP500 List,” ”j.a.pan's K Computer Tops 10 Petaflop/s to Stay Atop TOP500 List,” Top 500 Top 500, November 11, 2011, top500.org/lists/2011/11/press-release.6. Carver Mead, Carver Mead, a.n.a.log VLSI and Neural Systems a.n.a.log VLSI and Neural Systems (Reading, MA: Addison-Wesley, 1986). (Reading, MA: Addison-Wesley, 1986).7. ”IBM Unveils Cognitive Computing Chips,” IBM news release, August 18, 2011, /press/us/en/pressrelease/35251.wss. ”IBM Unveils Cognitive Computing Chips,” IBM news release, August 18, 2011, /press/us/en/pressrelease/35251.wss.8. ”j.a.pan's K Computer Tops 10 Petaflop/s to Stay Atop TOP500 List.” ”j.a.pan's K Computer Tops 10 Petaflop/s to Stay Atop TOP500 List.”

Chapter 9: Thought Experiments on the Mind1. John R. Searle, ”I Married a Computer,” in Jay W. Richards, ed., John R. Searle, ”I Married a Computer,” in Jay W. Richards, ed., Are We Spiritual Machines? Ray Kurzweil vs. the Critics of Strong AI Are We Spiritual Machines? Ray Kurzweil vs. the Critics of Strong AI (Seattle: Discovery Inst.i.tute, 2002). (Seattle: Discovery Inst.i.tute, 2002).2. Stuart Hameroff, Stuart Hameroff, Ultimate Computing: Biomolecular Consciousness and Nanotechnology Ultimate Computing: Biomolecular Consciousness and Nanotechnology (Amsterdam: Elsevier Science, 1987). (Amsterdam: Elsevier Science, 1987).3. P. S. Sebel et al., ”The Incidence of Awareness during Anesthesia: A Multicenter United States Study,” P. S. Sebel et al., ”The Incidence of Awareness during Anesthesia: A Multicenter United States Study,” Anesthesia and a.n.a.lgesia Anesthesia and a.n.a.lgesia 99 (2004): 83339. 99 (2004): 83339.4. Stuart Sutherland, Stuart Sutherland, The International Dictionary of Psychology The International Dictionary of Psychology (New York: Macmillan, 1990). (New York: Macmillan, 1990).5. David c.o.c.kburn, ”Human Beings and Giant Squids,” David c.o.c.kburn, ”Human Beings and Giant Squids,” Philosophy Philosophy 69, no. 268 (April 1994): 13550. 69, no. 268 (April 1994): 13550.6. Ivan Petrovich Pavlov, from a lecture given in 1913, published in Ivan Petrovich Pavlov, from a lecture given in 1913, published in Lectures on Conditioned Reflexes: Twenty-Five Years of Objective Study of the Higher Nervous Activity [Behavior] of Animals Lectures on Conditioned Reflexes: Twenty-Five Years of Objective Study of the Higher Nervous Activity [Behavior] of Animals (London: Martin Lawrence, 1928), 222. (London: Martin Lawrence, 1928), 222.7. Roger W. Sperry, from James Arthur Lecture on the Evolution of the Human Brain, 1964, p. 2. Roger W. Sperry, from James Arthur Lecture on the Evolution of the Human Brain, 1964, p. 2.8. Henry Maudsley, ”The Double Brain,” Henry Maudsley, ”The Double Brain,” Mind Mind 14, no. 54 (1889): 16187. 14, no. 54 (1889): 16187.9. Susan Curtiss and Stella de Bode, ”Language after Hemispherectomy,” Susan Curtiss and Stella de Bode, ”Language after Hemispherectomy,” Brain and Cogn Brain and Cognition 43, nos. 13 (JuneAugust 2000): 13538.10. E. P. Vining et al., ”Why Would You Remove Half a Brain? The Outcome of 58 Children after Hemispherectomy-the Johns Hopkins Experience: 1968 to 1996,” Pediatrics 100 (August 1997): 16371. M. B. Pulsifer et al., ”The Cognitive Outcome of Hemispherectomy in 71 Children,” E. P. Vining et al., ”Why Would You Remove Half a Brain? The Outcome of 58 Children after Hemispherectomy-the Johns Hopkins Experience: 1968 to 1996,” Pediatrics 100 (August 1997): 16371. M. B. Pulsifer et al., ”The Cognitive Outcome of Hemispherectomy in 71 Children,” Epilepsia Epilepsia 45, no. 3 (March 2004): 24354. 45, no. 3 (March 2004): 24354.11. S. McClelland III and R. E. Maxwell, ”Hemispherectomy for Intractable Epilepsy in Adults: The First Reported Series,” S. McClelland III and R. E. Maxwell, ”Hemispherectomy for Intractable Epilepsy in Adults: The First Reported Series,” Annals of Neurology Annals of Neurology 61, no. 4 (April 2007): 37276. 61, no. 4 (April 2007): 37276.12. Lars Muckli, Marcus J. Naumerd, and Wolf Singer, ”Bilateral Visual Field Maps in a Patient with Only One Hemisphere,” Lars Muckli, Marcus J. Naumerd, and Wolf Singer, ”Bilateral Visual Field Maps in a Patient with Only One Hemisphere,” Proceedings of the National Academy of Sciences Proceedings of the National Academy of Sciences 106, no. 31 (August 4, 2009), dx.doi.org/10.1073/pnas.0809688106. 106, no. 31 (August 4, 2009), dx.doi.org/10.1073/pnas.0809688106.13. Marvin Minsky, Marvin Minsky, The Society of Mind The Society of Mind (New York: Simon and Schuster, 1988). (New York: Simon and Schuster, 1988).14. F. Fay Evans-Martin, F. Fay Evans-Martin, The Nervous System The Nervous System (New York: Chelsea House, 2005), /doc/5012597/The-Nervous-System. (New York: Chelsea House, 2005), /doc/5012597/The-Nervous-System.15. Benjamin Libet, Benjamin Libet, Mind Time: The Temporal Factor in Consciousness Mind Time: The Temporal Factor in Consciousness (Cambridge, MA: Harvard University Press, 2005). (Cambridge, MA: Harvard University Press, 2005).16. Daniel C. Dennett, Daniel C. Dennett, Freedom Evolves Freedom Evolves (New York: Viking, 2003). (New York: Viking, 2003).17. Michael S. Gazzaniga, Michael S. Gazzaniga, Who's in Charge? Free Will and the Science of the Brain Who's in Charge? Free Will and the Science of the Brain (New York: Ecco/HarperCollins, 2011). (New York: Ecco/HarperCollins, 2011).18. David Hume, David Hume, An Enquiry Concerning Human Understanding An Enquiry Concerning Human Understanding (1765), 2nd ed., edited by Eric Steinberg (Indianapolis: Hackett, 1993). (1765), 2nd ed., edited by Eric Steinberg (Indianapolis: Hackett, 1993).19. Arthur Schopenhauer, Arthur Schopenhauer, The Wisdom of Life The Wisdom of Life.20. Arthur Schopenhauer, Arthur Schopenhauer, On the Freedom of the Will On the Freedom of the Will (1839). (1839).21. From Raymond Smullyan, From Raymond Smullyan, 5000 B.C. and Other Philosophical Fantasies 5000 B.C. and Other Philosophical Fantasies (New York: St. Martin's Press, 1983). (New York: St. Martin's Press, 1983).22. For an insightful and entertaining examination of similar issues of ident.i.ty and consciousness, see Martine Rothblatt, ”The Terasem Mind Uploading Experiment,” For an insightful and entertaining examination of similar issues of ident.i.ty and consciousness, see Martine Rothblatt, ”The Terasem Mind Uploading Experiment,” International Journal of Machine Consciousness International Journal of Machine Consciousness 4, no. 1 (2012): 14158. In this paper, Rothblatt examines the issue of ident.i.ty with regard to software that emulates a person based on ”a database of video interviews and a.s.sociated information about a predecessor person.” In this proposed future experiment, the software is successfully emulating the person it is based on. 4, no. 1 (2012): 14158. In this paper, Rothblatt examines the issue of ident.i.ty with regard to software that emulates a person based on ”a database of video interviews and a.s.sociated information about a predecessor person.” In this proposed future experiment, the software is successfully emulating the person it is based on.23. ”How Do You Persist When Your Molecules Don't?” ”How Do You Persist When Your Molecules Don't?” Science and Consciousness Review Science and Consciousness Review 1, no. 1 (June 2004), /articles/20040601.xhtml. 1, no. 1 (June 2004), /articles/20040601.xhtml.

<script>