CONNECTIONISM

A design and set of working assumptions which characterise a group of models of language processing. Connectionist models are often designed for computer implementation. They are constructed in a way that resembles the configuration of the human brain, where information is transmitted via massively interconnected neural networks. The belief is not that one necessarily replicates the operations of the brain by taking it as a model. It is (a) that a model which resembles the brain potentially provides a more plausible account than one that does not; and (b) that by using this kind of architecture we may gain incidental insights into at least some of the brain’s functions.

Like the brain, connectionist models consist of a large number of simple processing units with multiple connections linking them. Activation flows along the connections, just as electrical impulses transmit information through neurons in the brain. The ease with which activation spreads from one unit to another is determined by the strength of the connections along which it travels. The stronger the connection to a unit, the more readily that unit becomes activated. A connection’s strength depends upon how frequently it is used. Thus, over time, connections to a frequent word will become strong, ensuring that the word is activated more rapidly than other less common ones.

 One of the earliest connectionist models was the IAC (Interactive Activation and Competition) model (McClelland and Rumelhardt, 1981), which explored written word recognition. The architecture of the IAC contains a number of aspects which characterise later connectionist models. It consists of units at three different levels, corresponding to letter features (curves, vertical lines, oblique lines, etc.), whole letters and whole words. Figure C2 shows a simplified version. An important characteristic of all connectionist models is that the various levels of operation are regarded as being active simultaneously (in parallel).

Note: Dotted lines indicate weak activation. Note the interactive nature of the model, with knowledge of the word LOVE influencing recognition of the letters as well as vice versa. Within-level inhibitory links (e.g. evidence for LOVE reducing the activation of LIVE, are not shown.

Each unit is connected to all the units in the levels above and below it. These connections can be facilitatory (or excitatory), meaning that they enhance the level of activation in the unit to which they lead. Or they can be inhibitory, in which case they reduce the level of activation. The connection between a curve at feature level and the letter G at letter level would be facilitatory; but the connection between the curve and the letter F would be inhibitory since the curve provides evidence against the presence of an F.

In the IAC model, there are also inhibitory connections between units at the same level. As evidence accumulates for the presence of G, its activation increases and at the same time that of (say) V is reduced. Not all connectionist models include these within-level connections.

The operation of a connectionist model of word recognition is assumed to take place over time, with the activation of some candidate words building up gradually while that of others declines. As configured on computer, processing takes place in cycles, which correspond to the passage of time. After each cycle, the activation level of each unit is updated. In time, one unit outstrips all the others at that level, permitting recognition to occur; the system is then said to have achieved equilibrium.

Some connectionist models, including the IAC, permit activation to flow in a top-down direction: i.e. not just from letters up to words, but also from words down to letters. Suppose that evidence has built up at word level for the unit WORK. Activation then flows back down to letter level, where it facilitates the recognition of the final letter K but inhibits the letter F since there is no such word as WORF. Models which permit this kind of two-way flow of activation are described as highly interactive.

The connectionist concept has been widely applied. Just as the IAC simulates written word recognition, so a later model, TRACE (McClelland and Elman, 1986), simulates spoken word recognition. Connectionist principles also underlie standard accounts of lexical storage, in which connections link a particular concept to its characteristics (TOMATO is linked to ROUND, to RED and to SOFT), as well as to associated concepts including co-hyponyms and superordinates (LETTUCE and SALAD). As ever, the connections between units vary in strength. Thus, the connection between TOMATO and SALAD is stronger than the connection between TOMATO and FRUIT.

A strength of the connectionist approach is that, besides modelling processes such as word recognition, it can also model learning. In a computer simulation, each connection receives a number or weight to indicate its relative strength. At the outset, these can be set at 0; but, as connections are used, their weights are adjusted by means of a complex formula. If a connection is not used, its weight declines to a negative value, indicating an inhibitory relationship.

The effectiveness of this learning process has been increased by a feedback mechanism known as back propagation which provides the program with a kind of memory. It compares what a connectionist network outputs from a particular stimulus with what it should output (given the input SING, it might compare its output of the past tense form singed with the correct form sang). By dint of many repeated presentations of the input, some connections within the network become strengthened while others become weakened. In this way, the network can gradually be ‘trained’ to produce correct responses through a process of error reduction.

Using back propagation, a connectionist program has managed to simulate the acquisition of a set of regular and irregular English Past Simple verb forms. It succeeded in discriminating between cases where an-ed inflection was appropriate (WALK– WALKED) and those where a new form had to be learnt (WRITE– WROTE). It also, in the process, manifested the kind of U-shaped development observed both in first and second language acquisition, where a speaker acquires a correct irregular form, then later replaces it with a regular form that has been over-generalised (writed). Simulations such as this are sometimes cited in support of an empiricist view of language acquisition. They suggest that linguistic patterns can be identified through exposure to multiple examples, with no need to presuppose a genetically transmitted mechanism which drives the acquisition process.

 However, some caution is needed: the program only acquired a small subset of verb forms and did so after a large number of passes. It was also dedicated to a single learning operation, whereas a child has to acquire meaning as well as form and encounters a verb in many more forms (writing, written) than simply the past. In addition, current connectionist programs are dependent upon the operator inputting precise lexical or phonemic information; this cannot be said to resemble the connected nature and considerable variation of natural speech.

A feature of the early connectionist models was that each operating unit was deemed to represent a word. Thus, there was a unit for WRITE and another for WROTE, to which it was connected. Current theory questions whether words are explicitly represented in this way. Instead, the evidence for (say) WROTE may take the form of a state in which several abstract units reach a particular level of activation. The cues which characterise WROTE are thus distributed over a number of units. This more recent type of connectionist architecture is referred to as parallel distributed processing or PDP.

See also: Interactive activation, Parallel distributed processing

 Further reading: Ellis and Humphreys (1999); Elman et al. (1996: Chap. 2)

اخر الاخبار

اخبار العتبة العباسية المقدسة

المجمع العلمي للقرآن الكريم يصدر الكتاب الثاني من سلسلة يخشى

قسم الشعائر يكرّم أصحاب المواكب المشاركة في إحياء ذكرى شهادة السيدة الزهراء (عليها السلام)

جامعة العميد تستقبل طلبة المراحل الأولى المقبولين ضمن كلياتها للعام الدراسي الجديد

سماحة السيد الصافي يكرّم طلبة حفظ القرآن الكريم المتفوّقين ويؤكّد دور التدبّر في الهداية والتكامل

المزيد

الآخبار الصحية

طبيب يكشف تأثير برودة الشتاء على النوم

دون أن تعلم.. 10 عادات يومية تسرع شيخوخة جسمك

تقرير: النيكوتين خطر على القلب مهما كانت الطريقة

أيهما أفضل لصحتك.. زبدة الفول السوداني أم النوتيلا؟

المزيد

الآخبار التكنلوجية

السعودية تدشن أكبر نظام لتخزين الكهرباء بالبطاريات في العالم !

بأمر تنفيذي.. ترامب يؤكد هدف عودة الأميركيين إلى القمر في عام 2028

بعيدا عن البشر.. "قتال جوي في الفضاء" بين أميركا والصين

كشف مذهل في قاع "برمودا" !

المزيد

مواضيع ذات صلة

Problems with the dictionary view

The dictionary view

Dictionaries versus encyclopaedias

The encyclopaedic view of meaning

Embodiment and conceptual structure summary

Schematic systems

Semantic structure

Conceptual structure

Image schemas and abstract thought

Properties of image schemas

What is an image schema?

Image schemas