Lecture 6: Extrinsic and intrinsic motivation

(9-26-22)

A thing is worth whatever the buyer will pay for it.

Publius Syrus 50 B.C.E.

“What do you enjoy doing?” a question I often ask when in my private practice office

Your author uses the term “incentive” to refer to an environmental event that attracts or repels a person toward or away from an action: “incentives always precede behavior” (p. 101). This took me a while to get used to this use of the term because everyday language tends to use incentive as another word for consequence, what might follow a behavior to affect it’s frequency.

Dr. Reeve’s understanding of “behavioral psychology” seems based on very old models, such as the stimulus-response theories of Miller and Dollard. These were built on a single factor learning theory (respondent conditioning or learning, Pavlovian conditioning: the associative connection of stimuli occurring contiguously so that both now elicited responses originally only produced by one). A limitation of these theories was that they could only explain how behavior came under the control of different stimuli, not how new behavior were developed in the organism. These early learning theories were supplanted by two-factor models (the instrumental learning of Thorndike and operant learning of Skinner. One of the contributions of Dr. Skinner was a clear picture of how novel behavior could be “shaped” by differentially rewarding successive approximations of a desired outcome. (The two-factor models were, in turn, supplanted by the three-factor learning models of Rotter and Bandura as cognitive factors gained importance in understanding behavior: respondent conditioning, operant learning, observational learning [modeling, imitation]).

In the language of operant psychology such events as Dr. Reeve call incentives are referred to as “discriminative stimuli” (they inform organisms as to prevailing reinforcement contingencies). The light comes on in the Skinner box and signals that bar pressing by the rat will now be followed by food pellets. The 40 MPH sign on Main Street signals that it is now safe to increase your speed above the 30 MPH city street without attracting the attention of a passing patrol car. Discriminative stimuli can be thought of as signals that let us know what the current “rule” are for different outcomes (“contingencies of reinforcement”).

Stengths (and weaknesses) of “Skinnarian” (operant psychology, applied behavior analysis) approaches to understanding behavior (learning)

• Skinner was an operational behaviorist: he did not deny the existance or even the importance of mental event but did question their usefulness to a “science of behavior”
  • The goal of science is the prediction (sometimes control) of events (behavior)
  • This can best be achieved by focusing on observable actions/events that can be reliably be measured
  • This can best be achieved by focusing on highly control circumstances that allow control of most variables
  • This can best be achieved by focusing on frequency of occurrance as the primary outcome variable (topology of behavior ignored, except for shaping)
  • An “operant” behavior operates on the environment, producing a certain effect
  • While Skinner work is often presented as a “learning model”, it is really a “performance model”
• Positive consequences: understanding of contingencies of performance, stimulus effects, superstitious behavior, applied developments in clinical and school settings
• Limitations:
  • The emphasis on a strict two factor learning model minimized the attention given to observations learning (and language) along with other “cognitive” processes; Skinner attempts to explain both modeling and language from purely operant operations
  • Motivation (another cognitive, i.e. intervening variable) receives minmal attention. The concept of reinforcment is defined in terms of its effect on behavior
  • The issue of generalizing from highly controlled situations to “real life” (a constant barrage of multiple, competing reinforcement schedules and discrimination stimuli) remains elusive

Contingencies of reinforcement

• Your author’s use of the term “incentive” parrelles how operent psychologist would speak of “contingency”: the relationship between discriminative stimuli, behavior, and consequences.
• He identifies the “incentive” with “enviornmental event that attracts or repels a person toward or away from initiating a particular course of action” by “create in the person an expectation”, incentives are “learned through experience” “Incentive always precede behavior” leading to the “initiation of behavior”; “consequences”, “always follow the behavior” and “increase or decrease the persistence of behavior.” (p. 101).
• Your author believes that, “a reinforcer must be defined in a manner that is independent from its effects on behavior.” Actually, Dr. Skinner did not do this, he did the exact opposite: he define reinforcement in terms of it’s effects on behavior. This raises its own problems for both theory and application, but it is how he did it and how most discussions of “operant psychology” continue to define it. A practical difficulty of this “functional definition” of reinforcement is that you never know if a stimulus is going to be a reinforcer for a person at any given point in time, the best you know is that, in similar circumstances, in the past it functioned as a reinforcer.
• there are four types of consequences: positive reinforcers, negative reinforcers, [positive] punishment, [negative] punishment: usually referred to as “response cost” or “time out”
• Despite all my carping about Ch. 5, Dr. Reeve does a nice job on p. 103 of discussing many of the factors that will influence reinforcer effectiveness (or “power”): quantity or intensity, immediacy, degree of deprivation for that stimulus, value of that stimulus to recipient (what Dr. R. calls “fit” or “perceived value”)
• Dr. Reeve’s says there are three types of consequences: positive reinforcers, negative reinforcers, and punishers (p. 103). Most texts will tell you there are four types of consequences, corresponding to either the presentation or withdrawal of two categories of stimuli, positive and negative.
• Positive reinforcement: an action is followed by a reinforcing stimulus
• Negative reinforcement: an action is followed by removal of an aversive stimulus
• Punishment: an action is followed by an aversive stimulus
• Dr. Reeve gets all this correct (not a trivial success, I have seen college textbooks confuse negative reinforcement and punishment). What he leaves out is the fourth possibility:
• Response cost: an action is followed by removal of a reinforcing stimulus.
• Response costs (library fines, speeding tickets, loss of privilege , taking away something you want) are a common and important aspect of environmental control of behavior. Especially since humans tend to be “risk averse”, we are more heavily influenced by possible losses than by possible gains. (We’ll need to talk about that more latter.)

Factors influencing the impact of a previously reinforcing stimulus

• deprivation/satiety
• amount/intensity
• timing
• competing contingencies
• pleasure versus satisfaction
  • Aaron Beck: pleasure vs coping

A continuum of potential reinforcing consequences

• information
• attention/social reinforcers
• activity reinforcers
  • enjoyable activities, challenging activities, the Premack principle (access to high frequency behaviors will reinforce lower frequency behaviors)
• material reinforcers
  • consumable reinforcers (food, drink, scratch off lottery tickets)
  • non-consumable reinforcers (toys, clothes, a new smart phone)
• generalized conditioned reinforcers (token reinforcement): stimuli without any intrinsic value that can be exchanged for a range of other stimuli (think cash, Chuck e Cheese tokens)
• I would suggest that the power of these events increase as you go down the list, and also the difficulties and complications of you or a family employing these events (satiation, cost, availability, complexity, and other problems).

Schedule of consequences

• Another important aspect of reinforcement (and the other operations) is the schedule the contingencies arrange between the behavior and the consequence. This is usually discussed in terms of schedules positive reinforcement (but the same schedules apply to the other three operations):
• A “continuous” schedule of reinforcement means that every instance of the behavior under consideration is followed by a positive reinforcement (assuming we know what is reinforcing for the organism at that moment). Every time you perform the action you are paid off by the universe. This is actually rather rare in everyday life, most natural contingencies we observe are on some type of “variable schedule” (sometimes you get paid, sometimes you don’t).
• A continuous schedule of consequences generates the most rapid acquisition of behavior. It is often used in training a new behavior. Variable schedules of reinforcement tend to general more lasting histories of performance. The variable schedules of reinforcement (ration, interval, delayed) have interesting and sometimes practical characteristics (well understood by the people who operate gambling casinos) but that is for another course. What is especially important to our discussion is what happens when a schedule of reinforcement is discontinued: the behavior may occur but is, now, no longer followed by a positive reinforcer. The technical term is “extinction.”

Strengthening, suppressing, and weakening behavior probabilities

• reinforcement appears to strengthen the likelihood a a behavior occurring again the a similar situation
• punishment appears to suppress behavior (without affecting response strength)
• extinction weakens behavior response strength

Punishment

• Skinner defined aversive stimuli as consequences that could negatively reinforce a behavior that terminated this stimulus (Again, note: a “functional definition”: the stimulus is defined by the effect it has on behavior.). When aversive stimulus are presented following a behavior the operation is known as punishment.
• Skinner’s research (and that of others) demonstrated some important differences between the effects of punishment and extinction. Punishment can suppress behavior but tends to have little (Skinner believed no effect) effect on response strength. Extinction actually decreases response strength.
• Punishment does have a number of undesirable side effects, which your author discusses reasonable accurately. It often lead to emotional behavior and sometimes aggression in the recipient, can motivate non-helpful/nonadaptive avoidance behavior, models aversive control, and can lead to the punishing agent being feared (or, in extreme cases, over-identified with as in abused child and Stockholm syndrome cases). Finally, any beneficial effects of punishment are very dependent upon the punishment procedure being carried out close to optimal with respect to elements associated with effective suppression (maximum intensity, immediate delivery, for every instance of the target behavior), deviation from these optimum aspects lead to rapid decline of any effect. [In contrast, positive reinforcement procedures are very “robust”: you can deviate quite a bit from optimal characteristics and still get a good result. Moral of the day: It is far easier to encourage desirable behavior than get rid of undesirable behavior (Except by crowding it out with good behavior.).

Less desirable aspects of punishment

• undesirable escape and avoidance efforts
• counter-control efforts
• models aversive control
• beneficial effects are very dependent upon optimal learning circumstance (immediate, continuous schedule, maximum intensive)

Extinction effects: there are predictable short term effects associated with extinction:

• ncreased frequency of the response (“response bursting”)
• increased intensity (magnitude) of the response
• variable topology (form) of behavior
• emotional effects (distress, frustration, crying, anger)
• (sometimes) aggression
• [we do not like it when the universe cheats. This, I believe, accounts for most of your author’s arguments regarding the “hidden costs of reward”]
• Extinction is what you experience when you favorite pop/junk food machine “cheats” you by failing to deliver you selection. Consider you typical response: punch the button several more times, punch the button harder, hold the button depressed for a longer period, feel frustrated and angry, and (possibly) curse at or kick the machine. The machine has reliably functioned in the past and now appears to be broken–you make an effort to restore appropriate functioning. This is how an organism responds to being placed on an extinction schedule.
• I believe that understanding extinction phenomena is important in unraveling the results of experiments on the so-called cost of extrinsic reinforcement.
• [Truth in lecturing warning (Also known as Teacher Bias): I don’t believe these is a hidden cost effect of extrinsic reinforcement, I don’t believe that there is an essential difference between extrinsic and intrinsic reinforcement, I do believe that different consequences have different (and varying) values to us, and I do believe we all experience multiple schedules of reinforcement and multiple degrees of responsiveness of different consequences continuously throughout our life.]
• I believe that extinction effects and a restricted range of experimental circumstances are a better explanation for the reputed negative effects of extrinsic reinforcement than a hypothesized erosion of intrinsic motivation. But, you should not believe something just because I believe it. If this question interests you, go take a look at the experimental and clinical literature on reinforcement as a learning and therapeutic tool.
• Your author believe that there are three type of motivation: amotivation, extrinsic motivation, and intrinsic motivation.

Intrinsic Motivation

“Intrinsic motivation is the inherent desire to seek out novelty and challenge, to explore and investigate, and to stretch and extend one’s capacities” (p. 107), alternatively it could be view as the reinforcing effects of an optimum (medium) level of arousal.

• Amibile and colleagues (1994) begin their discussion of the Work Preference Inventory by remarking: “Some people seem to be driven by a passionate interest in their work” (p. 950) and go on to quote Nobel laureate Arther Schawlow: “The labor of love aspect is important” The Successful scientists . . . are just impelled by curiosity” (p. 950). They define this “labor of love” as intrinsic motivation: “the motivation to engage in work primarily for its own sake, because the work itself is interesting, engaging, or in some way satisfying.” (p. 950).
• Intrinsic motivation in inherent in the activity being performed. Deci & Ryan (1985) believe that extrinsic motivated behavior is forced by the envirment and intrinsically motivated behavior is freely chosen
  • However, behavior that is initially performed for extrensic motivation could, over time, become to be satisfying for itself. This shift of motivation was labelled by Allport (1937) as the concept of the, “functional autonomy of motives”, “what was a means to an end has become an end in itself.” (p. 150).
  • Deckers (2018) discusses possible purposes of intrinsically motivated behavior:
    • Curiosity: intrinsically motivated behavior may be in service of satisfying curiosity and developing competence and self-determination (p. 328).
    • Effectance motivation: “the motive to actively interact and control one’s environment” (p. 329). Mastery motivation, internal locus of control, efficacy motivation, and competence motivation are similar ideas.
    • Flow: the subject experience of being completely involved in some challenging activity that matches the individual’s skills (Csikzentmihalyi, 1975, 1988; Csikszentmihaly & Rathunde, 1993). Similar concepts can be found in the mindfulness meditation literature.

Extrensic Motivation

• Amibile et al., (1994) contrast this with extrinsic motivation: “some people seem to be motivated more by external inducements in their work.” (p. 950), “the motivation to work is primarily in response to something apart from the work itself, such as reward or recognition or the dictates of other people.” (p. 950).
• For Deckers (2018) incentive refers to the motivational properties of a reinforcer (p. 310). He discusses several factors which may influence the incentive value:
  • Incentive value refers to the attractiveness of an incentive and is based on objective properties (number or amount). Subject incentive value refers to an individual’s appraisal of the objective value. This distinction often plays into economic considerations of motivation.
    • for economists subjective value is synonymous with utility: the satisfation, pleasure, or usefullness of an economic good.
    • Fechner’s law refers to the smaller and smaller increases in sensation as a stimulus is increased. Some economists note a similar relationship between money and utility.
  • Loss aversion: we are often more sensitive to possible loss than possible gain, this greater psychological intensity of an incentive loss compared to a gain is refered to by Deckers as: losses loom larger than gains. This was studied by Kahneman and Tversky (Kahneman, 2011; Kahneman & Tversky, 1979, 1982).
  • There are individual differences among people in personality traits and stable psychological needs, this leads to different appraisals of incentive value. There are also difference in how acceptable different individuals find being influenced by others and how willingly they engage in actions to please others.
  • Schedules of reinforcement affect performance rates and resistence to extinction.
  • Contrast effects with other available reinforcers, which are affected by cognitive appraisals (Contrast effects are greater if people believe/are told the comparison stimuli belong to the same category rather than a different category. Contast effect depend on individual’s expectations.
  • How soon the reinforcer will be available affects the incentive value: delay discounting reflects a decreasing value of stimuli depending on when they weill become available. (This may reflect dlPRC activity. consider also the literature on delay of gratification, self-control, and the marshmellow test.)

Amotivation

• Your author suggests that amotivation is a consequence of a lack of competence, lack of autonomy, and lack of relatedness: “With amotiavtion, the person turns passive, ineffective (overwhelmed), and lacks purpose (p. 120).
• Alternatively amotivation can be viewed as an aspect of clinical depression: a lack of drive, anhedonia, and expectations of negative outcome.
• Alternatively amotivation can be a consequence of neurological disease or injury to the frontal or limbic systems associated with the initiation of behavior or the reinforcement of responses.

The interaction between extrinsic and intrinsic motivation

• Your author believe that extrinsic reward can undermine intrinsically motivated behavior. However, extrinsic motivation tends to enhance performance measures.
• Your professor believes that a more fundamental relationship (than considering intrinsic and extrinsic motivation) is that we usually do not like it when the universe cheats (in our opinion) and react badly (within our ability and choices).
• That said, many authors/researchers/clinicians/managers use the concepts of intrinsic and extrinsic motivation. It is useful to be familiar with these ideas (Also these ideas may be reflected in qustions on the GRE Psychology exam or state licensing board examinations)

Cross connections

• An interesting article by Di Domenico and Ryan (2017) expands the discussion of intrinsic motivation both historically and neurologically. They begin with the standard Self-determination theory: “Intrinsic motivation refers to the spontaneous tendency ‘to seek out novelty and challenges, to extend and exercise one’s capacity, to explore, and to learn’ (Ryan and Deci, 2000, p.70).”
  • They reject drive theory explanations of exploratory and mastery behaviors and review data that these, “are primarily energized by interest and appetitive mastery tendencies,not anxiety reduction.” (p. 3).
  • They review similar ideas: White (1959) effectance motivation: “a general behavioral and developmental propensity of many organisms” (p. 3), “White (1059) argued that effectance motivation is inherent to the activity of the central nervous system and described is as ‘what the neuromuscular system wants to do when it is otherwise unoccupied (e.g., by strong homeostatic drives) or is gently stimulated by the environment’ (p. 321).” (p. 3). “According to White (1959), the satisfactions associated with the effectance motive are not tied to consummatory activities, but are instead intrinsic to the arousal and maintenance of the activities that stem from it.” (p. 3).
  • They also cite DeCharms work on the origin/pawn position people take, “”DeCharms (1968) proposed that intrinsic motivation is based in people’s ‘primary propensity’ to experience themselves as causal agents, that is to experience their own actions as having an internal perceived locus of causality.” (p.3)
• They then turn to: “neuroethological perspectives on mammalian exploration: a starting point for conceptualizing intrinsic motivation in the brain”
  • They suggest, “the concept of intrinsically motivated exploration is consistent with the ‘affective neuroethologcal’ perspective of Panksepp and colleagues . . . . These researchers have argued that mammals are hardwired with a general-purpose SEEKING system that energizes many types of foraging and exploratory activities. Although the SEEKING system does service homeostatic imbalances and is responsible for energizing learned appetitive behaviors, it continuously operates to keep animals in a state of exploratory engagement with with environments. That is, the SEEKING system is believed to function as an objectless appetitive system–a “gord without a goal”–until the exploratory disposition it produces leads to the discovery and learning of useful regularities.” (p. 4). They cite Panksepp and Biven’s discuss of the SEEKING system (2012, p. 135).
  • “The core structures that comprise the SEEKING system in the rat are the ventral tegmental area (VTA), the nucleus accumbens (NAcc), the ventromedial prefrontal cortex (VMPRC), and the dopaminergic projections originating from the VTA the innervate these areas . . . . These regions are frequently called the ‘brain reward network.”
  • “The SEEKING system is thus believed to energize ‘many mental complexities that humans experience as persistent feelings of interest, curiosity, sensation seeking, and in the presence of a sufficiently complex cortex, the search for higher meaning’ (Panksepp, 1998, p. 145).” (p. 4)
• They consider related perspectives
  • Csikszentmihalyi (1990) concept of flow: “Flow refers to experiential states of total absorption, optimal challenge, and non-self-conscious enjoyment of an activity” 9p. 4), “like intrinsic motivation, when people experience flow, the satisfactions they experience are inherent to the activity itself and their behavior is ‘autotelic’ (auto = self, telos = goal) or performed for its own sake.” (p. 4)
  • Loewenstein (1994) proposed an ‘information gap’ hypothesis of curiosity according to which curiosity arises when people experience a discrepancy between what they know and what they want to know. Although this knowledge discrepancy is supposedly experienced as aversive, satisfying curiosity is pleasurable and people therefore voluntarily seek out elicit curiosity.” (p. 5). They go on the point out: “Perhaps the most notable divergence between SDT and Loewenstein’s account concerns his description of curiosity as a consummatory, drive-reduction process–i.e., the closure of information gaps.” (p. 5)
  • A final possibly related concept they discuss is the higher-order personality trait of plasticity (the shared variance of extraversion and openness/intellect) from the Five-Factor or Big Five model of personality. DeYoung (2010, 2013) argued that plasticity “represents stable interindividual differences in people’s exploratory tendencies” (p. 5), “people high in plasticity are hypothesized to ‘desire exploration for its own sake (i.e., they treat it as a goal in itself) and engage in it even at times when exploration will not obviously further their goals’ (DeYoung, 2013, p. 8).” (p. 5), and references , “association with dopamine” (p. 5).
• They propose a preliminary neurobiological model of human intrinsic motivation with two propositions:
  • Proposition I; intrinsic motivation is supported by dopaminergic systems
    • They review three lines of evidence: (1) intrinsic motivation is an elaboration of the exploratory activities subserved by the mammalian SEEKING system and dopamine is central to the neurochemistry of this system. (2) like I.M., dopamine is associated with increased positive affect, cognitive flexibility, creativity, and exploration in the face of novelty. (3) some evidence of a direct link between I.M and dopamine.
    • Dopamine neurons originate in the midbrain and have two modes of activity: tonic and phasic. In tonic mode: neurons exhibit a steady baseline rate of firing, promotes the normal functioning of relevant neural circuits and “may reflect the general strength of animals’ exploratory SEEKING tendencies (p. 6). in the phasic mode, dopamine neurons exhibit short burst os activity or inactivity, “The phasic mode of dopamine transmission may ‘transiently activate SEEKING patterns in coincidence with specific cue- or context-dependent information, attributing to such information an incentive motivational, action-orienting effect’ (Alcaro and Panksepp, 2011, p. 1810).” (p. 6).
    • Brobeg-Martin et al. (2010) proposed a model based on two types of dopamine neurons that exhibit distinct types of phasic activity: value-coding neurons and salience-coding neurons. Value-coding neurons are phasically excited by unexpected rewarding and inhibited by unexpected aversive events.” Following Tricomi and DePasque (2016), we therefore propose that a high rate of dopaminergic signaling within the value system is inherent to the performance of intrinsically motivating activities.”
    • “DeYoung ( 2013) proposed that salience-related dopaminergic activity energizes exploration ‘in response to the incentive value of the possibility of gaining information–that is, it drives curiosity and the desire for information’ (p. 4).” (p. 7)
  • Proposition II: intrinsic motivation entails dynamic switching between brain networks for salience detection, attentional control and self-referential cognition
    • salience network: believed to support the detection of subjectively important events and mobilization of attentional and working memory resources in service of goal-directed behavior
      • anchored in the anterior insula (AI) and dorsal ACC and includes major subcortical nodes in the amygdala, NAcc, the SN, and VTA (p. 8)
      • suggests “the AI functions as a dynamic hub for modulating the activity of two other large-scale brain networks: default mode network: high level of activity during passive resting states involving internally-focused, self-referential cognition and the central executive network (DLPFC Y posterior parietal cortex (PPC); important substraits of working memory and executive functions, show elevated activity during cognitively demanding, externally focused activities. The default mode and central executive networks often function in an antagonistic manner: activity in one is accompanied by suppressed activity in the other
      • They suggest that this antagonistic dynamic between the default mode and central executive networks, along with the role of the salience-mediating switching instigated by the anterior insula (AI) “may inform three characteristics of intrinsic motivation” (p. 9): (1) intrinsic motivation entails cognitive absorption and non-self-conscious enjoyment of an activity; (2) i.m. is reliably associated with enhanced performance, cognitive flexibility, and deeper conceptual learning: consistent with greater mobilization of central executive network; (3) “classic perspectives that describe autonomy or authenticity as a state of ‘organismic congruence’ (e.g., Rogers, 1961) characterize it as an embodied cognitive process whereby sensory and visceral information is permitted to access and direct one’s attention, in a bottom-up manner, to events of subjective importance and meaning”. . . . “The salience network, and the AI most specifically, with its receipt of sensory and visceral input and its interoceptive functions. . . . would seem well-suited to support this aspect of autonomy, especially during intrinsic motivation when people orient themselves to stimuli that spontaneously grip their attention and interest.” 9. 9)
• They finally consider the third leg” “beyond exploration, curiosity and mastery: intrinsically motivated social play”
  • The subcortical PLAY system governs the rough-and-tumble (R&T) interactions of mammals, energizing them to develop and refine their physical, emotional, and social competencies in a safe context” (pp. 10-11), “in early mammalian development, R&T play constitutes a type of embodied social cognition that provides a basis for cooperation and the adaptive self-regulation of aggression.” (p. 11), “We might therefore regard play as intrinsically motivated socialization . . . . an expression of people complementary tendencies toward autonomy and sociality in development, . . . . Indeed, research in SDT suggests that in addition to competence and autonomy, people have a basic psychological need for relatedness, a sense of feeling meaningfully connected with others (p. 11). They conclude that more work is needed to, “differentiate these types of intrinsic motivation in humans.” (p. 110)