Reinforcement Learning Recruits Somata and Apical Dendrites across Layers of Primary Sensory Cortex.

Detailed methods are provided in the online version of this paper and include the following: CONTACT FOR REAGENT AND RESOURCE SHARING Further information and requests for resources and reagents should be directed to and will be fulfilled by the Lead Contact, Randy M. Bruno ( randybruno@columbia.edu ).

EXPERIMENTAL MODEL AND SUBJECT DETAILS

To label specific cortical layers, we utilized several mouse transgenic Cre lines: Rbp4-Cre (GENSAT), Cux2-Cre ( Franco et al., 2012 ) (MMRC), and Nr5a1-Cre (Jackson Laboratories). All mice were > 8 wks old and bred as F1 hybrids on a C57B6/129svev background. Both male and females were used. Animals were group housed without enrichment. All procedures were approved by the Institutional Animal Care & Use Committee at Columbia University.

METHOD DETAILS Behavior

Behavioral experiments were performed with the Arduino-based OpenMaze open-source behavioral system, whose designs are fully described at www.openmaze.org . Prior to training on the tactile detection task, mice were trained to press a lever for a water reward for one week prior to headpost implantation and > 1 week while head-fixed. Lever-trained mice were then either injected with virus (below) or trained to a criterion of 70% correct responses on the tactile detection task before injection. Two weeks after injection, animals were implanted with a chronic cranial window and imaged during the detection task. The tactile detection task requires that the mouse hold a lever down for > 1 s to initiate a trial, in which a stepper motor moved a small pole (2.15-mm diameter, ~3–4-cm long wooden applicator stick), which started from a position 3–4 cm below the animal. The stepper motor rotated the pole to ~2 mm anterior of the nose and ~10 mm lateral of the nose (pole/Go trials) or in the opposite direction even further away from the whiskers (catch/NoGo trials). Whiskers were not hit passively by the pole during stimulus movement, except in a minority (a few percent) of trials where mice whisked or held their whiskers in a protracted position during pole presentation. Typically, mice initiated whisking after they heard the motor begin to move at the start of the trial. During each trial, the mouse had to lift its paw from the lever within 3 s to indicate the presence of the pole, or keep the lever depressed if the pole was absent. Correct pole/Go trials (“Hits”) were rewarded with a small droplet of water (~8 μl) from a water port, whereas incorrect lever lifts during the catch/NoGo trials (“false alarms”) were punished by an 8–10 s timeout before another trial could be initiated as well as a white noise sound. Imaging during the detection task was from animals that had achieved > 70% correct responses during a previous session. Animals performed 100–200 trials during a behavioral session, which typically lasted 20–30 minutes. Licks were detected with a capacitance-based touch sensor (Sparkfun). Whisking was monitored with a high-speed imaging camera (at 300 fps with a PhotonFocus CCD camera or at 187 fps with Sony PS3eye camera) and automatically measured offline using published software ( Clack et al., 2012 ). For experiments aimed at dissecting the effect of reward on apical tuft dendrites, random rewards were given during the inter-trial interval with 2%−5% probability, and pole/Go trial reward administrations were delayed by 0, 250, or 500 ms randomly each trial (0, 250 ms for Cux2-Cre mice). The overall reward rate, which varied across sessions and mice, was approximately 5–10 rewards/min for the pole task and 10–15 rewards/min for the lever task.

Surgery

Animals were anesthetized with isoflurane and implanted with a light-weight stainless steel headpost embedded in dental acrylic affixed to the mouse’s skull after application of a thin layer of Vetbond (3M). Mice recovered for one week before habituation to head fixation. For virus injections, mice were anesthetized with isoflurane and injected with adeno-associated virus (serotypes 1 or 9) encoding the fluorescent calcium indicator GCaMP6f in a Cre recombinase-specific manner. The human synapsin promoter (AAV-hSyn-FLEX-GCaMP6f; Penn Vector Core, GECI consortium) was used for Rbp4 and Cux2 lines, and the CAG promoter for the Nr5a1 line (titers ~2×10 13 cfu/mL). 100 nL of virus was injected at 1:2–8 dilution in ACSF using a pulled pipette (20–30 μm ID) at a depth appropriate for the cortical layer of interest (L5: 800 μm, L2/3: 200 μm, L4: 500 μm), 1.6 mm posterior to bregma and 3.2 mm lateral of the midline. Two-photon imaging was performed ~2 weeks after viral infection. For cranial window implantation (a few days prior to imaging), animals were injected with dexamethasone 1 h prior to surgery, at which time they were anesthetized with isoflurane. A 3-mm hole was drilled in the skull overlying the barrel cortex, and the dura removed from the region of the craniotomy. A 3-mm glass coverslip was inserted into the craniotomy and cemented into place with Vetbond.

Imaging

Cortical regions corresponding to particular whiskers were identified using intrinsic optical signal imaging. Single whiskers in isoflurane anesthetized mice were stimulated at 5 Hz using a piezoelectric bimorph while recording the reflectance of 700-nm incandescent light with a Rolera CCD camera (QImaging) using software custom-written in Labview (National Instruments). Two-photon imaging was performed using a Sutter movable objective microscope under the control of the ScanImage software package (V. Iyer, Janelia Farms). Scanning was performed at 4 fps using a Chameleon Ultra II laser (Coherent), tuned to 940 nm, and focused through a 16×/0.8NA water immersion lens (Nikon). Emitted light was collected with an HQ535/50 filter (Chroma) and GaAsP photomultiplier tubes (Hamamatsu). Images were acquired at a resolution of 128 × 128 or 256 × 256 pixels. Apical tuft dendrites in layer 1 were imaged at depths of 40–80 μm from the pial surface, and Layer 2/3 and 4 somata were imaged at 200–300 μm and 400–600 μm, respectively.

QUANTIFICATION AND STATISTICAL ANALYSIS

Movies were motion corrected using the SIMA image processing package ( Kaifosh et al., 2014 ). Spatial and temporal components for individual dendrites were extracted using large-scale sparse non-negative matrix factorization ( Pnevmatikakis et al., 2016 ). This method inherently corrects for background signal. Data were analyzed using custom-written routines implemented in MATLAB. Comparisons of frame-averaged calcium signals were performed based upon the average peak amplitude from each session for the time period of 2 s following each behavioral event type. Whisker angle was computed over 150-ms windows and isolated whisking bouts were classified as whisker angle change greater than 2 standard deviations above the mean, with a 1 s lockout. Random reward responsive dendrites were determined based upon proportions of calcium events following random inter-trial interval rewards versus calcium events during inter-trial intervals without random rewards. Dendrites were categorized as reward tracking if the slope of the latency of the second peak in calcium response after the lever lift was significantly related to reward delay time (linear regression). Proportions were compared using a normal approximation to a binomial distribution, and means were compared using t tests.

DATA AND SOFTWARE AVAILABILITY

Data and software are available upon request to the Lead Contact.

EXPERIMENTAL MODEL AND SUBJECT DETAILS

To label specific cortical layers, we utilized several mouse transgenic Cre lines: Rbp4-Cre (GENSAT), Cux2-Cre ( Franco et al., 2012 ) (MMRC), and Nr5a1-Cre (Jackson Laboratories). All mice were > 8 wks old and bred as F1 hybrids on a C57B6/129svev background. Both male and females were used. Animals were group housed without enrichment. All procedures were approved by the Institutional Animal Care & Use Committee at Columbia University.

METHOD DETAILS Behavior

Behavioral experiments were performed with the Arduino-based OpenMaze open-source behavioral system, whose designs are fully described at www.openmaze.org . Prior to training on the tactile detection task, mice were trained to press a lever for a water reward for one week prior to headpost implantation and > 1 week while head-fixed. Lever-trained mice were then either injected with virus (below) or trained to a criterion of 70% correct responses on the tactile detection task before injection. Two weeks after injection, animals were implanted with a chronic cranial window and imaged during the detection task. The tactile detection task requires that the mouse hold a lever down for > 1 s to initiate a trial, in which a stepper motor moved a small pole (2.15-mm diameter, ~3–4-cm long wooden applicator stick), which started from a position 3–4 cm below the animal. The stepper motor rotated the pole to ~2 mm anterior of the nose and ~10 mm lateral of the nose (pole/Go trials) or in the opposite direction even further away from the whiskers (catch/NoGo trials). Whiskers were not hit passively by the pole during stimulus movement, except in a minority (a few percent) of trials where mice whisked or held their whiskers in a protracted position during pole presentation. Typically, mice initiated whisking after they heard the motor begin to move at the start of the trial. During each trial, the mouse had to lift its paw from the lever within 3 s to indicate the presence of the pole, or keep the lever depressed if the pole was absent. Correct pole/Go trials (“Hits”) were rewarded with a small droplet of water (~8 μl) from a water port, whereas incorrect lever lifts during the catch/NoGo trials (“false alarms”) were punished by an 8–10 s timeout before another trial could be initiated as well as a white noise sound. Imaging during the detection task was from animals that had achieved > 70% correct responses during a previous session. Animals performed 100–200 trials during a behavioral session, which typically lasted 20–30 minutes. Licks were detected with a capacitance-based touch sensor (Sparkfun). Whisking was monitored with a high-speed imaging camera (at 300 fps with a PhotonFocus CCD camera or at 187 fps with Sony PS3eye camera) and automatically measured offline using published software ( Clack et al., 2012 ). For experiments aimed at dissecting the effect of reward on apical tuft dendrites, random rewards were given during the inter-trial interval with 2%−5% probability, and pole/Go trial reward administrations were delayed by 0, 250, or 500 ms randomly each trial (0, 250 ms for Cux2-Cre mice). The overall reward rate, which varied across sessions and mice, was approximately 5–10 rewards/min for the pole task and 10–15 rewards/min for the lever task.

Surgery

Animals were anesthetized with isoflurane and implanted with a light-weight stainless steel headpost embedded in dental acrylic affixed to the mouse’s skull after application of a thin layer of Vetbond (3M). Mice recovered for one week before habituation to head fixation. For virus injections, mice were anesthetized with isoflurane and injected with adeno-associated virus (serotypes 1 or 9) encoding the fluorescent calcium indicator GCaMP6f in a Cre recombinase-specific manner. The human synapsin promoter (AAV-hSyn-FLEX-GCaMP6f; Penn Vector Core, GECI consortium) was used for Rbp4 and Cux2 lines, and the CAG promoter for the Nr5a1 line (titers ~2×10 13 cfu/mL). 100 nL of virus was injected at 1:2–8 dilution in ACSF using a pulled pipette (20–30 μm ID) at a depth appropriate for the cortical layer of interest (L5: 800 μm, L2/3: 200 μm, L4: 500 μm), 1.6 mm posterior to bregma and 3.2 mm lateral of the midline. Two-photon imaging was performed ~2 weeks after viral infection. For cranial window implantation (a few days prior to imaging), animals were injected with dexamethasone 1 h prior to surgery, at which time they were anesthetized with isoflurane. A 3-mm hole was drilled in the skull overlying the barrel cortex, and the dura removed from the region of the craniotomy. A 3-mm glass coverslip was inserted into the craniotomy and cemented into place with Vetbond.

Imaging

Cortical regions corresponding to particular whiskers were identified using intrinsic optical signal imaging. Single whiskers in isoflurane anesthetized mice were stimulated at 5 Hz using a piezoelectric bimorph while recording the reflectance of 700-nm incandescent light with a Rolera CCD camera (QImaging) using software custom-written in Labview (National Instruments). Two-photon imaging was performed using a Sutter movable objective microscope under the control of the ScanImage software package (V. Iyer, Janelia Farms). Scanning was performed at 4 fps using a Chameleon Ultra II laser (Coherent), tuned to 940 nm, and focused through a 16×/0.8NA water immersion lens (Nikon). Emitted light was collected with an HQ535/50 filter (Chroma) and GaAsP photomultiplier tubes (Hamamatsu). Images were acquired at a resolution of 128 × 128 or 256 × 256 pixels. Apical tuft dendrites in layer 1 were imaged at depths of 40–80 μm from the pial surface, and Layer 2/3 and 4 somata were imaged at 200–300 μm and 400–600 μm, respectively.

Supplementary Material 1 2