e. the expectancy rating E) and the model (i.e. the value V), which is based on the negative log-likelihood (ln L; Lewandowsky & Farrell, 2011) summed over all participants and all trials The RW and the hybrid model were fitted to the data in several variations and the resulting deviances were then compared using likelihood ratio tests.

First, we fitted both models across all subjects and trials, and obtained one single set of parameter estimates. As the speed and accuracy of learning probably relates to each cue’s contingencies and changes in contingencies, we further sought to optimize model fit by fitting both models separately for each condition (resulting in one set of parameters for each contingency condition, i.e. each cue).

Deviances of the condition-wise fitted hybrid model were also compared with the hybrid model that AZD2281 was fitted across conditions. We finally adopted the condition-wise fitted parameters of the hybrid model (fitted across all subjects) for the subsequent imaging analysis, as these provided the closest fit to the behavioural data (see ‘Results’ and Table 1B). Model fitting and see more comparison were additionally performed on an individual level by fitting each of the above-mentioned models to each subject’s behavioural data. Moreover, all models were compared against a baseline model to assure that they outperform a model with random predictions. To estimate the deviance of the baseline model, we randomized model predictions (i.e. the values for V that are compared with the ratings E; see above). As the estimated deviance thus depends on the random selection of values for

V, we repeated this procedure 10 000 times and used the average deviance to compare the baseline model against the learning models (see Tables 1 and 2). Statistical parametric mapping (SPM8, Wellcome Trust Casein kinase 1 Centre for Neuroimaging, London, UK) was used for preprocessing and analysing the imaging data. The first four volumes of each session were discarded to account for T1 equilibrium effects. Functional images were realigned to the first remaining volume and co-registered to individual skull-stripped T1 images. Subsequently, the diffeomorphic image registration algorithm (DARTEL) toolbox was used to create a sample-specific structural template as well as individual flow fields, which were used in turn for spatial normalization of the functional images. Data were smoothed with a 4 mm full-width at half maximum isotropic Gaussian kernel and resampled to a voxel size of 1 × 1 × 1 mm³. A random-effects general linear model analysis was conducted on the fMRI data with separate predictors for each cue [cue A: CS– (acquisition) and new CS50 (reversal); cue B: CS50 (acquisition) and new CS100 (reversal); cue C: CS100 (acquisition) and new CS– (reversal)] at two points in time (CS and potential US onset).