Avoiding Bias in Measurements of Fundamental Constants from High Resolution Quasar Spectra ^†

Abstract

Recent advances in spectroscopic instrumentation and calibration methods dramatically improve the quality of quasar spectra. Supercomputer calculations show that, at high spectral resolution, procedures used in some previous analyses of spacetime variations of fundamental constants are likely to generate spurious measurements, biased systematically towards a null result. Developments in analysis methods are also summarised and a prescription given for the analysis of new and forthcoming data.

1. Introduction

High resolution spectra of distant quasars reveal numerous narrow absorption lines caused by gaseous components of galaxies intersecting the Earth–quasar sightline. The large number of atomic species and transitions detected allow precise measurements of the fine structure constant ${\alpha }_{SI}=e^2/4\pi {ϵ}_0\hslash c$ over cosmological distances, where e is the electron charge, ${ϵ}_0$ the vacuum permittivity, ℏ the reduced Planck constant, and c the speed of light. The dimensionless $\alpha$ is the ratio of the speed of an electron in the lowest energy orbit of the Bohr-Sommerfeld atom to the speed of light, and hence connects quantum mechanics (through ℏ) with electromagnetism (through the remaining quantities).

The 1999 invention of the Many Multiplet method (MM) [1,2] created an order of magnitude precision gain over previous methods in searches for spacetime variations of $\alpha$. The previously used method had been the Alkali Doublet method (AD), in which measurements were made relative to the same ground state, by-passing the invaluable sensitivity of the ground state to any change in $\alpha$. Further, the excited levels in the ${}^2{P}_{3/2}$–${}^2{P}_{1/2}$ AD structure itself generally have lower sensitivity compared to the commonly observed singly ionised multiplets of Fe, Zn, Cr, and others. In contrast, the MM method takes into account ground-state shifts. Also, excited state relativistic corrections in multiplets can be large; s–p and s–d transitions for example may even be of the opposite sign. Any real change in $\alpha$ therefore generate a unique pattern of observed wavelength shifts that is not degenerate with a simple cosmological redshift. The MM method thus produces sensitive results when applied to multiplets of the same atomic species, or to species having widely differing atomic masses.

New and forthcoming scientific facilities will intensify searches for spacetime variations of fundamental constants. In particular, the Echelle SPectrograph for Rocky Exoplanets and Stable Spectroscopic Observations (ESPRESSO) on the European Southern Observatory’s Very Large Telescope (VLT) [3] and the ArmazoNes high Dispersion Echelle Spectrograph (ANDES, formally named HIRES) on the forthcoming Extremely Large Telescope (ELT) e.g., [4,5] will play important roles in this field of research. As the data quality and quantity increase, it becomes all-important to ensure that analysis techniques produce fully unbiased and optimal estimates. Here we summarise recent methodological advances facilitating these things and to scrutinise assumptions made and procedures used in previously published analyses that can produce bias. The remainder of this paper is composed as follows: Section 2 and Section 3 summarise several recent advances in the varying $\alpha$ field. Section 4 presents some new calculations showing that “blinding” methods, as used in some published measurements, generate measurement bias. Section 5 gives a précis of the do’s and don’ts when analysing high quality absorption spectra.

2. Wavelength Calibration

Measuring $\alpha$ requires the wavelength scale of the astronomical spectrum to be established with high fidelity. For example, a change of $\Delta \alpha /\alpha \equiv ({\alpha }_z-{\alpha }_0)/{\alpha }_0=1\times {10}^{-6}$ gives rise to a relative shift in the wavelength of Fe ii 2383 of approximately $\Delta \lambda /\lambda =1\times {10}^{-7}$. For echelle spectrographs such as UVES on the VLT, the standard wavelength reference is established by imaging the spectrum of a ThAr arc lamp. A comparison between ThAr wavelength calibrated data and the solar spectrum (as first done by [6]), revealed the presence of long-range wavelength scale distortions with amplitudes as large as $\Delta \lambda /\lambda =1\times {10}^{-5}$. It was initially thought that distortions this large could potentially spoil fundamental constant measurements [7,8], although subsequent analyses showed that such distortions can be modelled and associated uncertainties on $\Delta \alpha /\alpha$ allowed for (see [9] and Appendix B9 of [10]).

The new generation of astronomical spectrographs aims to avoid the long-range distortions seen in some ThAr calibrated spectra altogether. Laser Frequency Comb technology (LFC, [11,12,13]) or Fabry-Perót etalons (FP) combined with ThAr can provide wavelength calibration with accuracy of the order $\Delta \lambda /\lambda =1\times {10}^{-8}$ [14,15,16]. Both of these advanced calibration sources are installed on the High Accuracy Radial-velocity Planet Searcher (HARPS, [17]) and ESPRESSO. One current difficulty is that none of the currently available astronomical LFCs provide wavelength calibration below 5000 Å. Wavelengths below this cut-off are currently calibrated by simultaneously imaging both ThAr and FP spectra. The ThAr + FP combination provides lower accuracy than LFCs. Since the spectral region below 5000 Å is generally very important for $\Delta \alpha /\alpha$ measurements and since, this important problem is yet to be solved, raising significant problems not only for varying constant measurements but also for redshift drift projects.

Another important concern that must be addressed in future high-precision spectroscopy is that using an LFC or ThAr+FP may not be sufficient to remove all systematic effects related to wavelength calibration. A comparison of two independent LFCs used simultaneously on HARPS revealed an unexpected offset in the zero-points of their wavelength calibrations of $\Delta \lambda /\lambda =1.5\times {10}^{-9}$ [14,15]. Although this is a very small effect, combining observations calibrated using either two different LFCs or a single LFC that has been modified in some way between two observations should be carefully performed.

3. Absorption Profile Modelling

3.1. VPFIT and AI-VPFIT

vpfit [18] is a non-linear least squares code for modelling high resolution absorption spectra that has been developed over a number of years [18,19] and it forms the core of our procedures. The theoretical background on which vpfit is based, plus some recent enhancements, are described in [10,20]. Throughout vpfit’s development, considerable effort has gone into ensuring high precision internal calculations. It has a comprehensive online user guide, updated frequently [21]. The ever-increasing quality of high resolution spectroscopic data requires extremely careful treatment of every aspect of profile calculation and fitting. Some of the studies carried out in this context are:

• The Voigt profile 2-level atom approximation is good enough, even at very high spectral signal to noise, for non-damped column densities i.e., $logN\le 20$ [22].
• Voigt function $H(a,u)$ look-up tables must be sufficiently high resolution in the relevant parameter (u) to render non-linear effects negligible. Interpolation within those tables must also be sufficiently precise [10].
• It is most important to allow for non-linearity in the Voigt profile shape by computing model Voigt profiles in extremely fine bins [19].
• As far as possible, any blends/interlopers must be allowed for, whether they arise in identified species or not. Failure to do can significantly increase the measurement error and bias individual measurements [23].
• Fitting region selection is important. If line wings/continuum regions are truncated, best-fit models for $\Delta \alpha /\alpha$ exhibit an unnecessarily large scatter [24].
• Voigt function derivatives must be accurate [10,20]. The Hessian is derived from the derivatives of $H(a,u)$, the inverse of which provides parameter uncertainties. The Hessian, of course, determines parameter search directions.
• How to select a “final” absorption system model, given non-uniqueness and alternative information criteria? [25].
• Absorption line broadening: models must include temperature as a free parameter and should not be assumed to be turbulent [26,27].
• Using the correct instrumental profile for model calculations is important [14].
• All spectral regions used in the measurement must be carefully checked for potential contaminating atmospheric features [28].

Using vpfit involves human decision making and the final results obtained can depend on that human input [25]. Whilst this has negligible consequences for many applications, measurements of fundamental constants at high redshift push the limits of the data and are more susceptible to small systematic errors or biases than other less challenging measurements. Measurements of $\Delta \alpha /\alpha$ thus motivate full automation in which all human input is avoided and all potential bias removed. Artificial Intelligence methods were first applied to this problem in [29,30] (gvpfit) and significantly extended in [23] (ai-vpfit). The advantages in avoiding human decision making are achieving objectivity, reproducibility, and the ability to explore multiple models to the same absorption system. These advantages turn out to be crucial for measurements of fundamental constants. The calculations reported in the following sections make use of both vpfit and ai-vpfit.

3.2. Information Criterion or ${\chi }_{\nu }^2$ to Select Models?

In selecting a single best-fit model for an absorption complex, many previous measurements have made use of a simple normalised ${\chi }^2$ approach, accepting a model once ${\chi }_{\nu }^2$ is “sufficiently” close to unity,

$${\chi }_{\nu }^2=\frac{1}{\nu }\sum _{i=1}^{n_d}{\left(\frac{d_i-f_i}{{\sigma }_i^2}\right)}^2$$

(1)

where $d_i$ is the spectral data array, $f_i$ is the model, ${\sigma }_i$ is the spectral error array, $n_d$ is the number of data points, and the number of degrees of freedom $\nu =n_d-n_p$, where $n_p$ is the total number of free parameters in the model. Alternatively, one can use an information criterion (IC) to select models, the general form of which is

$$\mathrm{IC}={\chi }^2+\mathcal{P}(n_p,n_d)$$

(2)

where ${\chi }^2=\nu {\chi }_{\nu }^2$ and $\mathcal{P}(n_p,n_d)$ is a penalty factor that increases with increasing number of model parameters. An IC applies a “principle of parsimony”, balancing parameter variance with model bias. The application of ICs in astrophysics has been discussed by [31,32] and a comprehensive treatise is given in [33]. Problems associated with noise characteristics in calculating ICs are discussed in [34]. Employing an IC allows an optimal number of model parameters to be identified in an objective and reproducible way [35].

Using an asymptotic ${\chi }_{\nu }^2$ as a means of deciding how many model parameters to use requires the user to choose a maximum acceptable value for ${\chi }_{\nu }^2$. This in itself is not particularly disadvantageous because one can formalise the problem through the relationship between ${\chi }_{\nu }^2$ and its probability distribution, hence defining an acceptance probability rather than a numerical value of ${\chi }_{\nu }^2$ for model selection. However, in practice it is generally difficult to obtain ${\sigma }_i$ accurately, so ${\chi }^2$ itself is only approximate. The difficulties in accurately estimating the spectral error array are well known and have been discussed in the section titled Modifying the error arrays in the rdgen user guide [18].

For the reasons outlined above, in the context of $\Delta \alpha /\alpha$ at least, it is preferable to use an IC for model selection rather than ${\chi }_{\nu }^2$. However, using an IC raises a question: what is the optimal form of the penalty term $\mathcal{P}(n_p,n_d)$ in Equation (2)? This point has been studied in [35], where three ICs are compared: the corrected Akaike Information Criterion AICc, the Bayesian Information Criterion BIC, and the Spectral Information Criterion SpIC. The latter is a new IC designed specifically for spectroscopy. All three ICs (there are others) perform in slightly different ways; AICc tends to over-fit the data, allowing too many free parameters, whilst the converse is true for BIC. SpIC appears to fall in the Goldilocks zone.

3.3. Contributions to the $\Delta \alpha /\alpha$ Error Budget

Fully understanding all potential contributions to the error budget, random and systematic, is of course a crucial aspect of assessing the reality of any deviation of $\Delta \alpha /\alpha$ from zero. Uncertainties associated with varying $\alpha$ measurements have been discussed in numerous papers, e.g., [36,37]. Since our understanding of uncertainties has significantly improved since those studies, we list here possible sources of error and their attribution:

Inherent uncertainties:

1.

Statistical error i.e., VPFIT covariance matrix error.

2.

If turbulent broadening is used to model the system and if the true intrinsic broadening is compound (see Equation (3)), a systematic error is introduced.

3.

Absorption system model non-uniqueness error.

4.

Continuum estimate error.

Errors from astrophysical factors:

5.

Isotopic relative abundances.

Errors associated with theoretical and experimental uncertainties:

6.

Q coefficient uncertainties (these bias towards $\Delta \alpha /\alpha =0$).

7.

Oscillator strength uncertainties.

8.

Laboratory wavelength uncertainties.

Errors associated with data extraction or instrumental factors:

9.

Wavelength calibration error (for pre-LFC/FP data).

10.

Bad pixels.

11.

Flat-fielding errors.

12.

Weak cosmic rays removal.

13.

Significant point-spread function variations across the detector.

3.4. Future Measurements Require Monte Carlo AI

The recent application of AI methods (ai-vpfit) provides full automation of modelling quasar absorption systems [23,30]. Automation has allowed us to explore in detail how final best-fit models depend on the construction sequence and hence the extent to which model non-uniqueness contributes to the total $\Delta \alpha /\alpha$ error budget. Preliminary studies of this sort have been made recently and so far indicate that indeed $\Delta \alpha /\alpha$ measurements are impacted by the sequence in which models are developed. The inference is that, for any particular absorption system, it is difficult to properly assess the overall uncertainty on $\Delta \alpha /\alpha$unless multiple models are produced, each one constructed differently (emulating the different approaches that would be taken by different human modellers). The contribution of non-uniqueness to the overall $\Delta \alpha /\alpha$ uncertainty budget can only be determined on a case by case basis because we now know that the degree of non-uniqueness differs from one absorption system to another [25]. In other words, it is necessary to form many models of each particular absorption system in order to quantify the intrinsic non-uniqueness behaviour and hence determine the overall uncertainty on $\Delta \alpha /\alpha$.

4. Spectral Simulations and Distortion-Blinding

The methods used to measure $\Delta \alpha /\alpha$ must be entirely free of any bias. To that end, a “blinding” method has been used in several published measurements [38,39,40,41], employing the following steps. During the initial data reduction stages, long-range and intra-order distortions of the wavelength scale are applied to individual exposures. The individual distortions are designed to leave a non-zero $\Delta \alpha /\alpha$ in the final co-added spectrum. vpfit is then used on that distorted spectrum, fixing $\Delta \alpha /\alpha$ to be zero. Once the final model has been obtained in this way, one final tweak of the model is carried out by fitting a non-distorted version of the spectrum, allowing the existing model parameters plus $\Delta \alpha /\alpha$ to vary freely. However, fixing $\Delta \alpha /\alpha =0$ corresponds to requiring that all rest-frame wavelengths involved in the initial absorption system modelling are precisely those measured in terrestrial laboratories. If the true value of $\Delta \alpha /\alpha$ is zero, the method just described should indeed be unbiased. However, if the true $\Delta \alpha /\alpha$ is not zero (and it is explicitly non-zero in the “blinded” data of [38,39,40,41]), forcing $\Delta \alpha /\alpha$ to be initially zero necessarily produces a flawed model. The final step of “switching on” $\Delta \alpha /\alpha$ as a free parameter may or may not subsequently be able to correct that flawed model. If it does not, the result is a systematically biased measurement. Notwithstanding the previous comments, it should be noted that the spectral resolution of the simulations in the analyses in this paper are far higher than in e.g., [38,39,40]. At lower spectral resolution, fewer components are detected. This could mean that the level of bias is diluted. Further calculations are needed to answer this point. Irrespective of this, what is instead needed is a method that is fully unbiased, no matter what the true value of $\Delta \alpha /\alpha$ is. We next describe some preliminary simple spectral simulations to illustrate the concerns just expressed. To distinguish the sort of blinding described above from other potential blinding methods, we will refer to the method described above as “distortion-blinding”.

4.1. Preliminary Illustration of How Distortion-Blinding + Turbulent Line Broadening Creates Bias

A simulated spectrum of a single component absorption system is created, with seven transitions: Mg ii 2796, 2803, Fe ii 2344, 2374, 2383, 2586, 2600 Å. A spectral resolution of $2.10$ km/s FWHM is used for the three transitions having ${\lambda }_{obs}<5120$ Å and $2.05$ km/s FWHM for the four transitions having ${\lambda }_{obs}>5540$ Å. The pixel size and signal to noise per pixel were 0.4 km/s and 75. The spectral parameters correspond to those of the ESPRESSO spectrum in [41]. The absorption line parameter values used are similar to one component from the $z_{abs}=1.15$ system towards the quasar HE0515–4414. The upper panel of

Table 1. Results obtained using vpfit on a single component simulation. See Section 4.1 for details. $v=(z-z_0)c/(1+z_0)$, where $z_0=1.1469691$ and c is the speed of light. In the lower panel, one of the Fe ii components fell below the vpfit detection limit, indicated in the $logN$ column by “-”. The negative velocities (v) for the turbulent case are a consequence of forcing $\Delta \alpha /\alpha =0$ (the true value is non-zero).

	Species	$log\mathit{N}$	${\mathit{b}}_{\mathbf{turb}}$ (km/s)	T (${10}^4$ K)	v (km/s)	$\mathbf{\Delta }\mathit{\alpha }/\mathit{\alpha }$ (${10}^{-5}$)
Generating	Mg ii	12.500	3.87	5.31	0.000	10.0
model	Fe ii	12.000
Fitting	Mg ii	12.499 (0.002)	3.70	5.53	0.000	10.1
(compound)	Fe ii	12.000 (0.007)	(0.27)	(0.34)	(0.028)	(0.4)
	Mg ii	12.493 (0.017)	7.02	-	−0.279
Fitting	Fe ii	-	(0.05)		(0.084)	-
(turbulent)	Mg ii	10.682 (1.099)	5.44	-	−2.152
	Fe ii	12.001 (0.007)	(0.10)		(0.070)

(labelled “Generating model”) shows the actual absorption line parameters used to generate the simulated spectrum. For the purposes of this preliminary illustration, we use an extreme value of $\Delta \alpha /\alpha ={10}^{-4}$, so that line shifts caused by the non-zero $\Delta \alpha /\alpha$ are easily visible in the plotted data. A subset of the simulated spectral lines is illustrated in Figure 1. As Table 1 shows, the simulated spectrum is generated using compound broadening i.e., line broadening comes from both turbulent and thermal contributions,

$$b_{obs}^2=b_{\mathrm{turb}}^2+\frac{2kT}{m}$$

(3)

Having created the data, for the purposes of this illustration we first fit all lines simultaneously with vpfit, using compound line broadening. $\Delta \alpha /\alpha$ is a free parameter throughout. No distortion-blinding is used. The second panel of Table 1 (labelled “Fitting (compound)”) illustrates the measured parameter values and their uncertainties, showing, as expected, that only a single absorption component is required and that all parameters returned are consistent with the input values.

We now reach the point of this illustration. The seven transitions are again fitted simultaneously using vpfit, but this time with two important differences: the data are fitted using a fixed $\Delta \alpha /\alpha =0$ and using turbulent line broadening, i.e., taking the limiting case of $T=0$. The numerical results are given in the third (lower) panel of Table 1 (labelled “Fitting (turbulent)”) and the resulting model is illustrated in Figure 2. This fit now requires two absorption components (one of the Fe ii components is below the column density threshold so tick marks are not shown). Figure 2 and Table 1 reveal that adopting a turbulent model creates the necessity for an additional (fake) velocity component and because of the distortion-blinding, that fake velocity component is required to correspond perfectly with (redshifted) terrestrial laboratory wavelengths.

The purpose of this simple simulation is purely to demonstrate how easy it is to derive a strongly biased result if distortion-blinding + turbulent broadening are used; the normalised residuals are plotted above each transition in Figure 2. The residuals show that the turbulent distortion-blinded fit forcing $\Delta \alpha /\alpha =0$ provides a good fit, even though it is far from zero in the data. The normalised ${\chi }^2$ for both fits quantify that: the non-blinded compound model produces an overall fit of ${\chi }_{\nu }^2=1.0250$ for 1394 degrees of freedom and AICc = 1440.882. The distortion-blinded $\Delta \alpha /\alpha =0$ turbulent model produces ${\chi }_{\nu }^2=1.0291$ for 1393 degrees of freedom and AICc = 1447.581. Both normalised ${\chi }^2$ values are acceptable. In Section 4.2 we examine this problem in more detail.

4.2. Detailed Calculations Using ai-vpfit

Section 4.1 illustrates that distortion-blinding + turbulent modelling has the capacity to significantly bias $\Delta \alpha /\alpha$ measurements towards zero. In this section we show that distortion-blinding + turbulent modelling does not merely have the capacity to bias, but that it inevitably does so.

We first generate a simple synthetic spectrum. The absorption line parameters used were extracted from measurements (by us) of a real absorption system at $z_{abs}=1.15$ towards the publicly available ESPRESSO spectrum of the quasar HE0515–4414 [41]. A small section of the system was used, rather than the entire system, to make calculating time shorter and because a small section serves the required purpose. The spectral characteristics are the same as the simulated spectrum described in Section 4.1. The absorption line parameters used to generate the synthetic spectrum are given in

Table 2. Absorption line parameters used to generate the spectrum illustrated in Figure 3. See Section 4.2.

log N(Mg ii)	log N(Fe ii)	${\mathit{b}}_{\mathbf{turb}}$ (km/s)	T (${10}^4$ K)	${\mathit{z}}_{\mathbf{abs}}$
12.66	12.20	1.49	1.64	1.1469691
12.61	12.16	7.19	4.06	1.1469952

. The synthetic spectrum was created using $\Delta \alpha /\alpha =8.08\times {10}^{-6}$, a value measured by us, without applying any kind of blinding, using the full absorption system (i.e., not a small section). The same seven transitions were generated as above.

As Table 2 shows, the line broadening used to generate the absorption simulation is compound i.e., each redshift has its own unique thermal and turbulent contribution. Many previous detailed analyses show that individual quasar absorption components exhibit compound and not turbulent broadening e.g., [27], so our synthetic spectrum emulates real data in this respect. The next stage of our analysis is to model the synthetic data using the same procedures that have been applied to real data in e.g., [41]. Models are fitted using turbulent line broadening and $\Delta \alpha /\alpha$ is fixed to zero i.e., we carry out the same distortion-blinding approach used in some previous works. We then apply ai-vpfit [23] multiple times, each time deriving an independently constructed best fit. AICc is used to select the best-fit model during each ai-vpfit calculation. In each case, after the best-fit model is established, one final iteration using vpfit is done, adding $\Delta \alpha /\alpha$ as an additional free parameter.

Figure 4 gives histograms for four sets of 100 ai-vpfit calculations (one for each combination of turbulent, compound, AICc, and SpIC models), revealing remarkable effects. The panel on the left demonstrates that using turbulent broadening with AICc generates a distribution that is clearly non-Gaussian, with severe bias towards $\Delta \alpha /\alpha =0$. No bias is seen for AICc/compound and the distribution looks well-behaved. The panel on the right shows results obtained using SpIC. In this case, significant bias is again seen if turbulent broadening is used, but not for compound broadening. The SpIC/compound $\Delta \alpha /\alpha$ results are more reliably determined than those for AICc/compound (compare the spreads of the compound points for AICc and SpIC results). The bin size in the histograms above is $1.25\times {10}^{-6}$ and the 1$\sigma$ statistical uncertainty for a typical compound broadened model with 2 components is $2.13\times {10}^{-6}$. Figure 4 shows that the combination of distortion-blinding + turbulent broadening is particularly damaging to the final result.

Figure 5 is a visual breakdown of the number of redshift components needed to fit the spectral lines illustrated in Figure 3, for the four different combinations of AICc, SpIC, turbulent, and compound. Light blue bars correspond to metals (i.e., Mg ii and Fe ii in this case). Remembering that the true number of components is 2, we see that the only combination with 100% success is SpIC/compound, with AICc/compound almost as good. The turbulent results badly overfit the data, both AICc and SpIC giving most probable numbers of 4. The yellow bars correspond to interlopers i.e., additional absorption components required by ai-vpfit to achieve an acceptable fit, but which were not identified as metals. No interlopers were present in the simulated data being fitted so these are weak features caused by chance correlated noise patterns that have no detectable impact (in terms of systematic bias) on the $\Delta \alpha /\alpha$ estimate.

Figure 6 plots the 100 individual ai-vpfit $\Delta \alpha /\alpha$ measurements. The distortion-blinded turbulent points (blue) are clearly heavily biased towards zero. Further, the majority of those points have far larger error bars (because of the overfitting and hence artificially increased blending) than a small subset closer to the correct value of $\Delta \alpha /\alpha =8.08\times {10}^{-6}$. The compound points (not distortion-blinded, red) have substantially smaller error bars (no overfitting) and do not suffer bias. Interestingly, the blue points clearly exhibit model non-uniqueness whereas the red ones do not.

4.3. A Retrospective on Distortion-Blinding

Distortion-blinding was an attempt at taking a conservative approach to the subject. It is only because we have developed AI Monte-Carlo methods that we can now identify the adverse impact of interactive modellers (each probably taking a slightly different approach) using distortion-blinding + turbulent broadening. ai-vpfit shows us how to avoid bias in measurements using high quality data from facilities like ESPRESSO on the VLT and, in the future, HIRES on the ELT. In fact, in the absence of distortion-blinding, any reasonable analysis is effectively inherently blind anyway. If the fitting is done “manually” (i.e., using vpfit), the process contains so many steps that it is inconceivable for a human to subconsciously construct a biased model unless the final measured value of $\Delta \alpha /\alpha$ is influenced directly by iteratively adjusting the model or adding new absorption components until a “preferred” value of $\Delta \alpha /\alpha$ is obtained. Given that analyses involving distortion-blinding have been carried out by different people at different times, the degree to which this has or has not happened in previously published measurements cannot be recovered.

Looking to future measurements, no tampering with the wavelength scale needs to be or should be done. If a fit is done using vpfit, a far simpler (and unbiased) method of “blinding” would be to switch off all output to the user containing any information about $\Delta \alpha /\alpha$ during the entire model construction process. $\Delta \alpha /\alpha$ must be a free parameter from the outset i.e., as soon as two or more transitions or species with differing sensitivities to $\alpha$ are incorporated—see [23] for a discussion on this important point. Switching off $\Delta \alpha /\alpha$ output naturally renders any conscious or subconscious steering by the user impossible.

Analyses using the ai-vpfit method, as described in [23], are automatically blinded. There is no interactive input during the modelling procedure.

5. Discussion

The calculations described in this work and accompanying discussions allow us to define some general requirements for spacetime measurements of varying fundamental constants:

• When modelling/solving for $\Delta \alpha /\alpha$, if fitting multiple species simultaneously, turbulent broadening should not be used. Instead only compound broadening should be used. If compound broadening is problematic because T is not sufficiently well constrained, either the measurement should be discarded, or possibly one could adopt a representative T as a fixed parameter, although in this case the uncertainty on $\Delta \alpha /\alpha$ may be artificially lowered. We have not investigated such an approach in this paper.
• Distortion-blinding biases results and should be avoided. Deliberately distorting a spectrum and then solving for its velocity structure with $\Delta \alpha /\alpha$ forced to be zero is most likely to create a result that is biased towards $\Delta \alpha /\alpha =0$. Previous $\Delta \alpha /\alpha$ measurements carried out in this way should be repeated.
• Some absorption systems suffer from model non-uniqueness. Therefore it is desirable to model each absorption system multiple times to quantify this.
• In obtaining the observational data, wavelength coverage should be done using LFCs or FPs. If observations from different observing runs at different epochs are to be combined/jointly analysed, there should be calibration redundancy i.e., ideally calibrations always done with two LFCs or FPs. Accurate calibration reaching as blue as the atmospheric cutoff is essential to pick up lower rest-wavelength transitions.

An aside on the above is an interesting consequence of turbulent broadening with important implications for abundance measurements. Turbulent modelling generates spurious absorption components and these appear with meaningless relative abundances. The result is that the relative abundances from one absorption component to another appear to be highly (artificially) scattered. This is seen in Section 4.1 and Figure 1 and Figure 2. Let $\Delta =logN$(Mg ii)$-logN$(Fe ii). Table 1 gives $\Delta$(true) $=0.5$, $\Delta$(compound) $=0.5$, $\Delta$(turbulent, $v=-2.152$) $=-1.3$, $\Delta$(turbulent, $v=-0.279$) $\sim 2.5$, where, for the purposes of this illustration, the latter assumes a column density detection threshold of 10.0, so is an order of magnitude estimate. This means that claims for significant relative abundance fluctuations across an absorption complex may be incorrect if turbulent broadening has been assumed. If, instead, summed column densities are used for a complex, no spurious effects would be seen.