SDMtoolbox 2.0: the next generation Python-based GIS toolkit for landscape genetic, biogeographic and species distribution model analyses

Introduction

SDMtoolbox is a Python-based ArcGIS toolbox for spatial studies of ecology, evolution and genetics. SDMtoolbox consists of a series of Python scripts (79 and growing) designed to automate complicated spatial analysis in ArcMap (ESRI, 2017) and Python. Since SDMtoolbox’s first release, in April 2014 (Brown, 2014), the program has been download over 30,000 times by users in 160 countries (from every sub-continent) and cited over 180 times. Surveying SDMtoolbox’s citations, most users (87%) used this toolkit for the preparation of SDM files, running SDMs and processing of SDM results. Since the first publication (Brown, 2014), SDMtoolbox has been under continuous development and expansion. SDMtoolbox remains a free, comprehensive Python-based toolbox for macroecology, landscape genetic and evolutionary studies to be used with ArcGIS 10.0 (or higher) with a Standard or Advance License and the Spatial Analyst extension. The toolkit simplifies many GIS analyses required for species distribution modelling and other spatial ecological analyses, alleviating the need for repetitive and time-consuming climate data pre-processing and post-SDM analyses.

Methods

SDMtoolbox is written in Python (v2.7) and incorporates Python libraries from ArcPy (ArcGIS 10.0-10.5), NumPy and SciPy that are incorporated into a single toolbox for use by ArcGIS 10.0-10.5 users. One tool, the Run MaxEnt: Spatially Jackknife tool, outputs java code batch file format to run the MaxEnt program.

Results

After the release of SDMtoolbox v1, we updated 28 (of 59) of the original tools to improve user performance and maximize compatibility with newer versions of ArcGIS 10.3-10.5 and the recent open source version of MaxEnt (version 3.4 or higher; Phillips, Dudík & Schapire, 2017). For several tools this required completely recoding the analysis pipeline (Table 1). In addition to updating many tools, we provide 20 new tools, increasing the total tools to 79 (Table 2). The adaption of the code in the Run MaxEnt: Spatially Jackknife tool for MaxEnt 3.4 enables output models in cloglog format and enabling/disabling use of threshold feature class in spatial jackknifing process. What follows are brief overviews of the new tools, organized by major tool category.

Discussion

As previously mentioned, a large proportion of SDMtoolbox v1 users cite using the program for assisting species distribution modeling. One of the reasons for this broad use appears to be the software’s balance of user control with the simplicity of the graphical user interface (GUI) and automated model parameterization. SDMtoolbox facilities modeling from start to finish, by aiding compilation of occurrence and environmental data, and spatially vetting occurrence data to reduce spatial biases in occurrence records. Lastly, it rigorously parametrizes each MaxEnt model by performing spatial jacking in conjunction with tuning experiments (adjusting regulation multipliers and feature classes, discussed in itemized points below). This parametrization approach can allow the selection of model settings that have high discriminatory ability and a model that minimizes overfitting to noise as well as to the spatial biases in occurrence data and corresponding environmental biases (Radosavljevic & Anderson, 2014). Low overfitting and high discriminatory ability are the two main desired qualities of a ‘good’ species distribution model (Lobo, Jiménez-Valverde & Real, 2008; Peterson et al., 2011; Warren & Seifert, 2011). Discriminatory ability characterizes the ability of the model to distinguish suitable from unsuitable areas and is typically measured with the area under the curve of the receiver operating characteristic plot (AUC/ROC) (Peterson et al., 2011) output from MaxEnt (and other programs). Overfitting is the tendency of a model to fit the random error (or any bias in the sample) rather than the true relationship between the calibration records and predictor variables. Often, over-fit models predict the calibration data very well, but perform poorly on other data sets. Overfitting is typically assessed with the false negative rate, also called omission error-rate (OER henceforth). With an appropriately selected threshold converting a continuous prediction into a binary one, OERs indicate the proportion of presences incorrectly classified as falling into unsuitable areas (typically resulting from a prediction that is too tightly fit to the conditions at calibration localities; Anderson, 2003). The best model output from SDMtoolbox is the model with the lowest OER, and of those models, if multiple, the model with the highest AUC. Lastly, if a model has an identically low OER and high AUC values, the feature class complexity is accounted for selecting the model with the lowest complexity. If there is a single model with the lowest OR, AUC values should be assessed in the MaxEnt outputs post-hoc and used as an independent metric of performance resulting from model tuning.

The primary ways SDMtoolbox minimizes model overfitting and properly parameterizes each species distribution model it creates (vs. using only MaxEnt without SDMtoolbox) are discussed in the following paragraphs. The first example concerns the curation of occurrence records. To perform well, most SDM methods require input-occurrence data to be spatially independent. However, researchers often introduce environmental biases into their SDMs from spatially autocorrelated occurrence points. It is important to eliminate spatial clusters of localities for model calibration and evaluation. When spatial clusters of localities exist, often models are over-fit towards environmental biases (reducing the model’s ability to predict spatially independent data) and model performance values are inflated (Fig. 1D; Veloz, 2009; Hijmans, 2012; Boria et al., 2014). In SDMtoolbox, this can be done in several ways, using the Spatially Rarefy Occurrence Data for SDMs (reduce spatial autocorrelation) tool set.

The second way SDMtoolbox minimizes model overfitting of each species distribution model it creates is by carefully controlling the background selection by using bias files. Bias files control where background points are selected and thereby avoid habitats greatly outside of a species’ known occurrence. Background points are meant to be compared with presence data to help identify the environmental conditions under which a species can potentially occur. Typically, background points are selected within a large rectilinear area. Within such areas, environmentally suitable but uncolonized or biogeographically isolated habitat often exists. The selection of background points within these habitats increases commission errors (false positives). As a result, the ‘best’ performing model tends to be over-fitted because the selection criterion favors a model that fails to predict the species in the un-colonized climatically suitable habitat (Anderson & Raza, 2010; Barbet-Massin et al., 2012). The likelihood that suitable unoccupied habitats are included in background sampling increases with distance from the realized range of the species. Thus, a larger study of spatial extent can lead to the selection of a higher proportion of less informative background points (Barbet-Massin et al., 2012). Such issues are ameliorated by being more selective in the choice of background points in MaxEnt (Barve et al., 2011; Merow, Smith & Silander, 2013). In SDMtoolbox, this can be done several ways, using the Gaussian Kernel Density of Sampling Localities, Sample by Buffered Local Adaptive Convex-Hull, Sample by Buffered MCP, and Sample by Distance from Obs. Pts. tools (Fig. 1G).

The MaxEnt program attempts to limit model complexity during parameterization by controlling model regularization. This regularization imposes a penalty for each term included in the model and for higher weights given to a term (Phillips, Anderson & Schapire, 2006; Anderson & Gonzalez, 2011). The current release of MaxEnt implements a regularization multiplier, which is a user-specified coefficient that is applied to the value of the respective beta parameters of each feature class incorporated into the model (Phillips, Anderson & Schapire, 2006). This regularization multiplier alters the overall level of regularization rather than changing each beta parameter individually. The default setting in MaxEnt is a value of 1. Researchers have reported that regularization multipliers as high as 2.0 to 4.0 were necessary to reduce overfitting resulting from lower regularization multiplier values (Radosavljevic & Anderson, 2014). Qualitative assessments of the geographical predictions reiterate this conclusion (Radosavljevic & Anderson, 2014). To evaluate and compare multiple regularization parameters, in absence of SDMtoolbox, MaxEnt requires users to run multiple instances of the program and then manually compare model performance statistics (i.e., OR and AUC). SDMtoolbox v2.0 allows you to input a range of regularization multipliers and automatically selects the value resulting in the best model (via the process clarified in the first paragraph of the ‘Discussion’).

Another key parameter in a MaxEnt model is the feature class, which determines the kinds of constraints allowed in a model. A feature is a function of user-input environmental variables and can be any single one or various combination of six feature classes implemented: linear (L), quadratic (Q), product (P), threshold (T), hinge (H) or category indicator (C) (Phillips, Anderson & Schapire, 2006; Phillips & Dudík, 2008). The MaxEnt model features impose varying constraints on the relationship between the occurrences and user input environmental variables and result in models of varying complexities. It is important to remember that even if multiple feature classes are allowed for model-building, not all classes will necessarily be incorporated in the final model. The default MaxEnt setting for feature class, called “auto features,” applies the class or classes estimated to be appropriate for the particular sample size of occurrence records based on extensive tuning experiments (Phillips & Dudík, 2008). The use of complex feature settings allows MaxEnt to produce a model that is more sensitive to details of a species’ environmental tolerances. However, complex feature classes can also lead to over-fit models. Phillips & Dudík (2008) selected the following feature classes for continuous variables as default for the corresponding occurrence record sample sizes: all feature classes for at least 80 occurrence records; L, Q and H for sample sizes 15 to 79; L and Q for 10 to 14 records; only L for below 10 records (Phillips & Dudík, 2008). These settings were subsequently implemented in the “auto features” of MaxEnt. However, as recommended by Shcheglovitova & Anderson (2013) to reduce model overfitting, SDMtoolbox does not limit feature class use by the number of occurrence records and allows users to compare models created from five different combinations of feature classes: 1. L; 2. L & Q; 3. H; 4. L, Q & H; 5. L, Q, H, P & T. For the fifth group of feature class combinations, in SDMtoolbox v2 the threshold feature class (T) can be enabled or disabled in accordance with the current MaxEnt 3.4.1 recommendations (where T is disabled) or previous versions (where T is enabled). The C class is reserved for categorical variables and independently applied to associated layers in all five feature class combinations. In addition to the methods implemented in SDMtoolbox, we urge users to view of responses curves in their final model and evaluate if they are biologically logical.

Spatial jackknifing (or geographically structured k-fold cross-validation) tests and evaluates performance of spatially segregated localities. The last step of model parameterization SDMtoolbox implements spatial jackknifing. To do this, SDMtoobox splits the landscape into 3–5 regions based on Voronoi polygons (a polygon whose interior is closest to an individual occurrence record) and spatial clustering of occurrence points (Fig. 1G). Models are calibrated with all permutations of the groups using occurrence points and background data from n − 1 spatial groups and then evaluated with the withheld group (Fig. 1H). Spatial jackknifing has demonstrated clear advantages over random sampling of test/training occurrence data (as is common practice and the default setting in MaxEnt). In experiments on the effects of these two treatments, randomly partitioned occurrence datasets produced inflated estimates of performance and led to over-fit models (Radosavljevic & Anderson, 2014). Under the spatial jackknifing approach, increasing the regularization multiplier did not sufficiently counteract the effects of the strong spatial bias in the localities used for model calibration (artificially inserted into their experimental approach; Radosavljevic & Anderson, 2014). In contrast, the spatial jackknifing approach was shown to sidestep problems of the artificial spatial bias (and any corresponding environmental biases) and allow for detection of overfitting to environmental biases that differed among the spatial partitions (Phillips, Dudík & Schapire, 2017). Spatial jackknifing, evaluation of multiple feature classes and multiple regularization parameters are implemented within the Run MaxEnt: Spatial Jackknifing tool.

Conclusions

The scripts in SDMtoolbox 2.0 streamline many geospatial analyses and simplify GIS processes associated with analyzing biological datasets. SDMtoolbox 2.0 can dramatically reduce the repetitive and time-consuming analyses commonly associated with species distribution modeling: data pre-processing, modeling parametrization, model evaluation, and post-SDM analyses. This release of SDMtoolbox allows researchers to use the most current ESRI, 2017 software and reduces the amount of time that would be spent developing common solutions. The latest version of SDMtoolbox, a user guide, and example data are freely available at http://www.sdmtoolbox.org. For questions or suggestions regarding SDMtoolbox 2.0, go to our Google Group (https://groups.google.com/forum/#!forum/sdmtoolbox) or email sdmtoolbox.help@gmail.com.

Supplemental Information