The risk screening of non-native species that are likely to be invasive in a defined risk assessment area is crucial for implementing strategies of rapid response and mitigation to protect native biodiversity and socio-economic activities. However, for successful risk-ranking of the screened species, scientifically defensible evidence in support of the screening outcomes must be provided, and computation of a correctly calibrated threshold to distinguish between medium-risk and high-risk species must be achieved. This paper reviews published applications of the “second-generation” Weed Risk Assessment-type decision support tools (i.e. the Aquatic Species Invasiveness Screening Kit and the Terrestrial Animal Species Invasiveness Screening Kit) and evaluates them in terms of the above two requirements. Several procedural errors were identified that involved: i) lack of provision of the report with details of the species-specific screenings; ii) incomplete justifications for the responses in the toolkit questionnaire; iii) incomplete details of the protocol used for the a priori categorisation of the screened species for threshold computation; iv) unaccepted or non-existent taxonomic names for the screened species (including typographical errors). Guidelines are provided for both assessors and reviewers to ensure that these procedural errors are avoided in future applications of these risk screening toolkits.
The introduction of non-native species worldwide is an increasingly challenging issue in invasion biology that is known to threaten biodiversity with harmful effects on ecosystem function and services (Vilà & Hulme 2017, Renault et al. 2022). To this end, identifying which existing and potential future non-native species are likely to be invasive in a defined risk assessment area is crucial to inform environmental managers, stakeholders and decision-makers of the associated risks. This outcome is necessary to implement strategies for rapid response and mitigation (Simberloff et al. 2013), to inform policy and management decisions for the protection of native biota (D'Antonio & Meyerson 2002, Rahel & Olden 2008, Carter et al. 2016), and to prevent losses in productive sectors and safeguard social activities (Vilà & Hulme 2017). However, for risk management decisions to be adequately informed, environmental biologists must be able to deliver “sound science” (Crawford-Brown 2005) in terms of consistency, accuracy and reliability of the findings.
Risk screening (or hazard identification) is the first step in risk analysis, followed by risk assessment and risk management and communication (Canter 1993, Copp et al. 2016a). For the risk screening of nonnative species, electronic decision-support tools are available that can be overall distinguished into Weed Risk Assessment (WRA) type (Pheloung et al. 1999, Gordon et al. 2008, Copp et al. 2016b, 2021, Vilizzi et al. 2022c) and “others” (D'hondt et al. 2015, Drolet et al. 2016; but see also Singh & Lakra 2011). The commonality of the score-based WRA-type toolkits is the computation of a calibrated threshold value with which to distinguish non-native species that pose a high risk of being (or becoming) invasive in the risk assessment area from those non-native species that pose a lower (i.e. low-to-medium) risk of invasiveness (see Vilizzi et al. 2022a, b). Identifying the high-risk species allows policy and decision-makers to prioritise species for more comprehensive (follow-up) risk assessment to inform decisions on the most appropriate management approach (Copp et al. 2016a). Briefly, risk assessment examines in detail the risks of: i) introduction (entry); ii) establishment (of one or more self-sustaining populations); iii) dispersal (more widely within the risk assessment area, i.e. so-called secondary spread or introductions); and iv) impacts (to native biodiversity, ecosystem function and services, and the introduction and transmission of diseases).
Of the WRA-type toolkits, the Australian WRA was the first to be developed for screening terrestrial plants (Pheloung et al. 1999) and was later adapted to aquatic plants (Gordon et al. 2008). The WRA questionnaire and related scoring system was the template for the creation of the freshwater Fish Invasiveness Screening Kit (FISK) and its “sister” -ISK toolkits for screening freshwater invertebrates, marine invertebrates, marine fish and amphibians (Copp et al. 2005a, b, Copp 2013, Lawson et al. 2013; review in Vilizzi et al. 2019). The FISK family of toolkits was eventually replaced by a single, taxon-generic decision-support tool, namely the Aquatic Species Invasiveness Screening Kit (AS-ISK: Copp et al. 2016b), which has since been extensively employed worldwide to screen a vast number of freshwater, brackish and marine species (see Vilizzi et al. 2021, 2022a, b). More recently, to complement the range of available WRA-type toolkits to screen other terrestrial taxa besides plants (cf. WRA), the Terrestrial Animal Species Invasiveness Screening Kit (TAS-ISK) was developed (Vilizzi et al. 2022c). A common feature of the “second-generation” screening toolkits AS-ISK and TAS-ISK that distinguishes them from the “first-generation” WRA-type toolkits (i.e. the WRA and FISK family), which are all based on 49 questions, is an additional set of six questions with which the assessor evaluates how predicted (future) climate conditions are likely to affect the various risks (of introduction, establishment, dispersal and impacts) by the species being screened. Additionally, the AS-ISK and TAS-ISK comply with the “minimum requirements” (Roy et al. 2018) for the assessment of invasive non-native species concerning EU Regulation 1143/2014 (European Council 2014), and their adoption has been approved officially in some countries (e.g. Iran: H. Valikhani, pers. comm.; The Philippines: A.S. Gilles, Jr., pers. comm.) or is currently being considered (Vietnam: K.A.T. Ta, pers. comm.).
As part of the ongoing effort to provide environmental biologists involved in the risk screening of non-native species with consistent guidelines for delivering the highest-quality and scientifically defensible research outputs (see Vilizzi et al. 2022a, b), this paper aims to provide a comprehensive review of the published AS-ISK and TAS-ISK applications to date to evaluate the consistency and reliability of the underlying risk screening process (as described in Vilizzi et al. 2022a). Specifically, published screening applications are evaluated in terms of: i) provision of scientifically defensible and thorough evidence in support of the species-specific risk outcome scores, and ii) level of detail and completeness of the information provided about the screened species for correct computation of a calibrated threshold for their risk-ranking. As a result, in the applications reviewed, corrections are made, where required, to the risk ranks of the screened species for a risk assessment area in case of procedural errors, including issues related to taxonomy. Of note, the present review will focus on the second-generation AS-ISK and TAS-ISK decision-support tools due to their inclusion of a climate change component and compliance with legislative requirements (as described above), and also considering the obsoleteness of the FISK family of toolkits (Vilizzi et al. 2019). Other decision-support tools will also not be included in this review due to their more restricted use and adoption and lack of computation of a risk assessment area specific (i.e. calibrated) risk threshold.
Material and Methods
Full descriptions of the AS-ISK and TAS-ISK (both available for free download at www.cefas.co.uk/nns/tools/) are provided in Copp et al. (2016b, 2021) and Vilizzi et al. (2022c), respectively. Briefly, these two decision-support tools are an adaptation of the WRA that combines the generic screening questions in the pre-screening module of the European Non-native Species in Aquaculture Risk Analysis Scheme (Copp et al. 2016a) with the architecture of the FISK v2 (Lawson et al. 2013). As with the WRA, the AS-ISK and TAS-ISK questionnaires consist of 49 Basic Risk Assessment (BRA) questions (hereafter, also referred to as “Qs” whenever applicable), but unlike the WRA, they include an additional six Climate Change Assessment (CCA) questions. In addition, the AS-ISK allows assessors to screen 27 taxonomic groups of aquatic organisms in their choice of 32 languages (Copp et al. 2016b, 2021), and the TAS-ISK nine taxonomic groups of terrestrial animals also in 32 languages (Vilizzi et al. 2022c).
In both the AS-ISK and TAS-ISK, to achieve a valid screening, the assessor must provide for each question: a response, a level of confidence for the response, and a justification based on literature sources. The outcomes are a BRA score and a (composite) BRA+CCA score. Scores < 1 suggest that the species poses a “low risk” of becoming invasive in the risk assessment area, whereas scores ≥ 1 indicate a”medium risk” or a “high risk”. The threshold value to distinguish between medium-risk and high-risk species is obtained by calibration based on Receiver Operating Characteristic (ROC) curve analysis (see Vilizzi et al. 2022a). This analysis requires the a priori categorisation of the screened species into invasive or non-invasive based on a (four-step) protocol that relies upon a search of databases and literature sources and is subject to the availability of a representative sample size of screened species together with a “near-balanced” proportion between species categorised a priori as either invasive or noninvasive (for full details of the methodology see Vilizzi et al. 2022a).
Following completion of the screenings, species-specific report(s) can be generated by both the AS-ISK and TAS-ISK (in a choice of Excel™ spreadsheet, PDF or MHTML format). Each report includes all information about each of the screened species by listing the 55 questions (full text) and corresponding responses, level of confidence and justification (for an example, see Appendix S1). The report's primary purpose is to allow transparency about the thoroughness and validity of the risk screening process given (peer-reviewed) publication and/or communication of the risk outcomes to environmental managers, stakeholders and decision-makers. To ensure that this requirement is met, since the release of the AS-ISK v2.1.1, in both toolkits a prompt has been included upon launching the Q&A dialogue to carry out the selected species' screening (see Copp et al. 2016b, Vilizzi et al. 2022c) that reminds the assessor of the six “ALIENS” principles in risk assessment (Table 1).
In a comprehensive review of published AS-ISK and TAS-ISK applications (as available to the authors and retrieved from a Google Scholar search), data were collated from each source study about: i) the risk assessment area(s) and ii) the provision (or lack thereof) of the species-specific report(s) (hereafter, for simplicity “report”). For the applications that provided a report, a distinction was made between those that included complete justifications for the responses to all questions as per ALIENS Principle S (“support justifications to the responses with relevant peer-reviewed literature as much as possible, followed by grey literature, using caution in the use of web-based information (including from databases)”: see Table 1) and those that did not.
All AS-ISK and TAS-ISK applications retrieved were then scanned for whether they implemented, if applicable, the a priori categorisation for computing a calibrated threshold (as per the recent correction by Vilizzi et al. 2022b) for the risk assessment area to risk-rank the screened species. For the studies that carried out such an implementation, a distinction was then made into those that: i) provided a “full” table of the a priori categorisation protocol (as per Vilizzi et al. 2022a), including the corresponding outcomes (i.e. invasive or non-invasive); ii) provided only the outcomes (hence, classified as “partial”); and iii) did not provide any information in regard (hence, despite computing a calibrated threshold). Of note, those AS-ISK applications published before 2018, hence before the (informal) implementation of the a priori categorisation protocol, were still classified as “full”, although with a caveat as they relied exclusively on a search of FishBase ( www.fishbase.org). Finally, all applications were screened regarding the correct implementation of the a priori categorisation for ROC curve analysis and reliability of the resulting species-specific risk ranks.
“ALIENS” principles in risk assessment as user-prompted in the Aquatic Species Invasiveness Screening Kit (AS-ISK) and the Terrestrial Animal Species Invasiveness Screening Kit (TAS-ISK) decision-support tools before carrying out a species-specific screening.
Provision and completeness of the report
In total, 50 published risk screening applications were retrieved, of which 49 of the AS-ISK and one of the TAS-ISK (Table 2). Notably, the “surrogate” AS-ISK application by Kopecký et al. (2019) was not included in this review because it provisionally and partially used the toolkit's questionnaire (i.e. Biology/Ecology section only: see Copp et al. 2016b) to screen terrestrial reptiles, i.e. a non-aquatic organisms group. Of the 50 applications, 27 (54.0%) provided a report and 23 (46.0%) did not. However, of these applications, the one by Lyons et al. (2020) provided the Excel database spreadsheet of screenings instead of the report proper (included either in Excel or PDF format in the other applications), hence shifting the above ratios to 26 (52.0%) to 24 (48.0%).
Of the 27 applications that provided a report (sensu lato), 23 (85.2%) included complete justifications for all screened species as per ALIENS Principle S, whereas four (14.8%) failed to do so, although to a different extent (Table 2). Specifically:
Interesova et al. (2020) – this screening of freshwater fishes for the River Ob Basin (Russia) included: the generic justification “FishBase” for 796 (46.7%) responses out of the 1,705 in total (i.e. 31 species × 55 Qs); for Q5 (“What is the quality of the climate matching data?”), the spurious entry “1” for all species; and Q26 (“Is the taxon likely to consume threatened or protected native taxa in the RA (risk assessment) area?”), the vague term “Redbook” for 28 (90.3%) out of the 31 species screened.
Killi et al. (2020) – this screening of marine invertebrates (jellyfishes) for the Mediterranean Sea included the generic term “expert opinion” (uzman görüşü, in the original Turkish language in which the screenings were performed) for 562 (22.7%) responses out of the 2,475 in total (i.e. 45 species × 55 Qs).
Stasolla et al. (2021) – this screening of marine invertebrates (crustacean decapods and barnacles) for the Mediterranean Sea included: the generic justification “personal communication” for 186 (11.7%) responses out of the 1,595 in total (i.e. 29 species × 55 Qs); the vague term “Ciesm” for 18 (1.1%) responses; and Q4 (“How similar are the climatic conditions of the risk assessment area and the taxon's native range?”) and 5 (“What is the quality of the climate matching data?”), the generic acronym “HESSD” for 19 (65.5%) out of the 29 species screened.
Published applications using the AS-ISK and TAS-ISK. For each application (Reference), the following information is provided: risk assessment area (multiple in some applications); Report(s) of the screened species (Y = Yes; Y* = Yes, but in database spreadsheet form; N = No; “-“= not applicable if not provided); implementation of the a priori species Categorisation as invasive or non-invasive (after Vilizzi et al. 2022a): F = full; F* = full with a caveat; P = partial; N = none; n.a. = not applicable (i.e. no risk assessment area specific calibration required) (see text for details).
Wei et al. (2021a) – this screening of freshwater fishes for the Lower Pearl River Basin (China) and the Chao Phraya River Basin (Thailand) (i.e. two different risk assessment areas) included the spurious term “justification” for 330 (18.8%) responses out of the 1,760 in total (i.e. 32 species × 55 Qs), though only relative to one of the six assessors involved in the study and for the latter risk assessment area.
Species a priori categorisation and calibrated threshold
Of the 50 applications, 23 (46.0%) did not require implementation of the a priori categorisation of the screened species for the computation of a calibrated threshold for the risk assessment area (Table 2); these applications dealt with either a single or a few species whose level of risk was evaluated either empirically or based on a “generalised” threshold (see Vilizzi et al. 2021). Of the 27 applications that implemented the categorisation: 10 (37.0%) provided a complete description of the protocol that was presented as a table, although four of these applications were published in 2017 (hence with a caveat), whereas the one by Tomanić et al. (2022), albeit published recently, still relied on the obsolete a priori categorisation approach; 14 (51.9%) included a partial description of the protocol limited to the listing of the invasive/non-invasive species-specific outcomes; three (11.1%) did not provide any information about the a priori invasiveness categorisation of the screened species (Table 2).
Additionally, the AS-ISK applications by Li et al. (2017), Saba et al. (2021), Yapici (2021), To et al. (2022) and Tomanić et al. (2022) were found to contain a range of inconsistencies and errors as described below.
In the application by Li et al. (2017) on freshwater fishes for the River Yarlung Zangbo (China), the a priori categorisation for three species according to FishBase only (i.e. before 2018) was incorrect, namely: “QiaoZuiBo” Culter alburnus (Basilewsky, 1855) and “XiaoHuangYouYu” Micropercops swinhonis (Günther, 1873), which were categorised as invasive instead of non-invasive, and bighead carp Hypophthalmichthys nobilis (Richardson, 1845) (inappropriately referred to in that study with the no longer accepted name Aristichthys nobilis: https://www.marinespecies.org/aphia.php?p=taxdetails&id=154606), which was categorised as non-invasive instead of invasive. As a result, the calibrated (BRA) threshold of 29 provided in that study (noting that no BRA+CCA threshold was computed) should be revised by reimplementing ROC curve analysis based on the different categorisation of the screened species. Additionally, the grass carp Ctenopharyngodon idella (Valenciennes, 1844) was incorrectly referred to as Ctenopharyngodon idellus.
In the application by Saba et al. (2021) on freshwater fishes for Klang Valley (Malaysia), the procedure for calibration (also incompletely described) with resulting thresholds and species risk ranks is to be regarded as statistically invalid due to the insufficient number of screened species in that study (n = 5), hence well below the requirement for the implementation of ROC curve analysis (i.e. a generalised threshold should instead have been used).
In the application by Yapici (2021) on marine fishes for the Mediterranean Sea, re-implementation in the present study of the a priori categorisation protocol (augmented with searches of the ELNAIS https://elnais.hcmr.gr/and NOBANIS https://www.nobanis.org/ region-specific databases, as per that study) resulted in all 22 species screened being categorised a priori as non-invasive (noting that, unlike in that study, no a priori categorisation was applicable to the sergeant majors Abudefduf spp. being a complex of species, as no indication to any “reference” species was provided to allow implementation of the protocol) (Table S1). This situation is unlike the 15 species categorised a priori as invasive in that study (again, after removal of Abudefduf spp.). As a result, because of the 0:22 proportion between a priori invasive and noninvasive species, no ROC curve analysis could be implemented based on the present re-evaluation. Therefore, the calibrated threshold of 27.5 provided in that study (noting that the BRA+CCA threshold of 33.0 is no longer applicable as per Vilizzi et al. 2022b) is to be regarded as invalid, and the generalised (BRA) threshold of 12.75 for marine fishes or the generalised (BRA) thresholds of 19.5 or 12.5 for marine fishes in temperate or tropical climates, respectively, provided in Vilizzi et al. (2021) must instead be used for risk-ranking the species. As a result, all species screened in that study will be ranked as high-risk (i.e. based on the 12.75, 19.5 or 12.5 thresholds), hence including the eight species incorrectly classified as medium-risk according to the invalid threshold of 27.5 (Table S1).
In the application by To et al. (2022) on freshwater fishes for Lake Taal (Philippines), a statement was made that the species' a priori categorisation (intrinsically binary, as per ROC curve analysis requirements: Bewick et al. 2004) was based on three “diagnosis scores”, namely “no record”, invasive and non-invasive. However, given the violation of the computational requirements to implement ROC curve analysis and the lack of any description of how the species categorised as “no record” were included in the analysis, the species-specific outcomes in that study must be regarded as invalid, including the calibrated threshold of 30 provided in that study (noting that the BRA+CCA threshold of 55 is no longer applicable as per Vilizzi et al. 2022b), which will have to be re-computed subject to revision of the a priori categorisation for the species incorrectly categorised as “no record”. Further, in that study, incorrect scientific names were provided for two of the screened taxa, namely “Albino plecostomus” instead of (presumably) suckermouth catfish Hypostomus plecostomus (Linnaeus, 1758) and “Synodontis valentiana” instead of (presumably) clown squeaker × cuckoo catfish Synodontis decorus (Boulenger, 1899) × Synodontis multipunctatus (Boulenger, 1898) (note that the two “pseudo-scientific” names are intentionally not italicised but given in quotes). Furthermore, following re-implementation of the a priori categorisation of the species originally “scored” as invasive and non-invasive (i.e. 10 out of the 25 in total), the giant pangasius Pangasius sanitwongsei (Smith, 1931) and the ripsaw catfish Oxydoras niger (Valenciennes, 1821) were categorised as noninvasive instead of incorrectly invasive (Table S2). Finally, of the four global databases (sensu Vilizzi et al. 2022a) mentioned in that study for use in the a priori categorisation of the screened species, the International Union for Conservation of Nature (IUCN: https://www.iucnredlist.org/) database does not contain any information about species invasiveness, whereas no database is (currently) accessible for the Invasive Species Specialist Group (ISSG: http://www.issg.org/index.html).
In the application by Tomanić et al. (2022) on marine fishes and invertebrates for the South Adriatic Sea, a pooled (i.e. for both organisms groups) calibrated threshold of 34 was computed (noting that the BRA+CCA threshold of 46 provided in that study is no longer applicable as per Vilizzi et al. 2022b). However, re-implementation in the present study of the a priori categorisation protocol (which in that study was limited to FishBase and the non-accessible ISSG, as per above, hence obsolete by ignoring the protocol by Vilizzi et al. 2022a) for the nine screened species in total, resulted in two of them being re-categorised from invasive to noninvasive. These species were the blunthead puffer Sphoeroides pachygaster (Müller & Troschel, 1848) and the northern brown shrimp Penaeus aztecus (Ives, 1891). Also, the latter species was wrongly reported as Farfantepanaeus aztecus, hence with a taxonomically unaccepted ( https://www.marinespecies.org/aphia.php?p=taxdetails&id=395176) and misspelt scientific name, i.e. instead of Farfantepenaeus aztecus (Ives, 1891) (Table S3). Because of the limited sample size (n = 9), re-computation of ROC curve analysis yielded an Area Under the Curve (AUC) equal to 0.575 (hence, far below acceptable discriminatory power: see Hosmer et al. 2013) and with unrealistic 5% and 95% confidence intervals (CIs) of 0.1295 and 1, respectively. As a result of the invalidity of the threshold used in that study, the generalised thresholds of 19.5 and 15.1 for marine fishes and marine invertebrates, respectively, in temperate climates (see Vilizzi et al. 2021) should be used to risk-rank the screened species. Based on the correct risk ranking, all screened species will be classified as high-risk, but noting that, unlike in any other AS-ISK applications, surprisingly, no risk ranking was provided in that study, even based on the erroneous thresholds. As a final note, the IUCN status for the bluespotted cornetfish Fistularia commersonii (Rüppell, 1838) (which was provided together with the FishBase details for all screened species) was incorrectly reported as “Not evaluated” despite this species being present and evaluated as “Least concern” in that database ( https://www.iucnredlist.org/species/18257780/115368874).
This review has identified several procedural errors in risk screening applications of the second-generation WRA-type decision-support tools. Specifically, almost half of the applications did not provide a report of the species-specific screenings (Table 2), despite this option having been available since the first release of the AS-ISK (Copp et al. 2016a) and, previously, as part of the FISK family of toolkits (Copp et al. 2005a, b). Although most of the applications that provided a report satisfied the overall requirements of the six ALIENS principles, a small proportion failed. This outcome indicates an oversight by the authors in ensuring quality control of the screenings before publication. Additionally, in the case of the database spreadsheet provided by Lyons et al. (2020) in lieu of the report, it must be noted that such file format (password-protected to make it editable only within the AS-ISK or TAS-ISK, as applicable), despite including the justifications, contains the coding for both responses and corresponding levels of confidence and only relative to the question numbers (instead of the entire text of the question), making the output less understandable compared to the report, and certainly of little use to decision-makers.
Unlike the report, details of the species' a priori categorisation were not provided in only a few applications. However, the majority of those that did comply with this requirement did not provide species-specific details of the relevant database and literature searches (i.e. partial description) despite these sources being listed in the Material and Methods section of the corresponding publication. Finally, one application (see Results), albeit providing a full (with a caveat) description of the a priori species categorisation, should be considered for revision due to the incorrect threshold and, possibly, resulting risk ranks for some species.
Concerning the procedural errors identified in three applications, and notwithstanding the invalidity of some risk outcomes in others (see Results), it must be noted that the listing of a species in a certain (global or region-specific: sensu Vilizzi et al. 2022a) database does not imply its being invasive, since the invasiveness and resulting impact (if any) of a nonnative species must be “demonstrated” (hence, not “assumed”) by at least one peer-reviewed reference (see Vilizzi et al. 2022a). This result is evinced by looking at the global and regional database search outcomes in Tables S1 and S2, where in several cases, the species, albeit listed, are not to be regarded as invasive and causing an impact and, in some cases, have not (yet) been evaluated. Clearly, the incorrect a priori categorisation of species will inevitably affect the outcome of ROC curve analysis (if applicable), hence the computation of a correct calibrated threshold for the risk assessment area. This issue is of special concern in the case of a species incorrectly classified as medium-risk instead of high-risk, with all consequences in terms of erroneous reporting to decision-makers for implementation of early detection and/or control/mitigation/eradication measures. Finally, careful checking of the requirements for correct implementation of ROC curve analysis must be ensured in terms of a sufficient sample size to achieve adequate discriminatory power. In this regard, a measure of the accuracy of the calibration analysis is the AUC (including the 5% and 95% CIs) whose values are interpreted as: 0.7 ≤ AUC < 0.8 = acceptable discriminatory power, 0.8 ≤ AUC < 0.9 = excellent, 0.9 ≤ AUC = outstanding (Hosmer et al. 2013).
Provision of the correct scientific name (including checking its taxonomic validity) for all screened species is another fundamental requirement for delivering scientifically defensible and accurate outcomes in risk screening studies. It is also proof of the validity of the screenings themselves. This situation was exemplified in this review by two applications (see Results) for which it is argued that peer-reviewed literature could be retrieved to justify the questions for the two incorrectly denominated “species”. Further, another important (and far from trivial) requirement highlighted in the present review is to ensure that the correct spelling of the scientific name for each of the screened species (as input by the assessor from the toolkit's New/Edit dialogue: Copp et al. 2016b, Vilizzi et al. 2022c) is provided in the AS-ISK and TAS-ISK database spreadsheets. Indeed, this is a surprisingly widespread problem, as testified by the first author of the present paper in his role for “quality control” of the databases of screenings prior to data analysis for several applications. As the scientific name for the screened species (together with the organisms group identifier, common name, assessor's name and risk assessment area) are required fields in the spreadsheet database that are ultimately displayed in the report, these must be carefully verified for correctness. Additionally, using a species' incorrect name may indicate that the relevant literature supporting the screening (cf. ALIENS Principle S) has not been searched in detail, with the result that most relevant studies may not have been retrieved and included. To quote Darwin (1881, p. 2): “The subject may appear an insignificant one, but we shall see that it possesses some interest; and the maxim “de minimis non curat lex”, does not apply to science”.
Overall, the findings of the present study should be regarded as a clarion call for future screening applications using second-generation WRA-type toolkits to comply fully with the requirements of accuracy, consistency and transparency of the underlying risk screening process (e.g. Peeler et al. 2007), as recently described by Vilizzi et al. (2022a). In this regard, the ALIENS principle S indicates that scientifically defensible justifications to the responses for questions should be provided to achieve a “valid” risk screening and, ultimately, risk ranking of the screened species. To satisfy this principle, clear justification for the responses should be based on relevant peer-reviewed literature as much as possible, followed by grey literature, and using caution in the use of web-based information, including from databases (see also Herrera-Viedma et al. 2006). However, this requirement may often be difficult to fulfil for those species whose information is either not easily accessible (e.g. documents in the local language and/or not available online though still of relevance), obscure to interpret, or requiring advanced skills and knowledge in the use of web applications, as in the case of the AS-ISK and TAS-ISK climate change component (Herrera-Viedma et al. 2006, Britt et al. 2014, Amano et al. 2016, Angulo et al. 2021). Further, in those cases where expert opinion needs to be used, full justification should still be provided to support the validity of the responses; this is the requirement and format in which supporting data and information are provided in all risk assessments prepared for the EU ( https://data.europa.eu/doi/10.2779/08867).
In conclusion, it is recommended that future applications of the second-generation WRA-type toolkits should provide:
1) complete justifications for the responses to all questions according to ALIENS principle S to ensure completeness of the screening for a particular species;
2) provision of the report in any peer-reviewed publication resulting from the screening study, which is usually included as supplementary material either in PDF format or as an Excel spreadsheet/workbook (see Table 2);
3) provision of a table indicating the sources of information (i.e. databases and other online resources) used for the a priori categorisation of the screened species (see Table 2);
4) proper implementation of ROC curve analysis in terms of sufficient sample size (i.e. the number of screened species) and discriminatory power of the AUC and related CIs;
5) correct taxonomy and scientific names of the screened taxa.
The above guidelines should not only be followed by the assessor(s) involved in a screening study but should also be verified by reviewers of peer-reviewed journal articles (and reports) concerning the application of the second-generation WRA-type decision-support tools.
Marina Piria was supported by the EIFAAC project “Management/threat of aquatic invasive species in Europe”. Gordon H. Copp (Centre for Environment, Fisheries and Aquaculture Science, Lowestoft, UK) provided the ALIENS principles prompt to the AS-ISK v2.1.1.
Study conceptualisation and writing the article: L. Vilizzi and M. Piria; data analysis: L. Vilizzi.
Supplementary online material
Appendix S1. Sample TAS-ISK report for the New Zealand flatworm Arthurdendyus triangulates ( https://www.ivb.cz/wp-content/uploads/JVB-vol.-71-2022-Vilizzi-L.-Piria-M.-Appendix-1.pdf).
Table S1. Marine fish species screened by Yapici (2021) for the Mediterranean Sea. Details of the species a priori categorisation with corresponding Outcome (Invasive, Non-invasive: in bold, changes from the original erroneous Invasive to the correct Non-invasive categorisation) are provided (after Vilizzi et al. 2022a) together with the re-computed risk ranks (in bold, changes from the original erroneous Medium to the correct High risk) based on the generalised threshold of 12.75 for marine fishes (after Vilizzi et al. 2021) instead of the invalid threshold of 33.0 provided in that study. Note that Abudefduf spp. No categorisation was possible as no reference species was provided. SSD = Species-specific database; FB = FishBase ( www.fishbase.org); CABI ISC = Centre for Agriculture and Bioscience International Invasive Species Compendium ( https://www.cabi.org/ISC); GISD = Global Invasive Species Database ( http://www.iucngisd.org/gisd/); EASIN = European Alien species Information Network ( https://easin.jrc.ec.europa.eu/easin/); ELNAIS = Ellenic Network on Aquatic Invasive Species ( https://elnais.hcmr.gr/); IESNA = Invasive and Exotic Species of North America ( www.invasive.org); NOBANIS = European Network on Invasive Alien Species ( https://www.nobanis.org/). N = no impact/threat; “-“= absent from database; n.e. = not evaluated, but present in database.
Table S2. Freshwater fish taxa screened by To et al. (2022) for Lake Taal (Philippines). Details of the species a priori categorisation with corresponding Outcome (Invasive, Non-invasive: in bold, changes from the original erroneous Invasive to the correct Non-invasive categorisation) are provided (after Vilizzi et al. 2022a). Database acronyms as per Table S1. N = no impact/threat; “-“= absent from database; n.e. = not evaluated, but present in database; n.a. = not applicable.
Table S3. Marine fish and invertebrate species screened by Tomanić et al. (2022) for the South Adriatic Sea. Details of the species a priori categorisation with corresponding Outcome (Invasive, Non-invasive: in bold, changes from the original erroneous Invasive to the correct Non-invasive categorisation) are provided (after Vilizzi et al. 2022a). Database acronyms and coding as per Table S2.