Skip to main content
  • Research article
  • Open access
  • Published:

Spatial multi-criteria decision analysis to predict suitability for African swine fever endemicity in Africa

Abstract

Background

African swine fever (ASF) is endemic in several countries of Africa and may pose a risk to all pig producing areas on the continent. Official ASF reporting is often rare and there remains limited awareness of the continent-wide distribution of the disease.

In the absence of accurate ASF outbreak data and few quantitative studies on the epidemiology of the disease in Africa, we used spatial multi-criteria decision analysis (MCDA) to derive predictions of the continental distribution of suitability for ASF persistence in domestic pig populations as part of sylvatic or domestic transmission cycles. In order to incorporate the uncertainty in the relative importance of different criteria in defining suitability, we modelled decisions within the MCDA framework using a stochastic approach. The predictive performance of suitability estimates was assessed via a partial ROC analysis using ASF outbreak data reported to the OIE since 2005.

Results

Outputs from the spatial MCDA indicate that large areas of sub-Saharan Africa may be suitable for ASF persistence as part of either domestic or sylvatic transmission cycles. Areas with high suitability for pig to pig transmission (‘domestic cycles’) were estimated to occur throughout sub-Saharan Africa, whilst areas with high suitability for introduction from wildlife reservoirs (‘sylvatic cycles’) were found predominantly in East, Central and Southern Africa. Based on average AUC ratios from the partial ROC analysis, the predictive ability of suitability estimates for domestic cycles alone was considerably higher than suitability estimates for sylvatic cycles alone, or domestic and sylvatic cycles in combination.

Conclusions

This study provides the first standardised estimates of the distribution of suitability for ASF transmission associated with domestic and sylvatic cycles in Africa. We provide further evidence for the utility of knowledge-driven risk mapping in animal health, particularly in data-sparse environments.

Background

African swine fever (ASF) is a severe and highly contagious viral infection of domestic pigs. Outbreaks have been reported from almost all the countries of Africa that occur on or below the equator, as well as several countries of West Africa and the islands of Madagascar and Cape Verde [1, 2]. The disease can impose severe limitations on pig production in endemic countries, as well as in adjoining areas, which may be at high risk of virus introduction [3, 4]. The ASF virus (ASFV) has repeatedly spread outside Africa via the movement of pigs or pig products and continues to pose a considerable threat to pig production in the wider world [1].

Conditions leading to ASF endemicity in domestic pigs in Africa are not fully understood but are likely to vary on a regional basis. In parts of eastern and southern Africa, for example, cycles involving wild suids, such as warthogs (Phacochoerus africanus) and soft tick vectors of the Ornithodoros moubata complex provide a reservoir of ASFV for domestic pigs [5]. Such ‘sylvatic’ cycles may allow the disease to persist in pig populations, even in areas with low pig population density. In parts of West Africa, on the other hand, sylvatic cycles have been infrequently described and endemicity is more likely to be the result of sustained virus transmission within high density and highly connected pig populations [2, 68]. Factors contributing to the transmission of ASF in these ‘domestic’ pig cycles may include the use of free-ranging husbandry methods, the movement of infected pigs between farms and to markets, and the feeding of contaminated domestic waste to pigs [1]. African swine fever endemicity in domestic pig populations in Africa may therefore be the result of cycles involving repeated virus introduction from wild suid reservoirs, sustained virus spread within domestic pig populations, or a combination of both.

Risk maps can be used to provide a visual representation of the spatial distribution of the risk of an event, and have been advocated as having a role in the development of targeted disease surveillance and control activities [9, 10]. Disease risk maps are traditionally developed as an extension of a statistical modelling process in which geographically explicit predictor variables are used to estimate the probability of disease occurrence [1113]. Such ‘data-driven’ methods are limited to those areas in which surveillance activities have effective coverage, or in which epidemiological studies can provide adequate data to describe the distribution of a disease. In areas where these data are unavailable, as is often the case for animal diseases in developing countries, a more pragmatic approach to risk mapping has been proposed [14, 15]. ‘Knowledge-driven’ risk mapping is one such approach that uses literature based-evidence or expert opinion, rather than a empirical exploration of available data, to describe the relative importance of risk factors for a disease [16]. In data sparse environments, this evidence can be integrated into a formal decision making process to predict the suitability of a geographic area for disease occurrence based on the presence of the identified risk factors.

The likelihood of infectious disease occurrence is typically influenced by multiple interacting factors. Knowledge-driven disease risk mapping should therefore be performed as part of a formal and systematic evaluation framework that takes into account the relative contribution each factor makes to the overall estimation of suitability. Multi-criteria decision analysis (MCDA), a methodology that allows the analysis of complex decision problems involving conflicting criteria, has previously been used within a geographic information system (GIS) for this purpose [14, 15, 17, 18]. Broadly, MCDA allows the prioritisation of the criteria that influence a decision, and provides a framework by which users can reach a decision that reflects these priorities. In the context of risk mapping, ‘criteria’ are risk factors for an undesirable event, such as disease occurrence, while the ‘decision’ relates to the estimation of suitability of an area for the event, and therefore the relative likelihood that it can occur.

In the absence of available (or reliable) disease data to adequately describe the distribution of ASF in domestic pigs in Africa, we used a spatial MCDA to predict suitability for repeated introduction of the virus into pig populations from sylvatic reservoirs and suitability for sustained spread within domestic pig populations. African swine fever has a complex epidemiology with multiple pathways for introduction and spread [1]. There remains considerable uncertainty about the relative importance of these different transmission routes, and the risk factors that influence them, particularly in Africa where few quantitative epidemiological studies have been performed. In order to incorporate some of this uncertainty into the MCDA procedure, we used a probabilistic framework that describes the contribution each risk factor makes to suitability of an area for ASF persistence in domestic pig populations via sylvatic or domestic transmission cycles.

Methods

Multi-criteria decision analysis (MCDA)

The MCDA procedure used in this study involved the following general steps (modified from Store and Kangas [19]):

  1. 1.

    Assessment of the suitability structure: identifying risk factors for ASF transmission and determining their importance in relation to the objective(s).

  2. 2.

    Producing spatial layers: raw data acquisition and transformation to the appropriate GIS layers.

  3. 3.

    Cartographic modelling: combining risk factor layers based on the suitability structures defined.

  4. 4.

    Validation: comparison of suitability predictions with available disease data.

Assessment of the suitability structure

Probable risk factors for ASF transmission in Africa

On the basis of a literature review, risk factors were identified that are expected to influence the suitability of an area for repeated transmission of ASF into domestic pig populations from sylvatic reservoirs (the ‘sylvatic’ cycle) (objective 1) or suitability for sustained transmission of ASF within domestic pig populations (the ‘domestic’ cycle) (objective 2). Repeat transmission via either cycle could be expected to lead to ASF persistence and endemicity. Risk factors for ASFV transmission, and the availability of evidence to support their role in the epidemiology of the disease, have been extensively reviewed in a number of recent papers [1, 2, 5, 7, 20], and we thus provide only a summary of the literature.

Warthogs (Phacochoerus africanus) have long been recognised as a major reservoir of ASFV for domestic pigs in Africa and outbreaks are frequently reported when pigs are reared in areas in which warthogs are common [5]. The transmission of ASFV from infected warthogs to domestic pigs typically occurs via Ornithodoros spp. tick vectors [21], although transmission may also occur as a result of contact with contaminated wild suid carcasses [22]. Free-living bushpigs (Potamochoerus porcus and P. larvatus) and giant forest hogs (Hylochoerus meinertzhageni) have also been found to be infected with ASFV [5] and may therefore act as a reservoir in areas in which they occur. The epidemiological significance of bushpigs or giant forest hogs, and the extent to which these species interact with Ornithodoros spp. ticks in the transmission of ASFV to domestic pigs, has not been fully determined, but is generally considered to be small in comparison to warthogs [5, 20].

As well as parasitising wild suids, Ornithodoros spp. inhabit cracks and crevices in pig pens and people’s homes, where they may feed on domestic pigs [23]. Some Ornithodoros spp. ticks have been shown to be capable of transovarial, transtadial and sexual transmission of ASFV [24], and can survive for long periods without feeding [25]. These tick vectors may therefore contribute to the persistence of ASF within domestic pig populations, even in the absence of wild suid reservoirs and, occasionally, in the long term absence of viraemic domestic pig hosts [26]. Ornithodoros spp., which occur widely in sub-Saharan Africa [23, 2729], are thought to contribute to disease endemicity in Malawi [30], but little evidence has been found for a role for soft tick vectors in domestic (or sylvatic) cycles in West Africa [31], and few studies have been conducted in the rest of sub-Saharan Africa.

The role of pig population density has not, to our knowledge, been determined for ASF in Africa, although it was found to be a predictor of disease risk during recent outbreaks in Russia [32]. High pig or pig farm density has been shown to be associated with infection risk for a variety of viral pig diseases [33, 34] and the risk of introduction of infection from wildlife has been related to higher domestic animal density more generally [35].

Infected pigs can shed ASFV for several weeks, hence pig movement between farms, to markets and to slaughter is likely to be a major route by which the virus spreads within domestic pig populations [1], although there have been relatively few studies to quantify the importance of these factors in Africa [6, 8, 36]. A higher density of road networks was found to increase ASF risk in Russia [32], further highlighting the probable importance of trade on disease transmission as part of the domestic cycle.

Given the available information, we identified the following risk factors as potentially contributing to the suitability of an area for objective 1 (‘sylvatic cycles’): occurrence of warthogs; occurrence of bushpigs; occurrence of giant forest hogs; occurrence of Ornithodoros tick spp.; pig population density. The following factors were considered important in describing the suitability of an area for objective 2 (‘domestic cycles’): proximity to major market centres, acting as a proxy for trade in pigs and their products; pig population density; and the occurrence of Ornithodoros tick spp.

Suitability structure

Warthogs, bushpigs and giant forest hogs are likely to have variable importance in the epidemiology of ASF in Africa, with each interacting differently with Ornithodoros spp. and pig population density in the transmission of the disease to domestic pigs [5]. The suitability of an area for the introduction of ASF from wildlife reservoirs (objective 1) was therefore considered using the hierarchical approach shown in Figure 1. For this, the likelihood of introduction of ASF from each species was assessed separately based on the interaction between wild suid habitat suitability (as a proxy for probability that the wild suid species occurs in an area), Ornithodoros spp. habitat suitability (as a proxy for the probability tick vectors occur in an area) and pig population density. The overall suitability could then be considered as the weighted average of the individual suitabilities for each wild suid species, with weights assigned based on the perceived importance of each species in the transmission of ASF to domestic pigs. The suitability of an area for sustained spread within domestic pig populations considered the interaction between pig population density, Ornithodoros spp. habitat suitability (as a proxy for the probability tick vectors occur in an area) and proximity to major market centres.

Figure 1
figure 1

Framework for the MCDA procedure.

As well as these ‘suitability’ criteria (i.e. risk factors), possible constraints on the suitability of an area for sustained transmission of ASFV through sylvatic or domestic cycles were identified. For objective 1, constraints on the suitability of an area for the introduction of ASF from each wild suid species were considered to be those areas in which warthogs, bushpigs or giant forest hogs were presumed to be entirely absent (i.e. outside the species’ usual range). For both objectives, areas of Africa in which domestic pigs are absent were considered to constrain suitability for sustained ASF transmission via either cycle.

Estimating criteria weights

Comparing risk factors

A pairwise comparison approach [37] was used to define the relative importance of each of the identified suitability criteria (risk factors) in relation to objective 1 or 2. Each criterion was compared with each of the other criteria for each objective, and assigned a score using the preference statements described in Table 1. Hence, risk factor y could be considered equally to extremely more (less) important when compared to risk factor z in relation to the objective of interest. To approximate the uncertainty that existed in these estimates of relative importance, a Betapert distribution was created for each pairwise comparison using the @Risk add-on for Microsoft Excel (@Risk for Excel, Version 5.5., Palisade Corp., Newfield, NY). The distribution was defined on the basis of the expected minimum, most likely and maximum preference value for each pairwise comparison.

Table 1 Preference statements and values used in pairwise-comparisons (after Saaty[35])

All pairwise comparisons were conducted by the lead author based on his own subjective assessment of the evidence available through literature review, and were reviewed with the co-authors who have all previously contributed peer-reviewed articles on aspects of the epidemiology of ASF in Africa. The final minimum, most likely, and maximum preference statements for each pairwise-comparison were those that were agreed by all authors.

Risk factor weights

In order to derive a weight for each suitability criterion that would correspond to its relative importance in relation to each objective, a comparison matrix was built using single values drawn from the distributions of pairwise comparisons. The resulting matrix was reciprocal, so that the pairwise-comparison for risk factor y and risk factor z was, a yz  = a yz − 1 and all of its diagonal elements were unity, so that a yz  = 1 when  y = z.

The preference values for each factor were summarised to derive a vector of weights by normalising the eigenvector associated with the maximum eigenvalue of the pairwise comparison matrix [38], so that:

a yz = a yz y 1 1 a yz for all z = 1 , 2 .... n .

and the specific weights for each factor were given by:

w y = a ^ yz

To prevent the generation of comparison matrices that could be considered inconsistent (i.e. in which the magnitude of the preference statement assigned to y vs. x and z vs. x was not consistent when comparing z vs. y), we assessed the overall consistency of each comparison matrix by calculating a ‘consistency index’ (CI) as the eigenvector of the normalised pairwise comparison matrix as:

CI = γ max p p 1

where γ max is the largest eigenvalue that can be obtained once we have its associated eigenvector and p is the number of columns of the matrix [39]. The resultant matrix specific consistency index was then divided by a randomly generated consistency index to produce a ‘consistency ratio’ (CR). Comparisons were considered to be consistent if the CR was less than 0.1 based on the system defined by Saaty [37]. All comparison matrices in which the consistency ratio was 0.1 or greater were discarded. This process was repeated over multiple iterations, with comparison matrices populated by values drawn by Monte Carlo sampling from the Betapert distribution representing each pairwise-comparison, until a large number of consistent matrices and weights (at least 3000) were generated for each risk factor for each of the two objectives. The output was considered to represent the distribution of possible weights for each factor under consideration, given the authors’ uncertainty in the factor’s importance in relation to each objective.

The process was automated using the VBA programming language (Microsoft Corp) and implemented in Microsoft Excel (2010).

Producing spatial layers

Raster maps for the suitability criteria and constraints identified through the literature review were sourced and mapped in ArcMap 10 (ESRI, Redlands, CA). The source and structure of the spatial data used to represent suitability layers are given in Table 2. In the absence of data of sufficient accuracy to represent areas in which domestic pigs do not occur, a human population density of zero per km2 (as defined by Landscan 2008, High Resolution Global Population Data Set, UT-Battelle, LLC, operator of Oak Ridge National Laboratory, US) was considered to provide the best evidence of areas in which domestic pigs cannot occur. This constraint layer and that representing the predicted home ranges of warthogs, the two bushpig species and giant forest hogs (as sourced from the African Mammals Databank (AMD) [40]), were converted into Boolean functions of ‘suitable’ and ‘unsuitable’ and assigned a value of 1 or 0, respectively.

Table 2 Data and standardisation approaches to derive suitability layers for the MCDA procedure

Continuous values from each raster layer representing the suitability criteria (Table 2) and the Boolean constraints were extracted from each cell of a grid with a cell size 0.8 × 0.8 decimal degrees over the whole of mainland Africa and exported directly into Microsoft Excel, resulting in a dataset consisting of approximately 360,000 individual data points for each of the criteria. Suitability values were standardised to a common monotonic linear scale between 0 and 1 using the maximum score procedure [39], so that for linearly increasing criteria:

x ' ij = x ij x jmax

And for linearly decreasing criteria:

x ' ij = 1 x ij x jmax

Where x ij is the raw score for criterion j and x jmax   represents the point at which its contribution to predictions of suitability becomes zero: all values above x jmax were assigned a value of 1 for criteria that increased linearly or 0 for criteria that decreased linearly. Where uncertainty existed in the appropriate value of, x jmax a uniform distribution was defined using @risk in which maximum and minimum values for x jmax were described by the 99th and 90th percentile of the distribution of values of each variable within the areas defined as suitable within the constraints for each cycle (Table 2).

Cartographic modelling

In order to derive an estimate of suitability, S, for each sampled point i in relation to each objective, the standardised suitability values and the weight for each criterion derived from the pairwise-comparison procedure (w) were combined using weighted linear combination (WLC) [39], so that:

S = w j x ij c i

where c i represents constraint layers coded as 1 or 0 to describe suitability or unsuitability, respectively.

To incorporate the full range of possible combinations of w for each criterion, as well as the uncertainty associated with the standardisation of suitability layers, the process was repeated over 3000 iterations, with a specific weight combination randomly selected by Monte Carlo sampling for each iteration. Hence, each sampled point was associated with 3000 possible estimates of suitability (S) for each objective based on the linear combination of values of x. Three thousand iterations were used to allow the incorporation of the full range of weights (w) generated through the iterative pairwise comparison procedure (n = 3000).

The mean value and the 5th and 95th percentiles were extracted from the resulting distributions and exported to ArcMap where point values were converted directly to a continuous raster surface for the visual presentation of results.

To enable formal comparisons of the geographic coverage of areas of predicted suitability for persistence as part of either cycle, we defined a fixed suitability threshold of 0.5 on a 0 to 1 scale [41]. The total land surface area with suitability predictions above this threshold for the whole of Africa and for each of the UN sub-regions of Africa (Northern, Western, Central, Eastern, and Southern Africa) was calculated using the zonal statistics tool in ArcMap and the Lambert Azimuthal Equal Area projection.

Validation

ASF outbreak report data from 2005 to September 2012 for mainland Africa were downloaded from the World Animal Health organisation WAHID system (OIE, 2012). When geographic co-ordinates for the location of a reported outbreak were not provided but a location name was given, ‘Fuzzy Gazetteer’ (isodp.hof-university.de/fuzzyg) was used to select the most likely co-ordinates for the location of the outbreak.

Validation was performed on the mean suitability estimate for sylvatic cycles, domestic cycles, and a combination of the two. Given the non-independence of predictions for each objective (which were defined using some of the same criteria but with different weightings applied), the combined suitability estimate was derived by combining the mean value for each cycle using an ordered approach where the highest value for either objective was used to define the value at each pixel.

A partial ROC analysis was implemented on the combined suitability estimates and the mean estimates for each cycle using the Partial-ROC tool developed by Barve [42]. Only suitability estimates from countries in which outbreaks had been reported to the OIE were used for validation. The partial ROC is a plot of sensitivity (proportion of correctly predicted disease cases) against the proportion of study area predicted as being suitable for disease occurrence. Hence, the approach does not rely on absence data, which are difficult to generate for ASF for much of Africa, where animal disease surveillance systems are often inadequate. Moreover, the approach allows for the inclusion of a user generated error term (E) to account for the amount of error admissible along the true-positives axis (the omission error) [43]. The latitude and longitude describing the geographical location of ASF outbreaks reported to the OIE is typically given at one or two decimal places, and represents the focus of an outbreak that can cover a wide geographic area: the validation dataset can therefore be expected to contain substantial amounts of geo-referencing based error that could contribute to omission error.

The partial ROC output is the ratio of the area under the curve (AUC) of the restricted ROC curve (i.e. the region where the omission error is less than E) against the AUC of the restricted null model. Bootstrapping to evaluate the statistical significance of the AUC ratios was performed by resampling 50 % of test points 1000 times from the pool of outbreak data, as described by Peterson et al. [43]. The value of E was set at 20 % to represent the likely high levels of geo-referencing error present in the validation set.

Results

Multi-criteria decision analysis

Criteria weights

Objective 1

The pairwise-comparisons used for objective 1 are shown in Table 3 and the resulting weightings for each criterion in Table 4.

Table 3 Minimum, most likely and maximum values for the pairwise comparison of criteria for objective 1
Table 4 Distributional estimates of weights for criteria for objective 1 derived from the pairwise comparisons in Table  3 (see text)

The bushpigs and giant forest hogs show a similar pattern of suitability weightings, with habitat suitability considered to be the most important factor in determining the suitability of an area for repeated introduction of ASF from these species, followed by pig density with relatively minor contributions from tick habitat suitability. Tick habitat suitability was considered to be substantially more important in the suitability of an area for sustained transmission of ASF to domestic pigs from warthogs.

The preference values used to describe the general importance of each wild suid species in acting as a reservoir of ASF for domestic pigs are also shown in Table 3, and the associated criterion weights in Table 4. Warthogs were considered to be the most important reservoir species for ASF for domestic pigs, followed by the bushpigs. Giant forest hogs were considered to be comparatively unimportant.

Objective 2

The preference statements from the pairwise estimation of the relative importance of each criterion in relation to objective 2 and the associated weightings are presented in Table 5 and Table 6, respectively. Pig population density was considered to be the most important factor influencing the sustained spread of ASF within domestic pig populations, with proximity to market centres and tick habitat suitability making smaller contributions.

Table 5 Minimum, most likely and maximum values for the pairwise comparison of criteria for objective 2
Table 6 Distributional estimates of weights for criteria for objective 2 derived from the pairwise comparisons in Table  5 (see text)

Cartographic modelling

Objective 1

Areas with the highest estimated suitability for repeated introduction of ASF into domestic pig populations from sylvatic reservoirs were predicted in East and Central sub-Saharan Africa, with sustained sylvatic transmission appearing to be relatively unlikely in West Africa, as well as the vast majority of northern and south-western Africa (Figure 2 and Table 7).

Figure 2
figure 2

Suitability for ASF persistence as part of sylvatic cycles (5 th percentile (left); average (middle); 95 th percentile (right)).

Table 7 Total predicted land area above a 0.5 suitability threshold for the distributional estimates (5 th percentile, mean and 95 % percentile) for each transmission cycle in the UN sub-regions of Africa

The southern Democratic Republic of Congo (DRC), Burundi, Rwanda, eastern Zambia, Malawi, Tanzania, parts of Kenya, Uganda, Zimbabwe, Mozambique and the north-east of South Africa all appear to have areas of relatively high potential suitability for repeated introduction of ASF to domestic pigs from sylvatic cycles.

Objective 2

Areas with apparently high suitability for cycles involving sustained spread of ASF within domestic pig population pathways can be found in all regions of Africa excluding Northern Africa (Figure 3 and Table 7). Countries with the highest suitability for sustained spread within domestic pig populations were predicted to be in South Africa, Malawi, Nigeria, Angola, Cameroon, Burkina Faso, Benin, Togo, Guinea-Bissau, Rwanda and Burundi, as well as smaller parts of Uganda and the DRC.

Figure 3
figure 3

Suitability for ASF persistence as part of domestic cycles (5 th percentile (left); average (middle); 95 th percentile (right)).

Validation

A total of 252 primary ASF outbreak reports from 20 countries were identified. The combined suitability estimates showed a very high degree of visual agreement with the outbreaks reported to the OIE (Figure 4). The average partial ROC AUC ratio for the combined sylvatic and domestic suitability estimate was 1.28 (range 1.20 – 1.36), where 0 of 1000 iterations resulted in a ROC AUC ratio ≤1, suggesting a consistently good degree of model predictability for the reported ASF outbreaks. The average AUC ratio when considering mean domestic suitability alone was 1.46 (range 1.30 – 1.62) and was 1.08 (range 1.01 – 1.14) for sylvatic cycles.

Figure 4
figure 4

Combined estimates of mean suitability for ASF endemicity in domestic pig populations, with location of ASF outbreaks between 2005 and 2012 overlaid (right).

Discussion

To our knowledge, this is the first attempt to predict the spatial distribution of ASF in Africa using a standardised approach. The combined suitability estimates derived from the MCDA procedure reveal a good degree of agreement with the distribution of cases reported to the OIE between 2005 and 2012, although the number of officially reported ASF cases are likely to represent only a small proportion of those that actually occur across Africa, particularly in areas in which the disease is endemic. The predicted distribution of areas with high suitability for repeated introduction of ASF from wildlife reservoirs and for sustained transmission within pig populations also supports the anecdotal distribution of sylvatic and domestic pig cycles identified by a number of authors [1, 2, 5, 44].

Outputs from the MCDA procedure suggest that most of western, south-western, and west-central Africa are relatively unsuitable for repeated introduction of ASF from wildlife reservoirs. African swine fever is endemic or approaching a situation of endemicity in several countries in these areas, including Nigeria [45], Senegal [31], Guinea Bissau [46], and Angola [47]. There are areas of widespread suitability for domestic ASF transmission in these countries, and it is likely that the persistence of ASF in such areas is the result of sustained domestic pig transmission cycles, without the need for repeated introduction from wildlife reservoirs [5, 20].

Countries that are predicted to have widespread suitability for repeated introduction from sylvatic reservoirs are Tanzania, Zimbabwe, and Malawi. The southern DRC, eastern parts of Zambia, and smaller areas of Uganda, Kenya, Angola, Botswana, South Africa, Burundi, Rwanda and Mozambique also appear to be highly suitable for ASF transmission associated with sylvatic cycles. Despite this apparently high suitability, the majority of recent ASF outbreaks in these regions are likely to have been caused by the movement of pigs and their products rather than sylvatic spill over [4850]. We also predict that all of these countries, except Botswana, have a widespread distribution of areas of apparent high suitability for sustained transmission as part of domestic pig cycles. This is particularly the case in Angola, Malawi, Uganda, South Africa and the western DRC. The potential importance of transmission as part of domestic rather than sylvatic cycles in these areas, and continental Africa as a whole, is supported by the considerably higher predictive ability (based on average AUC ratio) of domestic suitability estimates for all areas in which ASF outbreaks have been reported compared to the combined suitability estimates and those for sylvatic cycles alone. However, sporadic introduction from wildlife reservoirs should not be ignored as potential risk in such areas [44, 51]. Indeed, although bushpigs were given a moderate weight in this study, recent work implicated P. larvatus as a possible source of ASF infection for domestic pigs living in close proximity to a National park in Kenya [52]. African swine fever viruses isolated from warthogs living on a cattle ranch in central Kenya were also shown to be genetically similar to those causing recent outbreaks in Kenya and Uganda [53]. Hence, it is quite possible that sporadic introduction of ASFV from sylvatic reservoirs could lead to onward transmission via domestic cycles, particularly in those areas of Central, East and Southern Africa where we predict high suitability for both transmission routes.

Through the use of a stochastic approach, this study has attempted to incorporate uncertainty into several of the key decision making steps in the MCDA procedure, and particularly into the assignment of relative importance scores to risk factor layers. The BetaPert distribution has been widely used for modelling expert opinion [54], and allowed us to incorporate the uncertainty that existed in our subjective assessment of the available evidence. To our knowledge, this is the first example of the formal introduction of stochasticity into a spatial MCDA in the area of disease or health mapping. Such a step is particularly important in the knowledge-driven mapping of diseases such as ASF where, in the absence of quantitative epidemiological studies, the contribution of several putative risk factors is unclear [14, 16] and, as in this study, a highly subjective approach is used to assess this contribution. The overall geographic range of suitability for repeated transmission of ASFV as part of the domestic or sylvatic cycle was generally the same across the distribution of estimates. However, the application of a somewhat arbitrary (but commonly used [41]) threshold of 0.5 to dichotomise the output into unsuitable and suitable reveals the extent and impact of our uncertainty on the percentage coverage of areas of potential suitability for either cycle.

Pairwise-comparison is a method that has been applied widely to assess the importance of one factor over another in a decision-making process [38, 55], and has been used previously for disease mapping using spatial MCDM [16, 17]. An important issue in the pairwise-comparison procedure is that matrices are often inconsistent, potentially leading to senseless decision making [56]. Through the use of an iterative approach, in which the pairwise-comparison matrix is populated by values drawn at random from a distribution defining each pairwise-comparison, it is likely that the frequency of inconsistency in the construction of the matrix and resulting estimation of factor weights was increased in this study. Although the rule described by Saaty [37] was used to combat such inconsistency, the validity of the 10 % cut-off in the resultant consistency ratio has been questioned [57]. Kwiesielewicz and Uden [58] showed that a matrix that passes a consistency test successfully may still be contradictory. Hence, it is acknowledged that, on the basis of the stochastic approach adopted, some of the allocated weights selected from the distributional ranges used may have been the result of inconsistent comparisons. However, given that the resultant risk maps are the summarised estimates of a large number of independent weight combinations, the impact of any such inconsistency is likely to be small.

It is important that outputs from knowledge-driven risk mapping are interpreted in the light of the methodology adopted, which remains highly subjective and combines data sources of variable quality, most of which originated from internet-based data repositories. Moreover, the approach can only incorporate those risk factors that can be mapped. Trade of pigs and their products is likely to be a major mechanism by which ASFV spreads within and between regions [1]. The movement of middle-men between farms, for example, has been reported as an important risk factor in the spread of ASFV in Africa [8], but such activities cannot be mapped directly in the absence of spatial data describing such movements. Proximity to major market centres was used as a proxy for this, and other such trade-related activities, but the use of such a variable for this purpose cannot be expected to capture the local scale variability in the movement of animals and contaminated people, equipment and products that may contribute to virus spread.

The MCDA approach relies on knowledge that already exists: to date, there have been few quantitative epidemiological studies of ASF in Africa, and whilst MCDA provides a useful framework with which to aggregate the work that has been conducted, it remains limited by the scope of the available evidence. Given the limited consensus on risk factors for ASF in the literature, our model used limited number of criteria: as additional studies on the epidemiology of ASF in Africa are conducted, the criteria to describe ASF risk and in particular the perceived relative importance of these criteria, could be expected to change [20].

Moreover, this study is an interpretation of the existing knowledge and relationships between these risk factors by the authors only, since no other experts were consulted. Whilst we attempted to limit the potential impact of our subjective assessments through modelling the relative importance of criterion using probability distributions, a different set of experts may have chosen different minimum, most likely and maximum values for the BetaBert distributions, and thus derived somewhat different suitability estimates. The predictions from this work should be should be interpreted accordingly, but it would be reasonable to expect, given the current state of knowledge on ASF in Africa, that another set of experts’ average suitability estimates using the same set of criteria would be likely to fall between the 5th and 95th percentiles of our own.

Conclusion

This study used a limited number of known or hypothesised risk factors for ASF and our own subjective assessment of the importance of those risk factors to predict the distribution of ASF persistence via sylvatic and domestic transmission cycles in Africa. The suitability predictions derived are not intended to be interpreted as definitive, instead, they provide a ‘best guess’ estimate based on the currently available evidence, and our own interpretation of that evidence. With these limitations in mind, outputs from this study indicate that areas of high suitability for transmission as part of domestic cycles occur throughout sub-Saharan Africa. Areas of sylvatic suitability are restricted to Central, East and Southern Africa, but commonly overlap areas of suitability for domestic cycles. Hence, whilst domestic transmission of ASF is likely to be the major means by which the disease persists throughout sub-Saharan Africa, and should probably continue to be the focus of disease control and prevention efforts, the continued potential for spill over from wildlife reservoirs should not be ignored.

Abbreviations

AMD:

African Mammals Databank

AUC:

Area under the curve

ASF:

African swine fever

ASFV:

African swine fever virus

CI:

Consistency index

CR:

Consistency ratio

DRC:

Democratic Republic of Congo

GFH:

Giant Forest Hog

GIS:

Geographic information system

MCDA:

Multi-criteria decision analysis

OIE:

World Organisation for Animal Health

ROC:

Receiver operating characteristic

WAHID:

World Animal Health Information Database

WLC:

Weighted linear combination.

References

  1. Costard S, Wieland B, de Glanville W, Jori F, Rowlands R, Vosloo W, Roger F, Pfeiffer DU, Dixon LK: African swine fever: how can global spread be prevented?. Philos Trans R Soc Lond B Biol Sci. 2009, 364: 2683-2696. 10.1098/rstb.2009.0098.

    Article  PubMed Central  PubMed  Google Scholar 

  2. Penrith ML, Vosloo W: Review of African swine fever: transmission, spread and control. J S Afr Vet Assoc. 2009, 80: 58-62.

    Article  CAS  PubMed  Google Scholar 

  3. el Hicheri K, Gomez-Tejedor C, Penrith ML, Davies G, Douati A, Edoukou GJ, Wojciechowski K: The 1996 epizootic of African swine fever in the Ivory Coast. Rev - Off Int Epizoot. 1998, 17: 660-673.

    CAS  PubMed  Google Scholar 

  4. Roger F, Ratovonjato J, Vola P, Uilenber G: Ornithodoros porcinus ticks, bushpigs, and African swine fever in Madagascar. Exp Appl Acarol. 2001, 25: 263-269. 10.1023/A:1010687502145.

    Article  CAS  PubMed  Google Scholar 

  5. Jori F, Bastos ADS: Role of wild suids in the epidemiology of African swine fever. Ecohealth. 2009, 6: 296-310. 10.1007/s10393-009-0248-7.

    Article  PubMed  Google Scholar 

  6. Babalobi OO, Olugasa BO, Oluwayelu DO, Ijagbone IF, Ayoade GO, Agbede SA: Analysis and evaluation of mortality losses of the 2001 African swine fever outbreak, Ibadan, Nigeria. Trop Anim Health Prod. 2007, 39: 533-542. 10.1007/s11250-007-9038-9.

    Article  CAS  PubMed  Google Scholar 

  7. Costard S, Mur L, Lubroth J, Sánchez-Vizcaíno JM, Pfeiffer DU: Epidemiology of African swine fever virus. Virus Res. 2013, 173: 191-197. 10.1016/j.virusres.2012.10.030.

    Article  CAS  PubMed  Google Scholar 

  8. Olugasa BO, Ijagbone IF: Pattern of spread of African swine fever in south-western Nigeria, 1997–2005. Vet Ital. 2007, 43: 621-628.

    PubMed  Google Scholar 

  9. Blanton JD, Manangan A, Manangan J, Hanlon CA, Slate D, Rupprecht CE: Development of a GIS-based, real-time Internet mapping tool for rabies surveillance. Int J Health Geogr. 2006, 5: 47-10.1186/1476-072X-5-47.

    Article  PubMed Central  PubMed  Google Scholar 

  10. Clements ACA, Pfeiffer DU: Emerging viral zoonoses: frameworks for spatial and spatiotemporal risk assessment and resource planning. Vet J. 2009, 182: 21-30. 10.1016/j.tvjl.2008.05.010.

    Article  PubMed  Google Scholar 

  11. Elnaiem D-EA, Schorscher J, Bendall A, Obsomer V, Osman ME, Mekkawi AM, Connor SJ, Ashford RW, Thomson MC: Risk mapping of visceral leishmaniasis: the role of local variation in rainfall and altitude on the presence and incidence of kala-azar in eastern Sudan. Am J Trop Med Hyg. 2003, 68: 10-17.

    PubMed  Google Scholar 

  12. Haque U, Magalhães RJS, Reid HL, Clements ACA, Ahmed SM, Islam A, Yamamoto T, Haque R, Glass GE: Spatial prediction of malaria prevalence in an endemic area of Bangladesh. Malaria J. 2010, 9: 120-10.1186/1475-2875-9-120.

    Article  Google Scholar 

  13. Ortiz-Pelaez A, Pfeiffer DU, Tempia S, Otieno FT, Aden HH, Costagli R: Risk mapping of Rinderpest sero-prevalence in Central and Southern Somalia based on spatial and network risk factors. BMC Vet Res. 2010, 6: 22-10.1186/1746-6148-6-22.

    Article  PubMed Central  PubMed  Google Scholar 

  14. Clements ACA, Pfeiffer DU, Martin V: Application of knowledge-driven spatial modelling approaches and uncertainty management to a study of Rift Valley fever in Africa. Int J Health Geogr. 2006, 5: 57-10.1186/1476-072X-5-57.

    Article  PubMed Central  PubMed  Google Scholar 

  15. Stevens KB, Gilbert M, Pfeiffer DU: Modeling habitat suitability for occurrence of highly pathogenic avian influenza virus H5N1 in domestic poultry in Asia: a spatial multicriteria decision analysis approach. Spat Spatiotemporal Epidemiol. 2013, 4: 1-14.

    Article  PubMed  Google Scholar 

  16. Stevens KB, Pfeiffer DU: Spatial modelling of disease using data- and knowledge-driven approaches. Spat Spatiotemporal Epidemiol. 2011, 2: 125-133. 10.1016/j.sste.2011.07.007.

    Article  PubMed  Google Scholar 

  17. Hongoh V, Hoen AG, Aenishaenslin C, Waaub J-P, Bélanger D, Michel P, Lyme-MCDA Consortium: Spatially explicit multi-criteria decision analysis for managing vector-borne diseases. Int J Health Geogr. 2011, 10: 70-10.1186/1476-072X-10-70.

    Article  PubMed Central  PubMed  Google Scholar 

  18. Symeonakis E, Robinson T, Drake N: GIS and multiple-criteria evaluation for the optimisation of tsetse fly eradication programmes. Environ Monit Assess. 2007, 124: 89-103. 10.1007/s10661-006-9210-0.

    Article  PubMed  Google Scholar 

  19. Store R, Kangas J: Integrating spatial multi-criteria evaluation and expert knowledge for GIS-based habitat suitability modelling. Landsc Urban Plann. 2001, 55: 79-93. 10.1016/S0169-2046(01)00120-7.

    Article  Google Scholar 

  20. Jori F, Vial L, Penrith ML, Pérez-Sánchez R, Etter E, Albina E, Michaud V, Roger F: Review of the sylvatic cycle of African swine fever in sub-Saharan Africa and the Indian ocean. Virus Res. 2013, 173: 212-227. 10.1016/j.virusres.2012.10.005.

    Article  CAS  PubMed  Google Scholar 

  21. Thomson GR: The epidemiology of African swine fever: the role of free-living hosts in Africa. Onderstepoort J Vet Res. 1985, 52: 201-209.

    CAS  PubMed  Google Scholar 

  22. Thomson GR, Gainaru MD, Van Dellen AF: Experimental infection of warthogs (Phacochoerus aethiopicus) with African swine fever virus. Onderstepoort J Vet Res. 1980, 47: 19-22.

    CAS  PubMed  Google Scholar 

  23. McCall PJ, Hume JCC, Motshegwa K, Pignatelli P, Talbert A, Kisinza W: Does tick-borne relapsing fever have an animal reservoir in East Africa?. Vector Borne Zoonotic Dis. 2007, 7: 659-666. 10.1089/vbz.2007.0151.

    Article  CAS  PubMed  Google Scholar 

  24. Rennie L, Wilkinson PJ, Mellor PS: Transovarial transmission of African swine fever virus in the argasid tick Ornithodoros moubata. Med Vet Entomol. 2001, 15: 140-146. 10.1046/j.1365-2915.2001.00282.x.

    Article  CAS  PubMed  Google Scholar 

  25. Wardley RC, De M, Andrade C, Black DN, De Castro Portugal FL, Enjuanes L, Hess WR, Mebus C, Ordas A, Rutili D, Sanchez Vizcaino J, Vigario JD, Wilkinson PJ, Moura Nunes JF, Thomson G: African swine fever virus. Arch Virol. 1983, 76: 73-90. 10.1007/BF01311692.

    Article  CAS  PubMed  Google Scholar 

  26. Pérez-Sánchez R, Astigarraga A, Oleaga-Pérez A, Encinas-Grandes A: Relationship between the persistence of African swine fever and the distribution of Ornithodoros erraticus in the province of Salamanca, Spain. Vet Rec. 1994, 135: 207-209. 10.1136/vr.135.9.207.

    Article  PubMed  Google Scholar 

  27. Plowright W, Parker J, Pierce MA: African Swine fever virus in ticks (Ornithodorus moutaba) collected from animal burrows in Tanzania. Nature. 1969, 221: 1071-1073. 10.1038/2211071a0.

    Article  CAS  PubMed  Google Scholar 

  28. Theiler G, Robinson BN: Tick survey (Part VIII) Checklists of ticks recorded from the Belgian Congo and Ruanda Urundi, from Angola and Northern Rhodesia. Onderstepoort J Vet Res. 1954, 26: 447-461.

    Google Scholar 

  29. Van der Merwe S: Some remarks on the ‘tampans’ of the Ornithodoros moubata complex in Southern Africa. Zool Anz. 1968, 181: 280-289.

    Google Scholar 

  30. Haresnape JM, Mamu FD: The distribution of ticks of the Ornithodoros moubata complex (Ixodoidea: Argasidae) in Malawi, and its relation to African swine fever epizootiology. J Hyg (Lond). 1986, 96: 535-544. 10.1017/S0022172400066341.

    Article  CAS  Google Scholar 

  31. Vial L, Wieland B, Jori F, Etter E, Dixon L, Roger F: African swine fever virus DNA in soft ticks, Senegal. Emerg Infect Dis. 2007, 13: 1928-1931. 10.3201/eid1312.071022.

    Article  PubMed Central  PubMed  Google Scholar 

  32. Gulenkin VM, Korennoy FI, Karaulov AK, Dudnikov SA: Cartographical analysis of African swine fever outbreaks in the territory of the Russian Federation and computer modeling of the basic reproduction ratio. Prev Vet Med. 2011, 102: 167-174. 10.1016/j.prevetmed.2011.07.004.

    Article  CAS  PubMed  Google Scholar 

  33. Allepuz A, Saez M, Alba A, Napp S, Casal J: Exploratory spatial analysis of Aujeszky's disease during four phases of the eradication programme in Catalonia, Spain (2003–2007). Prev Vet Med. 2008, 86: 164-175. 10.1016/j.prevetmed.2008.04.005.

    Article  CAS  PubMed  Google Scholar 

  34. Poljak Z, Dewey CE, Martin SW, Christensen J, Carman S, Friendship RM: Prevalence of and risk factors for influenza in southern Ontario swine herds in 2001 and 2003. Can J Vet Res. 2008, 72: 7-17.

    PubMed Central  PubMed  Google Scholar 

  35. Rhyan JC, Spraker TR: Emergence of diseases from wildlife reservoirs. Vet Pathol. 2010, 47: 34-39. 10.1177/0300985809354466.

    Article  CAS  PubMed  Google Scholar 

  36. Fasina FO, Agbaje M, Ajani FL, Talabi OA, Lazarus DD, Gallardo C, Thompson PN, Bastos ADS: Risk factors for farm-level African swine fever infection in major pig-producing areas in Nigeria, 1997–2011. Prev Vet Med. 2012, 107: 65-75. 10.1016/j.prevetmed.2012.05.011.

    Article  CAS  PubMed  Google Scholar 

  37. Saaty TL: The Analytic Hierarchy Process. New York: McGraw-Hill; 1980.

    Google Scholar 

  38. Boroushaki S, Malczewski J: Implementing an extension of the analytical hierarchy process using ordered weighted averaging operators with fuzzy quantifiers in ArcGIS. Comput Geosci. 2008, 34: 399-410. 10.1016/j.cageo.2007.04.003.

    Article  Google Scholar 

  39. Malczewski J: GIS and Multicriteria Decision Analysis. New York: John Wiley and Sons; 1999.

    Google Scholar 

  40. Boitani L, Corsi F, De Biase A, Carranza I, Ravagli M, Reggiani G, Sinibaldi I, Trapanese P: AMD African mammals databank. A databank for conservation and management of the African mammals. Roma: Instituto di Ecologia Applicata: Report to the Directorate General for Development (GDVIII/A/1) of the European Commission; 1999.

    Google Scholar 

  41. Liu C, Berry P, Dawson T, Pearson R: Selecting thresholds of occurrence in the prediction of species distributions. Ecography. 2005, 28: 385-393. 10.1111/j.0906-7590.2005.03957.x.

    Article  Google Scholar 

  42. Barve N: Tool for Partial-ROC. 2008, Biodiversity Institute: Version 1. Lawrence, KS, Available: http://kuscholarworks.ku.edu/dspace/handle/1808/10059

    Google Scholar 

  43. Peterson AT, Papeş M, Soberón J: Rethinking receiver operating characteristic analysis applications in ecological niche modeling. Ecol Model. 2008, 213: 63-72. 10.1016/j.ecolmodel.2007.11.008.

    Article  Google Scholar 

  44. Penrith ML, Vosloo W, Jori F, Bastos ADS: African swine fever virus eradication in Africa. Virus Res. 2013, 173: 228-246. 10.1016/j.virusres.2012.10.011.

    Article  CAS  PubMed  Google Scholar 

  45. Otesile EB, Ajuwape AT, Odemuyiwa SO, Akpavie SO, Olaifa AK, Odaibo GN, Olaleye DO, Adetosoye AI: Field and experimental investigations of an outbreak of African swine fever in Nigeria. Revue d'élevage et de médecine vétérinaire des pays tropicaux. 2005, 58: 21-26.

    Google Scholar 

  46. Penrith ML, Bastos ADS, Thomson GR, Wilkinson PJ: Epidemiology of African Swine Fever in West Africa. Melborne, Australia: Proceedings of the 16th Biennial Congress of the International Pig Veterinary Society; 2000:560.

    Google Scholar 

  47. Plowright W, Thomson GR: African Swine Fever. Infectious Diseases of Livestock, with Special Reference to Southern Africa. Edited by: Coetzer JAW, Thomson GR, Tustin RC. Cape Town: Oxford University Press; 1994.

    Google Scholar 

  48. Atuhaire DK, Ochwo S, Afayoa M, Mwiine FN, Kokas I, Arinaitwe E, Ademun-Okurut RA, Okuni JB, Nanteza A, Ayebazibwe C, Okedi L, Olaho-Mukani W, Ojok L: Epidemiological Overview of African Swine Fever in Uganda (2001–2012). J Vet Med. 2013, doi:10.1155/2013/94963847

    Google Scholar 

  49. Gallardo C, Ademun AR, Nieto R, Nantima N, Arias M, Martín E, Pelayo V, Bishop RP: Genotyping of African swine fever virus (ASFV) isolates associated with disease outbreaks in Uganda in 2007. Afr J Biotechnol. 2011, 10: 3488-3497.

    CAS  Google Scholar 

  50. Misinzo G, Kasanga CJ, Mpelumbe-Ngeleja C, Masambu J, Kitambi A, Van Doorsselaere J: African swine fever virus, Tanzania, 2010–2012. Emerg Infect Dis. 2012, 18: 2081-2083. 10.3201/eid1812.121083.

    Article  PubMed Central  PubMed  Google Scholar 

  51. Lubisi BA, Bastos ADS, Dwarka RM, Vosloo W: Molecular epidemiology of African swine fever in East Africa. Arch Virol. 2005, 150: 2439-2452. 10.1007/s00705-005-0602-1.

    Article  CAS  PubMed  Google Scholar 

  52. Okoth E, Gallardo C, Macharia JM, Omore A, Pelayo V, Bulimo DW, Arias M, Kitala P, Baboon K, Lekolol I, Mijele D, Bishop RP: Comparison of African swine fever virus prevalence and risk in two contrasting pig-farming systems in South-west and Central Kenya. Prev Vet Med. 2013, 1: 198-205.

    Article  Google Scholar 

  53. Gallardo C, Okoth E, Pelayo V, Anchuelo R, Martín E, Simón A, Llorente A, Nieto R, Soler A, Martín R, Arias M, Bishop RP: African swine fever viruses with two different genotypes, both of which occur in domestic pigs, are associated with ticks and adult warthogs, respectively, at a single geographical site. J Gen Virol. 2011, 92: 432-444. 10.1099/vir.0.025874-0.

    Article  CAS  PubMed  Google Scholar 

  54. Vose D: Risk Analysis: a Quantitative Guide. 3rd edition. Chichester, West Sussex: John Wiley and Sons; 2008.

    Google Scholar 

  55. Vaidya OS, Kumar S: Analytic hierarchy process: an overview of applications. Eur J Oper Res. 2006, 169: 1-29. 10.1016/j.ejor.2004.04.028.

    Article  Google Scholar 

  56. Bozóki S, Rapcsák T: On Saaty’s and Koczkodaj’s inconsistencies of pairwise comparison matrices. J Glob Optim. 2007, 42: 157-175.

    Article  Google Scholar 

  57. Koczkodaj WW: A new definition of consistency of pairwise comparisons. Math Comput Model. 1993, 18: 79-84.

    Article  Google Scholar 

  58. Kwiesielewicz M, van Uden E: Inconsistent and contradictory judgements in pairwise comparison method in the AHP. Comput OR. 2004, 31: 713-719. 10.1016/S0305-0548(03)00022-4.

    Article  Google Scholar 

Download references

Acknowledgements

This work was funded by the ASFRISK project; “Evaluating and controlling the risk of African swine fever in the EU” (EC, FP7-KBBE-2007-1, Project #211691). The funder had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. We thank Kim Stevens for discussions on methodology.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to William A de Glanville.

Additional information

Competing interests

No author has any competing interest to declare.

Authors’ contributions

WAdeG, SC, BW and DUP designed the study. WAdeG performed data analysis as part of the study. All authors contributed to the development of risk factor weightings. All authors made contributions to the conception, design, and revision of the manuscript. All authors read and approved the final manuscript.

Authors’ original submitted files for images

Rights and permissions

This article is published under license to BioMed Central Ltd. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article

de Glanville, W.A., Vial, L., Costard, S. et al. Spatial multi-criteria decision analysis to predict suitability for African swine fever endemicity in Africa. BMC Vet Res 10, 9 (2014). https://0-doi-org.brum.beds.ac.uk/10.1186/1746-6148-10-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://0-doi-org.brum.beds.ac.uk/10.1186/1746-6148-10-9

Keywords