â–¡ .â–¡
Fraction of background predicted
Omission on training samples
Omission on test samples
Predicted omission
10
20
30
70
80
90
100
40 50 60
Cum Illative til res It old
Figure C18. An evaluation of omission error rates for training (dark blue line) and test (light blue line) data as a function of the
cumulative threshold and overall predicted area. The red line indicates the overall fraction of the map area fitting each value of
the cumulative threshold. The black line is the predicted omission rate for each cumulative threshold.
Appendix C  25
Sens
itivity
vs. Specificity for
Sprag
ue_s_
Pipit
1.D
D.9
O.S
I0.7
Â£=
SO.6
E
;go.4
I0.3
0.2
â–¡ .1
a.u
1 1 1
â€” 
f
/
I
/
\
/
/
/
/
/
/
/
/
Training data (AUC = 0.988)
Test data (AUG =0.976)
Random Prediction (AUC = 0.5)
0.0
0.1
0.:
0.9
1.0
0.3 0.4 0.5 0.6 0.7 0.8
G3ecifi:ity (1  Fractional Predicted Area)
Figure C19. Receiver Operating Characteristic (ROC) curve evaluating the overall predictive power of the model with the Area
Under the Curve (AUC). The AUC value indicates that when two random locations are chosen the model has that probability
of assigning a higher cumulative threshold value to the location with more suitable habitat. The light blue line indicates how a
neutral or random model would perform (i.e., it only has a 50% probability of assigning a higher cumulative threshold value to
a random location with more suitable habitat than a random location with less suitable habitat). The further toward the top left
of the graph the training (red) and test (blue) data lines are, the better the model is at predicting the presences contained in the
training data. Sensitivity (plotted on the yaxis) is the proportion of positive locations that were correctly classified by the model.
Sensitivity is also known as the true positive rate and can be thought of as the degree of absence of omission errors. Specificity is
the proportion of random locations chosen from the background (these pseudoabsences are used instead of true negative loca
tions) that were correctly classified by the model as negative. One minus the Specificity (plotted on the xaxis) is known as the
false positive rate and represents the commission error rate.
Appendix C  26
Log response of Sprague_5_Plpltto aspect90m_mtS3cllp
TO
aspect9(]m_mtB3clii
Log response of Sprague_5_Pipit to nlGd90m_mtS3
Hh i
t t
I
ir
Log res
pc
nse
of Sprag
ue
s Pi
3i
to
statsgo90m
mt83
clip
1
Log response of Sprague
Pipit to curve planSOir
mfSZ
1.0
a A
Log respo
nse
of Sprague s Pipit to elevSOm
mtS3clip
\
\l
1
\
Log res
ponse
cuive_
of Sprague
ilan90m_mtB3
s Pipit to precip
annSOm mtS3
a, 3
^ 0.2
I""
511,1
D3
QA
Log response of Sprague s Pipit to slopes
m mtSSclip
Log response Of Sprague s Pipit to tmax90m mt83
:i
â–¡5
I
^
Â£'
s ""
i 35
\
\
.411
\
\
""
s
Log resp
Elope9(]m_m183clip
3nse of Sprague s Pipit to tminSOm mtS^clip
\
~
L
â– ~^
^
â– ^^
Log resp
onse
of Sprague
s Pipit to geo
mbrr
g90mt83clJF
!
1
_
_
_
_
_
_
_
_
_
_
_
_
_
Log respo
nseo
Sprague s
Pipit to so
il tmpSOm
mtS3
msi(3am_mt83
Figure C20. Response curves for individual environmental variables showing how the logistic prediction changes as each environmental variable is varied while all other envi
ronmental variables are held constant at their average sample values. The value on the yaxis is predicted probability of suitable conditions as given by the logistic formula P(x)
= exp(cl * fl(x) + c2 *f2(x) + c3 *f3(x)...) / Z. Note that if any of the environmental variables are correlated, the marginal response curves can be misleading (e.g., two highly
correlated variables with opposite response curves could effectively cancel each other out). Value definitions and/or links to metadata containing these definitions can be found in
the Descriptions of Environmental Input Layers section of the appendix above.
tmin9Dm_mte3clip
tmax90m_mt83
statsgo90m_mt83_clip
soil_tmp90m_mt83
slope9Dm_mt83clip
piecip_ann90m_mt83
nlcd90m_mt83
geombmg90mt83clip
elev9Dm_mt83clip
c u ive_p I a n 9 m_mt8 3
aspect90m_mt83clip
Q.Q
Jackknife of Training gain for SpraguesPipit
Without variable
With only variable
With all variables
0.5
1.0
1.5
Training gain
2.0
2.5
3.0
Figure C21. Jackknife chart showing the relative importance of environmental variables as a function of the change in "gain"
(the log of the number of grid cells minus the average of the negative log probabilities of the sample locations) resulting from the
exclusion or sole inclusion of the environmental variable in the model Variables with the highest training gain resulting from
sole inclusion of those variables (dark blue bars) are the best individual variables at describing suitable habitat for the species.
Jariables with the greatest reduction in training gain resulting from their exclusion (light blue bars) contain information on the
species habitat use that is not present in other variables. The red bar indicates the maximum gain achieved with inclusion of all
variables.
Appendix C  28
Brewer's Sparrow {Spizella breweri)
TO
s
X
Figure C22. The hottocold color map indicates the suitability of each grid cell as a function of the environmental variables at that grid cell. Hotter colors indicate areas that
are predicted to have more suitable habitat for the species. Black dots are positive data used to build the model Gray dots are locations where a survey capable of detecting the
species has been performed. A shaded relief map, BLM Field Office boundaries, and county lines are included for reference.
Omission vs. Predicted Area for Brewer_s_
Sparrow
â–¡ .9
O.B
0.7
â– io.6
0.5
^0.4
D.3
n.2
â–¡ .1
â–¡ .â–¡
4
/
/
/
J
1
1
/
/
/
\
/
J
y
\
/
r
//
f
\
/
â– ^
r
\
\
/
^
^^
/
^
<i
^
A
^
.^^^â– â€¢^
â€” .
Fraction of background predicted
Omission on training sampies
Omission on test sampies
Predicted omission
10
20
30
70
80
90
100
40 50 60
Cumulative tiiiesiiold
Figure C23. An evaluation of omission error rates for training (dark blue line) and test (light blue line) data as a function of the
cumulative threshold and overall predicted area. The red line indicates the overall fraction of the map area fitting each value of
the cumulative threshold. The black line is the predicted omission rate for each cumulative threshold.
Appendix C  30
Sensitivity vs. Specificity for E
rewer
_s_Sp
narrow
i.a
D.9
o.s
I0.7
Â£=
S0.6
E
;go.4
I0.3
0.2
â–¡ .1
a.u
1
j*"^
'^
 â€”
/
/
y
/
//
/
f
/
/
/
/
/
/
/
Training data (AUC = 0.959)
Test data (AUC= 0.940]
Random Prediction (AUG = 0.5)
0.0
0.1
0.3 0.4 0.5 0.6 0.7 0.8
53ecifi:ity (1  Fractional Predicted Area)
0.9
1.0
Figure C24. Receiver Operating Characteristic (ROC) curve evaluating the overall predictive power of the model with the Area
Under the Curve (AUC). The AUC value indicates that when two random locations are chosen the model has that probability
of assigning a higher cumulative threshold value to the location with more suitable habitat. The light blue line indicates how a
neutral or random model would perform (i.e., it only has a 50% probability of assigning a higher cumulative threshold value to
a random location with more suitable habitat than a random location with less suitable habitat). The further toward the top left
of the graph the training (red) and test (blue) data lines are, the better the model is at predicting the presences contained in the
training data. Sensitivity (plotted on the yaxis) is the proportion of positive locations that were correctly classified by the model.
Sensitivity is also known as the true positive rate and can be thought of as the degree of absence of omission errors. Specificity is
the proportion of random locations chosen from the background (these pseudoabsences are used instead of true negative loca
tions) that were correctly classified by the model as negative. One minus the Specificity (plotted on the xaxis) is known as the
false positive rate and represents the commission error rate.
Appendix C  31
TO
Log response of Brewer_5_Sparrowto a5pect90m_mt83clip
f
Log response
aspec19Dm_mta3tlii
of Brewer s Sparrow to
nIcddOm mt83
1
1 1
'
1
M.
, ;
1
1
r
1
r
JU
!
03 B3 B4 35 91 93
Log response of Brewer
5 Sparrow
tc
statsgo90m
mtS3 clip
,
ll
1
1

1
III III
II'' 1
~T ! i ! 1 ! ! ! 1 ! !
: 513 557 E03 647 E91
Log response of Brewer s Sparrow to curve planSO
m mtS3
Log response of Brewer_s_Sparrowto elev90m_mt83clip
curve
Log response of Brewer s
plan9[lm_mta3
Sparrow to precip annSDm
mtS3
r
^1,0
Ins
.,7
;Â§ii5
j
1"
D.2
â–¡ ,1
â–¡ ,0
ID 30 30
Log response of Brewer s Sparrow to tmaxSOm mts:
\
,
1
1

J
J
V
â€”
Log response of Brewer_s_Sparrowto geombmg90mt83clip
5(10 ICIDCI
Log respon
se of Brew
r 5 Sparrow to 5lope90m mt83t:ljp
â€”
ll
â–¡.5
^
1
s '"'
\
s
^^â– ^
L
E
slope9[]m_m103clir)
Log response of Brewer s Sparrow to tmin90m mt83clip
'
1
. "â– '
^"^
 .
10.
s
t "l
^"..B
U.J
\,
I.U
Log response
of Brewer s Sparrow to s
oil tmpSOm
mtS3
'
â– â– 1
,
tmln90m_m1S3clii
Figure C25. Response curves for individual environmental variables showing how the logistic prediction changes as each environmental variable is varied while all other envi
ronmental variables are held constant at their average sample values. The value on the yaxis is predicted probability of suitable conditions as given by the logistic formula P(x)
= expfcl * fl(x) + c2 *f2(x) + c3 *f3(x)...) / Z. Note that if any of the environmental variables are correlated, the marginal response curves can be misleading (e.g., two highly
correlated variables with opposite response curves could effectively cancel each other out). Value definitions and/or links to metadata containing these definitions can be found in
the Descriptions of Environmental Input Layers section of the appendix above.
Jackknife of Training gain for BrewersSp arrow
tmin9Dm_mte3clip
tmax90m_mt83
statsgo90m_mt83_clip
soil_tmp90m_mt83
slope9Dm_mt83clip
piecip_ann90m_mt83
nlcd90m_mt83
geombmg90mt83clip
elev9Dm_mt83clip
c u ive_p I a n 9 m_mt8 3
aspect90m_mt83clip
Without variable
With only variable
With all variables
0.0 0.2 0.4 0.6
0.8 1.0 1.2
Training gain
1.4 1.6 1.8 2.0 2.2
Figure C26. Jackknife chart showing the relative importance of environmental variables as a function of the change in "gain"
(the log of the number of grid cells minus the average of the negative log probabilities of the sample locations) resulting from the
exclusion or sole inclusion of the environmental variable in the model Variables with the highest training gain resulting from
sole inclusion of those variables (dark blue bars) are the best individual variables at describing suitable habitat for the species.
Jariables with the greatest reduction in training gain resulting from their exclusion (light blue bars) contain information on the
species habitat use that is not present in other variables. The red bar indicates the maximum gain achieved with inclusion of all
variables.
Appendix C  33
Lark Bunting {Calamospiza melanocorys)
TO
s
X
Figure C27. The hottocold color map indicates the suitability of each grid cell as a function of the environmental variables at that grid cell. Hotter colors indicate areas that
are predicted to have more suitable habitat for the species. Black dots are positive data used to build the model Gray dots are locations where a survey capable of detecting the
species has been performed. A shaded relief map, BLM Field Office boundaries, and county lines are included for reference.
Omission vs. Predicted Area for Lark_Bunting
1.D
D.9
â–¡.e
â–¡ 7
15
0.5
(J
CIS
â–¡ .3
D.2
D.1
,
^
4
?^
A
7
^
V
4
7
1
/
/>
/
\
A
^
/
\
A
1^
/
x^
J^
^
^^
,
D.G
Fraction of background predicted
Omission on training sampies
Omission on test sampies
Predicted omission
10
?0
30
70
SO
90
100
40 50 60
Cumuiative tiiresiioid
Figure C28. An evaluation of omission error rates for training (dark blue line) and test (light blue line) data as a function of the
cumulative threshold and overall predicted area. The red line indicates the overall fraction of the map area fitting each value of
the cumulative threshold. The black line is the predicted omission rate for each cumulative threshold.
Appendix C  35
Sensitivity vs. Specificity for Lark_Bunting
i.a
D.9
o.s
q;0.7
0.6
:eo.4
0.3
0.2
0.1
0.0
1 1
â€” 
A
"^
'^^
/
y
/
/
/
/
/
/
/
[
/
/
Training data (AUC = 0.956)
Test data (AUC= 0.940]
Random Prediction (AUG = 0.5)
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8
Specificity (1  Fractionai Predicted Area)
0.9
1.0
Figure C29. Receiver Operating Characteristic (ROC) curve evaluating the overall predictive power of the model with the Area
Under the Curve (AUC). The AUC value indicates that when two random locations are chosen the model has that probability
of assigning a higher cumulative threshold value to the location with more suitable habitat. The light blue line indicates how a
neutral or random model would perform (i.e., it only has a 50% probability of assigning a higher cumulative threshold value to
a random location with more suitable habitat than a random location with less suitable habitat). The further toward the top left
of the graph the training (red) and test (blue) data lines are, the better the model is at predicting the presences contained in the
training data. Sensitivity (plotted on the yaxis) is the proportion of positive locations that were correctly classified by the model.
Sensitivity is also known as the true positive rate and can be thought of as the degree of absence of omission errors. Specificity is
the proportion of random locations chosen from the background (these pseudoabsences are used instead of true negative loca
tions) that were correctly classified by the model as negative. One minus the Specificity (plotted on the xaxis) is known as the
false positive rate and represents the commission error rate.
Appendix C  36
TO
s
Log response of Lark Bunting to aspectSOrr
mtSSclJp
II
1
1
11
1
1
r
1
Of)
por
aspects Dm,
If Lark Bu
mta
itin
Cl[)
qto
nic
d90
m mtsa
sâ€ž
S
III
^U
1
1
^
Log response of Lark_Bunting to &tatsgD9Dm_mt83_clip
I 244 289 333 377 â–
510 554 599 543 687
Log res
ponse
of Lark Bunting to curve planSOm
mtS3
1,Q
r
I"'
d4
D2
U.I
â–¡.0
Log response of Lark_Bunting to geombmg90mtS3cllp
cuive_ilan9Dn
Log response of Lark Buntin
_nitB3
to precip
ann90m mt83
D,05
HOD
[].DS
1
1
Â£ "'^
on3[i
D,2S
D,3:
01135
Â°n,4[i
â–¡,45
â–¡.SD
â–¡55
Log response of Lark_Bunting to tmaxS0m_mtS3
r
\
\
Log response of Lark Bunting
5lape90m mtSSclip
slopegDm_m183clip
Log response of Lark_Bunting to tmln90m_mt83clip
J
^
\
\
\
N
Ofl
res
por
se
>f Lark
Bu
iting to
nIc
d90
m mt83
1
1 1
1
III
II
Log response
of Lark
Sunting to soil
tmpSOm mt83
r
~
/
/
/
/
tmln90m_m183clip