tions) that were correctly classified by the model as negative. One minus the Specificity (plotted on the xaxis) is known as the
false positive rate and represents the commission error rate.
Appendix C  51
Log response of McCown_5_Long5pur to a5pect90m_mtS3clip
TO
S
n
Lofl
es
asiect9am_mtB3tlip
onseofMcCown s LongspurtonI
d90m
mtS3
.
II
1
II
.og response of McC
own
s
Long
pur
to statsgoSOm mtS3 c
ip
109 155 20D 24* 209 333 377 421
510 554 599 643 6B7
Log response of h^
cCown 5 Longspurto
curve
planSOm mtS3
Log response of McCown_5_Long5pur to elev90m_mtS3clip
Log response of McCown s
mts;
Log response of McCown_5_Long5pur to tmax90m_mtS3
\
\^
\
\
\
\
V
\
tmax90m_mt83
,
500 500
lip
Log response of McCown s Longspurto tmin90m mtS3
clip
'
Log response
3f McCown s
Longspurto


Log response ot ft
cCown s Longspurto
soil tmp90
Ti m1S3
1,1
/
1
/
09
/
/
07
06
05
Â°04.
= D3
â–¡ 2
â–¡ ,1
â–¡ ,0
/
1
/
1
1
1
1
I
Figure C45. Response curves for individual environmental variables showing how the logistic prediction changes as each environmental variable is varied while all other envi
ronmental variables are held constant at their average sample values. The value on the yaxis is predicted probability of suitable conditions as given by the logistic formula P(x)
= expfcl *fl(x) + c2 *f2(x) + c3 *f3(x)...) / Z. Note that if any of the environmental variables are correlated, the marginal response curves can be misleading (e.g., two highly
correlated variables with opposite response curves could effectively cancel each other out). Value definitions and/or links to metadata containing these definitions can be found in
the Descriptions of Environmental Input Layers section of the appendix above.
tmin9Dm_mte3clip
tmax90m_mt83
statsgo90m_mt83_clip
soil_tmp90m_mt83H
slope9Dm_mt83clip i
piecip_ann90m_mt83
nlcd90m_mt83
geombmg90mt83clip
elev9Dm_mt83clip
c u ive_p I a n 9 m_mt8 3 f
aspect90m_mt83clip
0.0
Jackknife of Training gain for IVIcCownsLongspur
Without variable
With only variable
With all variables
0.5
1.0
1.5 2.0
Training cjGin
2.5
3.0
3.5
Figure C46. Jackknife chart showing the relative importance of environmental variables as a function of the change in "gain"
(the log of the number of grid cells minus the average of the negative log probabilities of the sample locations) resulting from the
exclusion or sole inclusion of the environmental variable in the model Variables with the highest training gain resulting from
sole inclusion of those variables (dark blue bars) are the best individual variables at describing suitable habitat for the species.
Jariables with the greatest reduction in training gain resulting from their exclusion (light blue bars) contain information on the
species habitat use that is not present in other variables. The red bar indicates the maximum gain achieved with inclusion of all
variables.
Appendix C  53
Chestnutcollared Longspur {Calcarius ornatus)
TO
s
X
Figure C47. The hottocold color map indicates the suitability of each grid cell as ajiinction of the environmental variables at that grid cell. Hotter colors indicate areas that
are predicted to have more suitable habitat for the species. Black dots are positive data used to build the model. Gray dots are locations where a survey capable of detecting the
species has been performed. A shaded relief map, BLM Field Office boundaries, and county lines are included for reference.
Omission vs. Predicted Area for Chestnutcollared_Longspur
1.D
D.9
Q.B
â–¡ 7
ra
I 0.5
(J
â–¡ .3
D.2
D.1
a.a
Fraction of background predicted
Omission on training sampies
Omission on test sampies
Predicted omission
10
30 40 50 60
Cumuiative tiiresiioii
70
SO
90
100
Figure C48. An evaluation of omission error rates for training (dark blue line) and test (light blue line) data as a function of the
cumulative threshold and overall predicted area. The red line indicates the overall fraction of the map area fitting each value of
the cumulative threshold. The black line is the predicted omission rate for each cumulative threshold.
Appendix C  55
Sensitivitpr vs. 1  Specificity for Chestnutcollared_Longspur
1.D
D.9
O.S
q;0.7
0.6
:eo.4
0.3
0.2
0.1
0.0
Training data (AUC = 0.983)
Test data (AUG =0.976)
Random Prediction (AUC = 0.5)
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8
1  Specificity [Fractionai Predicted Area)
0.9
1.0
Figure C49. Receiver Operating Characteristic (ROC) curve evaluating the overall predictive power of the model with the Area
Under the Curve (AUC). The AUC value indicates that when two random locations are chosen the model has that probability
of assigning a higher cumulative threshold value to the location with more suitable habitat. The light blue line indicates how a
neutral or random model would perform (i.e., it only has a 50% probability of assigning a higher cumulative threshold value to
a random location with more suitable habitat than a random location with less suitable habitat). The further toward the top left
of the graph the training (red) and test (blue) data lines are, the better the model is at predicting the presences contained in the
training data. Sensitivity (plotted on the yaxis) is the proportion of positive locations that were correctly classified by the model.
Sensitivity is also known as the true positive rate and can be thought of as the degree of absence of omission errors. Specificity is
the proportion of random locations chosen from the background (these pseudoabsences are used instead of true negative loca
tions) that were correctly classified by the model as negative. One minus the Specificity (plotted on the xaxis) is known as the
false positive rate and represents the commission error rate.
Appendix C  56
TO
s
n
Respor
se of Chestnu
col
are
Longs
urto aspectSOm
mtSSclip
I
1
â€”
1
1
1
:
.
1
1
â€”
h
M
1
1
1
1
1
1
1
1
1
1
1
L
D.ao
Respo
Of Chestnutc
ect90m_
ollared
T1183CIIP
Lorqs
pur
to nIcdSOm mtS3
O.S!
DSQ
â–¡.75
â–¡ 7:
0.65
D55
o.sa
D46
40
0.35
1



















n



Response
of Ch
estnutCDilared
Lon
jspu
r to 5tatsgo9C
m mt83
clip
.
1
1
1
1
1
i
1
1
1
i
1
1
1
1

Response
of Chestnutcollared Longsp
rto
planSOm
mtS3
Response of Chestnutcollared_Longspur to precip_ann90m_mtS3
Response of Chestnutco
_Bnn9am_mtB3
lared Longsp
urto tmax90m mt83
90
â– sro85
d75
â„¢D70
I'D, 65
ooen
^â–¡56
â–¡ 50
45
\
\
\
\
\
\
\
\
Re
ponse
of Chestnutco
ared Longspurto elevSOm mtSSclip
â–¡.sn
= "â– '"
~
d5!
Id.so
/
/
â– a" '"
/
_,U.J.
/
â–¡ 3:
/
U.J
Â°0 7D
i'0 65
Res
ponse of Chestnutco
lared Lon^
spurtoslopeSOm mtSSclip
slo5e90m_nitB3clii
Response of Chestnutcollared Longspurto tmlnSOm mtS3clip
OS
,^06
15 0.5
04
1^3
to.
0,1
0,0
~^
N
\
\
\
\
\
\,
\
Response of Chestnut
collared
Lor
gspurto
geombm
g90mt83Glip

J
Longspurto
soil tmp9Dm
mtS3
â–
1 â–
1 I
â– I
â–
â–
â– I
â– I
1 I
1 â–
1 â–
1
il_tmpE)am_mt83
31 67 108 153 199 344 388 333 378 432 466 510 555 599 643 687
sta1sgo90m_mtB3_(lip
tm85f90m_m183
tmin90m_mt83(lip
Figure C50. Response curves for individual environmental variables showing how the logistic prediction changes as each environmental variable is varied while all other envi
ronmental variables are held constant at their average sample values. The value on the yaxis is predicted probability of suitable conditions as given by the logistic formula P(x)
= exp(cl "^ fl(x) + c2 "^ f2(x) + c3 "^ f3(x) ...) / Z. Note that if any of the environmental variables are correlated, the marginal response curves can be misleading (e.g., two highly
correlated variables with opposite response curves could effectively cancel each other out). Value definitions and/or links to metadata containing these definitions can be found in
the Descriptions of Environmental Input Layers section of the appendix above.
Jackknife of regularized training gain for CliestnutcollaredLong
n 1 1 1 1
c u ive_p I a n 9 m_mt8 3
elev90m_mt83clip
geornbmg90mt83clip
nlcd90m_mt83
piecip_ann90m_mt83
slope90m_mt83clip
soil_tmp90m_mt83
statsgo90rn_mt83_clip
tmax90m_mt83
tmin9Dm_mt83clip
spur
Without variable
With only variable
With all variables
Q.Q 0.5 1.0 1.5
regularized training gain
2.0
2.6
Figure C51. Jackknife chart showing the relative importance of environmental variables as a function of the change in "gain "
(the log of the number of grid cells minus the average of the negative log probabilities of the sample locations) resulting from the
exclusion or sole inclusion of the environmental variable in the model Variables with the highest training gain resulting from
sole inclusion of those variables (dark blue bars) are the best individual variables at describing suitable habitat for the species.
Jariables with the greatest reduction in training gain resulting from their exclusion (light blue bars) contain information on the
species habitat use that is not present in other variables. The red bar indicates the maximum gain achieved with inclusion of all
variables.
Appendix C  58
Bobolink {Dolichonyx oryzivorus)
TO
s
X
Figure C52. The hottocold color map indicates the suitability of each grid cell as a function of the environmental variables at that grid cell. Hotter colors indicate areas that
are predicted to have more suitable habitat for the species. Black dots are positive data used to build the model Gray dots are locations where a survey capable of detecting the
species has been performed. A shaded relief map, BLM Field Office boundaries, and county lines are included for reference.
Omission vs. Predicted Area for Bobolink
1.D
D.9
â–¡ .3
â–¡ 7
ra
I 0.5
(J
â–¡ .3
â–¡ .2
â–¡ .1
Q.a
^
4
A
A
J
/(â–
r
/
A
J
J
^
f
rr
/
y
y
^
r^
fl
Y
/
/
L
y
r^
_^
^
Fraction of background predicted
Omission on training sampies
Omission on test sampies
Predicted omission
10
30 40 50 60
Cumuiative tiiresiioii
70
SO
90
100
Figure C53. An evaluation of omission error rates for training (dark blue line) and test (light blue line) data as a function of the
cumulative threshold and overall predicted area. The red line indicates the overall fraction of the map area fitting each value of
the cumulative threshold. The black line is the predicted omission rate for each cumulative threshold.
Appendix C  60
Sensitivity vs. Specificity
for Bobolink
i.a
D.9
o.s
I0.7
Â£=
E
;go.4
I0.3
â–¡.2
â–¡.1
Q.D
{
/
_/
^
/
m
r
â€”/
/
i
J
/
r'
/
/
/
/
/
/
/
Training data (AUC = 0.995)
Test data (AUG =0.901)
Random Prediction (AUC = 0.5)
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8
Specificity (1  Fractionai Predicted Area)
0.9
1.0
Figure C54. Receiver Operating Characteristic (ROC) curve evaluating the overall predictive power of the model with the Area
Under the Curve (AUC). The AUC value indicates that when two random locations are chosen the model has that probability
of assigning a higher cumulative threshold value to the location with more suitable habitat The light blue line indicates how a
neutral or random model would perform (i.e., it only has a 50% probability of assigning a higher cumulative threshold value to
a random location with more suitable habitat than a random location with less suitable habitat). The further toward the top left
of the graph the training (red) and test (blue) data lines are, the better the model is at predicting the presences contained in the
training data. Sensitivity (plotted on the yaxis) is the proportion of positive locations that were correctly classified by the model.
Sensitivity is also known as the true positive rate and can be thought of as the degree of absence of omission errors. Specificity is
the proportion of random locations chosen from the background (these pseudoabsences are used instead of true negative loca
tions) that were correctly classified by the model as negative. One minus the Specificity (plotted on the xaxis) is known as the
false positive rate and represents the commission error rate.
Appendix C  61
TO
s
n
On
Log res
ponse of Bobolink to asp
sctSOm mt83cllp
[14
1
â–
1
1
II
â–
. II
1
D.1
D.2
D.3
â–¡ 4
â–
I
' 11
i
1
1
s
l
l
â–
â–
u a
B2 83 B4 B5 91
Lpg response of Bobolink to curve
pl<in90m mtS3
1
Log response of Bobolink to elev90m_mtS3clip
Log resp
sspect90m_mtB3clip
onse of Bobolink to nIcdSOm
mt83
1
II
III
II
1
II
1
T
1
Log
cuive_plan9am_mta3
response of Bobolink to precip an
n90m
mtS3
D.DB
^ [1[12
Â§â–¡â–¡2
'â–¡â–¡6
LIU.
10
Log respon
se
of Bob
â–¡ 1
nkto statsgoSOm
mtS3 clip
1
1
! 1
1 ! 1
Log response of Bobolink to trniiK9Dm
mt83
;
/"
L_^
/
/
Log response of Bobolink to
slopeSOm
mtSSclip
â–¡
DS
â– 3.5
â– 4
,
\
slope9[]m_m103clip
Log response of Bobolink to tmin90m
nrit83clip
\
Log response
of Bobo
ink to geomb
mg90mtS3clip
1
Log response of E
obolinktosoil tmp90m mtS3
544 5B7 S31 674
in90m_m1S3clii
Figure C55. Response curves for individual environmental variables showing how the logistic prediction changes as each environmental variable is varied while all other envi
ronmental variables are held constant at their average sample values. The value on the yaxis is predicted probability of suitable conditions as given by the logistic formula P(x)
= expfcl * fl(x) + c2 *f2(x) + c3 *f3(x)...) / Z. Note that if any of the environmental variables are correlated, the marginal response curves can be misleading (e.g., two highly
correlated variables with opposite response curves could effectively cancel each other out). Value definitions and/or links to metadata containing these definitions can be found in
the Descriptions of Environmental Input Layers section of the appendix above.
tmin9Dm_mte3clip
tmax90m_mt83
statsgo90m_mt83_clip
soil_tmp90m_mt83
slope9Dm_mt83clip
piecip_ann90m_mt83
nlcd90m_mt83
geombmg90mt83clip
elev9Dm_mt83clip
c u ive_p I a n 9 m_mt8 3
aspect90m_mt83clip
Jackknife of Training gain for Bobolink
^^ 1 1
^^^^^^"
^^^^^H
' n 1
^^^^
â„¢
i ^^^^^^
1 :
^^^
.
1 1 1 1 1 1 1 1
Q.O
Without variable
With only variable
With all variables
0.5
1.0
1.5 2.0
Training gain
2.5
3.0
3.5
Figure C56. Jackknife chart showing the relative importance of environmental variables as a function of the change in "gain"
(the log of the number of grid cells minus the average of the negative log probabilities of the sample locations) resulting from the
exclusion or sole inclusion of the environmental variable in the model Variables with the highest training gain resulting from
sole inclusion of those variables (dark blue bars) are the best individual variables at describing suitable habitat for the species.
Jariables with the greatest reduction in training gain resulting from their exclusion (light blue bars) contain information on the