Of the 5,930 fully scanned U.S. sites we tested, 759 (13%) have a robots.txt problem. Another 1,085 (18%) have no robots.txt file at all. No check fails for that, so they count as passing here. A robots.txt file tells search engines and AI agents which parts of a site they may crawl and where to find the sitemap. When a site has one, it has to follow the format and not block the whole site. A site fails if it fails any of these checks: Robots validand Robots allowed.
759 of 5,930
Sites that have a robots.txt problem
13%
Share of tested sites that fail these checks
5,171
Tested sites that pass these checks
6,097 of 7,046
Sites fully scanned
The 759 sites that have a robots.txt problem
Sorted by name. Each site links to its botability report and shows its botability and overall grades from the same scan.
| Site | Botability | Overall |
|---|---|---|
| alamo.edu alamo.edu |
C | B |
| alvincollege.edu Alvin Community College |
B | B |
| aspiracitycollege.edu ASPIRA City College |
B | B |
| aum.edu Auburn University at Montgomery |
B | B |
| bakersfieldcollege.edu Bakersfield College |
A | A |
| barry.edu Barry University |
B | B |
| heichaldovid.org Beis Medrash Heichal Dovid |
D | C |
| belhaven.edu Belhaven University |
A | B |
| broward.edu Broward College |
B | C |
| cameron.edu Cameron University |
A | B |
| cnm.edu Central New Mexico Community College |
C | B |
| cocc.edu Central Oregon Community College |
B | A |
| cerrocoso.edu Cerro Coso Community College |
A | B |
| chaffey.edu Chaffey College |
A | B |
| champlain.edu Champlain College |
B | A |
| chapman.edu Chapman University |
B | B |
| ccms.edu Cincinnati College of Mortuary Science |
A | B |
| crcds.edu Colgate Rochester Crozer Divinity School |
C | C |
| collegeofsanmateo.edu College of San Mateo |
B | C |
| cscc.edu Columbus State Community College |
A | A |
| cornellcollege.edu Cornell College |
A | D |
| oversight.gov Council of Inspectors General on Integrity and Efficiency |
F | C |
| cps.unh.edu cps.unh.edu |
B | B |
| cuesta.edu Cuesta College |
B | B |
| dakotacollege.edu Dakota College at Bottineau |
B | B |
| home.dartmouth.edu Dartmouth College |
B | B |
| wwtg.gov Department of Commerce |
B | B |
| stopfakes.gov Department of Commerce |
B | B |
| cuidadodesalud.gov Department of Health and Human Services |
B | A |
| worklife4you.com Department of Health and Human Services |
F | C |
| healthcare.gov Department of Health and Human Services |
B | A |
| namrs.net Department of Health and Human Services |
C | F |
| dsac.gov Department of Justice |
F | B |
| neapmap.com Department of Labor |
B | B |
| jcmarketplace.com Department of Labor |
B | B |
| lacoast.gov Department of the Interior |
D | F |
| usdebitcard.gov Department of the Treasury |
F | D |
| dvc.edu Diablo Valley College |
B | C |
| dillard.edu Dillard University |
A | B |
| edisonohio.edu Edison State Community College |
B | B |
| endicott.edu Endicott College |
B | B |
| usds.gov Executive Office of the President |
B | A |
| ftscc.org Faith Theological Seminary and Christian College |
B | B |
| fedpartnership.gov Federal Reserve Board of Governors |
C | C |
| flcc.edu Finger Lakes Community College |
A | A |
| fscj.edu Florida State College at Jacksonville |
D | C |
| fvtc.edu Fox Valley Technical College |
B | B |
| franklinpierce.edu Franklin Pierce University |
B | B |
| login.gov General Services Administration |
B | B |
| paymentaccuracy.gov General Services Administration |
D | C |
| fdlp.gov Government Publishing Office |
F | D |
| hmsom.edu Hackensack Meridian School of Medicine |
C | C |
| hpu.edu Hawaii Pacific University |
C | D |
| hillcollege.edu Hill College |
A | C |
| hiu.edu Hope International University |
B | D |
| hgtc.edu Horry-Georgetown Technical College |
B | B |
| fhlbcin.com Interagency Collaboration |
F | D |
| jscc.edu Jackson State Community College |
D | B |
| jacksonvillecollege.edu Jacksonville College-Main Campus |
B | C |
| sunyjefferson.edu Jefferson Community College |
A | B |
| jcjc.edu Jones County Junior College |
D | C |
| lecom.edu Lake Erie College of Osteopathic Medicine |
B | A |
| lsc.edu Lake Superior College |
A | C |
| lyon.edu Lyon College |
C | C |
| marist.edu Marist University |
B | B |
| mhu.edu Mars Hill University |
C | C |
| maryville.edu Maryville University of Saint Louis |
D | B |
| msudenver.edu Metropolitan State University of Denver |
A | A |
| midmich.edu Mid Michigan College |
B | B |
| macu.edu Mid-America Christian University |
B | B |
| mvsu.edu Mississippi Valley State University |
F | C |
| moreheadstate.edu Morehead State University |
B | B |
| ncc.edu Nassau Community College |
C | C |
| globe.gov National Aeronautics and Space Administration |
C | B |
| nwic.edu Northwest Indian College |
B | B |
| nrc.gov Nuclear Regulatory Commission |
B | B |
| oakleyca.gov oakleyca.gov |
B | C |
| opm.gov Office of Personnel Management |
B | A |
| occc.edu Oklahoma City Community College |
C | C |
| occ.edu Ozark Christian College |
B | A |
| piu.edu Pacific Islands University |
B | B |
| pacificoaks.edu Pacific Oaks College |
B | B |
| ptseminary.edu Pentecostal Theological Seminary |
B | D |
| plymouth.edu Plymouth State University |
D | C |
| pclob.gov Privacy and Civil Liberties Oversight Board |
C | C |
| providencecc.edu Providence Christian College |
D | D |
| rabbinicalcollegeboboveryeshiva.edu Rabbinical College Bobover Yeshiva Bnei Zion |
B | A |
| rabbinical.org Rabbinical Seminary of America |
B | B |
| rcc.mass.edu Roxbury Community College |
C | C |
| sahuaritaaz.gov sahuaritaaz.gov |
B | C |
| steu.edu Saint Elizabeth University |
B | C |
| saybrook.edu Saybrook University |
B | B |
| se-pr.edu Seminario Evangelico de Puerto Rico |
D | D |
| southeasttech.edu Southeast Technical College |
C | C |
| sco.edu Southern College of Optometry |
D | C |
| swcciowa.edu Southwestern Community College |
D | B |
| swosu.edu Southwestern Oklahoma State University |
B | C |
| sjf.edu St. John Fisher University |
A | B |
| newpaltz.edu State University of New York at New Paltz |
D | C |
| sunyopt.edu SUNY College of Optometry |
A | B |
| taylor.edu Taylor University |
B | B |
| tntech.edu Tennessee Technological University |
B | B |
| tvaoig.gov Tennessee Valley Authority |
D | D |
| twu.edu Texas Woman's University |
B | B |
| thechicagoschool.edu The Chicago School |
B | B |
| collegesoflaw.edu The Colleges of Law |
C | A |
| ustaxcourt.gov U.S. Courts |
B | B |
| nutrition.gov U.S. Department of Agriculture |
B | B |
| lcacommons.gov U.S. Department of Agriculture |
D | C |
| usmc-mccs.org U.S. Marine Corps |
F | D |
| uah.edu University of Alabama in Huntsville |
A | C |
| uaa.alaska.edu University of Alaska Anchorage |
C | D |
| manchester.unh.edu University of New Hampshire at Manchester |
C | B |
| law.unh.edu University of New Hampshire-Franklin Pierce School of Law |
C | B |
| uncw.edu University of North Carolina Wilmington |
D | B |
| sc.edu University of South Carolina-Columbia |
B | B |
| uopeople.edu University of the People |
C | C |
| usnh.edu University System of New Hampshire System Office |
B | B |
| valenciacollege.edu Valencia College |
B | B |
| vinu.edu Vincennes University |
A | B |
| ws.edu Walters State Community College |
A | B |
| wgu.edu Western Governors University |
A | A |
| mooretech.edu William R Moore College of Technology |
C | D |
| ysu.edu Youngstown State University |
D | C |
| barbourcountyal.gov Barbour County, AL |
B | C |
| birminghamal.gov City of Birmingham, AL |
C | B |
| coffeecounty.us Coffee County, AL |
B | C |
| covingtoncountyal.gov Covington County, AL |
B | C |
| dekalbcountyal.us DeKalb County, AL |
F | C |
| jacksoncountyal.gov Jackson County, AL |
B | C |
| shelbyal.com Shelby County, AL |
B | C |
| stclairco.com St. Clair County, AL |
B | C |
| fnsb.gov Fairbanks North Star Borough, AK |
B | B |
| borough.ketchikan.ak.us Ketchikan Gateway Borough, AK |
B | C |
| kodiakak.us Kodiak Island Borough, AK |
B | C |
| cochise.az.gov Cochise County, AZ |
B | C |
| lapaz.gov La Paz County, AZ |
B | C |
| maricopa.gov Maricopa County, AZ |
B | C |
| fayetteville-ar.gov City of Fayetteville, AR |
B | C |
| crosscountyar.org Cross County, AR |
C | C |
| garlandcounty.org Garland County, AR |
B | C |
| perrycoarkansas.org Perry County, AR |
D | F |
| pulaskicounty.net Pulaski County, AR |
C | C |
| alpinecountyca.gov Alpine County, CA |
B | C |
| elmonteca.gov City of El Monte, CA |
B | C |
| escondido.gov City of Escondido, CA |
B | C |
| fontanaca.gov City of Fontana, CA |
B | C |
| modestogov.com City of Modesto, CA |
B | C |
| murrietaca.gov City of Murrieta, CA |
B | C |
| cityofpalmdaleca.gov City of Palmdale, CA |
B | C |
| rialtoca.gov City of Rialto, CA |
B | C |
| richmondca.gov City of Richmond, CA |
B | C |
| sanbernardino.gov City of San Bernardino, CA |
C | C |
| cityofsanmateo.org City of San Mateo, CA |
B | C |
| temeculaca.gov City of Temecula, CA |
B | C |
| countyofcolusaca.gov Colusa County, CA |
B | C |
| solanocounty.gov County of Solano, CA |
B | B |
| humboldtgov.org Humboldt County, CA |
C | C |
| lakecountyca.gov Lake County, CA |
B | C |
| lassencounty.gov Lassen County, CA |
B | C |
| mariposacounty.gov Mariposa County, CA |
B | C |
| countyofmodoc.gov Modoc County, CA |
B | C |
| monocounty.ca.gov Mono County, CA |
B | C |
| nevadacountyca.gov Nevada County, CA |
C | C |
| plumascounty.us Plumas County, CA |
B | C |
| sierracounty.ca.gov Sierra County, CA |
B | C |
| trinitycounty.org Trinity County, CA |
B | C |
| alamosacounty.org Alamosa County, CO |
B | C |
| broomfield.org Broomfield, CO |
C | C |
| westminsterco.gov City of Westminster, CO |
B | C |
| clearcreekcounty.us Clear Creek County, CO |
B | C |
| deltacountyco.gov Delta County, CO |
B | C |
| elbertcounty-co.gov Elbert County, CO |
B | C |
| lakecountyco.gov Lake County, CO |
B | C |
| ouraycountyco.gov Ouray County, CO |
B | C |
| parkcountyco.gov Park County, CO |
B | C |
| pitkincounty.com Pitkin County, CO |
B | C |
| co.routt.co.us Routt County, CO |
B | C |
| sanmiguelcountyco.gov San Miguel County, CO |
B | C |
| tellercounty.gov Teller County, CO |
B | C |
| baycountyfl.gov Bay County, FL |
B | C |
| hialeahfl.gov City of Hialeah, FL |
B | C |
| miamigardens-fl.gov City of Miami Gardens, FL |
B | C |
| desotobocc.com DeSoto County, FL |
B | C |
| jeffersoncountyfl.gov Jefferson County, FL |
B | B |
| levycounty.org Levy County, FL |
B | C |
| monroecounty-fl.gov Monroe County, FL |
B | D |
| nassaucountyfl.com Nassau County, FL |
B | C |
| orangecountyfl.net Orange County, FL |
F | D |
| pinellas.gov Pinellas County, FL |
B | B |
| polkfl.gov Polk County, FL |
C | B |
| santarosa.fl.gov Santa Rosa County, FL |
B | C |
| sumtercountyfl.gov Sumter County, FL |
B | C |
| mywaltonfl.gov Walton County, FL |
B | C |
| baldwincountyga.com Baldwin County, GA |
B | C |
| bankscountyga.org Banks County, GA |
B | C |
| barrowga.org Barrow County, GA |
B | C |
| camdencountyga.gov Camden County, GA |
B | C |
| carrollcountyga.gov Carroll County, GA |
B | C |
| charltoncountyga.us Charlton County, GA |
B | C |
| chattoogacounty.gov Chattooga County, GA |
B | C |
| augustaga.gov City of Augusta, GA |
B | C |
| savannahga.gov City of Savannah, GA |
B | C |
| cityofsouthfultonga.gov City of South Fulton, GA |
B | C |
| accgov.com Clarke County, GA |
B | C |
| columbiacountyga.gov Columbia County, GA |
B | B |
| coweta.ga.us Coweta County, GA |
B | C |
| dadecounty-ga.gov Dade County, GA |
B | C |
| dawsoncountyga.gov Dawson County, GA |
B | C |
| douglascountyga.gov Douglas County, GA |
B | C |
| effinghamcounty.org Effingham County, GA |
B | C |
| emanuelco-ga.gov Emanuel County, GA |
B | C |
| franklincountyga.gov Franklin County, GA |
B | B |
| fultoncountyga.gov Fulton County, GA |
C | B |
| gordoncountyga.gov Gordon County, GA |
B | C |
| greenecountyga.gov Greene County, GA |
C | C |
| hallcounty.org Hall County, GA |
B | C |
| jacksoncountygov.com Jackson County, GA |
B | C |
| jeffersoncountyga.gov Jefferson County, GA |
B | D |
| laurenscoga.org Laurens County, GA |
B | C |
| libertycountyga.gov Liberty County, GA |
C | C |
| lowndescounty.com Lowndes County, GA |
B | C |
| lumpkincounty.gov Lumpkin County, GA |
C | C |
| thomson-mcduffie.gov McDuffie County, GA |
B | C |
| mcintoshcountyga.com McIntosh County, GA |
B | C |
| meriwethercountyga.gov Meriwether County, GA |
B | C |
| morgancountyga.gov Morgan County, GA |
B | C |
| murraycountyga.org Murray County, GA |
B | C |
| oconeecountyga.gov Oconee County, GA |
B | C |
| paulding.gov Paulding County, GA |
B | C |
| pickenscountyga.gov Pickens County, GA |
B | C |
| putnamcountyga.us Putnam County, GA |
B | C |
| seminolecountyga.com Seminole County, GA |
B | C |
| sumtercountyga.us Sumter County, GA |
B | C |
| unioncountyga.gov Union County, GA |
B | C |
| upsoncountyga.org Upson County, GA |
B | C |
| waltoncountyga.gov Walton County, GA |
B | C |
| washingtoncountyga.gov Washington County, GA |
B | C |
| whitecountyga.gov White County, GA |
B | C |
| wilcoxcounty.live-website.com Wilcox County, GA |
F | C |
| blainecountyid.gov Blaine County, ID |
B | C |
| clark-co.id.gov Clark County, ID |
B | C |
| goodingcounty.org Gooding County, ID |
B | C |
| jcgov.us Jefferson County, ID |
B | C |
| jeromecountyid.us Jerome County, ID |
B | C |
| kcgov.us Kootenai County, ID |
C | C |
| lemhicountyidaho.org Lemhi County, ID |
B | C |
| minidoka.id.us Minidoka County, ID |
B | C |
| powercounty.gov Power County, ID |
B | C |
| co.washington.id.us Washington County, ID |
B | C |
| elginil.gov City of Elgin, IL |
B | C |
| peoriagov.org City of Peoria, IL |
B | C |
| rockfordil.gov City of Rockford, IL |
C | C |
| henrycty.com Henry County, IL |
B | C |
| jacksoncounty-il.gov Jackson County, IL |
B | C |
| jodaviesscountyil.gov Jo Daviess County, IL |
B | C |
| k3county.net Kankakee County, IL |
C | C |
| lakecountyil.gov Lake County, IL |
B | C |
| lasallecountyil.gov LaSalle County, IL |
B | C |
| leecountyil.com Lee County, IL |
C | C |
| mcleancountyil.gov McLean County, IL |
B | C |
| menardcountyil.gov Menard County, IL |
D | D |
| peoriacounty.gov Peoria County, IL |
B | C |
| rockislandcountyil.gov Rock Island County, IL |
B | C |
| whitesidecountyil.gov Whiteside County, IL |
B | C |
| woodford-county.org Woodford County, IL |
B | C |
| co.adams.in.us Adams County, IN |
B | C |
| blackfordcounty.com Blackford County, IN |
B | D |
| browncounty-in.gov Brown County, IN |
B | C |
| co.clark.in.us Clark County, IN |
F | C |
| daviess.org Daviess County, IN |
B | C |
| co.fulton.in.us Fulton County, IN |
B | C |
| hamiltoncounty.in.gov Hamilton County, IN |
B | C |
| hancockin.gov Hancock County, IN |
B | C |
| henrycounty.in.gov Henry County, IN |
B | C |
| jeffersoncounty.in.gov Jefferson County, IN |
B | C |
| lawrencecounty.in.gov Lawrence County, IN |
B | C |
| miamicountyin.gov Miami County, IN |
B | C |
| montgomerycounty.in.gov Montgomery County, IN |
C | C |
| portercountyin.gov Porter County, IN |
C | C |
| sjcindiana.gov Saint Joseph County, IN |
B | C |
| scottcounty.in.gov Scott County, IN |
B | C |
| starke.in.gov Starke County, IN |
B | B |
| tippecanoe.in.gov Tippecanoe County, IN |
B | C |
| blackhawkcounty.iowa.gov Black Hawk County, IA |
B | C |
| dallascountyiowa.gov Dallas County, IA |
B | C |
| dubuquecountyiowa.gov Dubuque County, IA |
C | C |
| floydco.iowa.gov Floyd County, IA |
B | C |
| millscountyiowa.gov Mills County, IA |
B | C |
| storycountyiowa.gov Story County, IA |
B | C |
| washingtoncounty.iowa.gov Washington County, IA |
B | C |
| atchisoncountyks.org Atchison County, KS |
B | C |
| brcoks.org Brown County, KS |
B | C |
| claycountykansas.org Clay County, KS |
B | C |
| coffeycountyks.org Coffey County, KS |
B | C |
| cowleycountyks.gov Cowley County, KS |
B | C |
| ellisco.net Ellis County, KS |
B | C |
| fordcounty.net Ford County, KS |
B | C |
| gearycounty.gov Geary County, KS |
B | C |
| grantcoks.org Grant County, KS |
C | C |
| hamiltoncountyks.org Hamilton County, KS |
B | C |
| jacksoncountyks.com Jackson County, KS |
B | C |
| jfcountyks.com Jefferson County, KS |
B | C |
| kingmancoks.org Kingman County, KS |
B | C |
| kiowacountyks.gov Kiowa County, KS |
B | C |
| linncountyks.gov Linn County, KS |
B | C |
| lyoncountyks.gov Lyon County, KS |
B | C |
| marioncoks.net Marion County, KS |
C | C |
| mcphersoncountyks.us McPherson County, KS |
C | C |
| meadeco.org Meade County, KS |
C | C |
| miamicountyks.gov Miami County, KS |
B | C |
| mgcountyks.org Montgomery County, KS |
B | C |
| morriscountyks.org Morris County, KS |
B | B |
| mtcoks.com Morton County, KS |
B | C |
| neoshocountyks.org Neosho County, KS |
B | C |
| osageco.org Osage County, KS |
B | C |
| pottcounty.org Pottawatomie County, KS |
B | C |
| prattcounty.org Pratt County, KS |
B | C |
| rileycountyks.gov Riley County, KS |
B | C |
| russellcountykansas.com Russell County, KS |
B | C |
| sewardcountyks.org Seward County, KS |
C | C |
| thomascountyks.gov Thomas County, KS |
B | C |
| wbcounty.org Wabaunsee County, KS |
B | C |
| wallacecountyks.gov Wallace County, KS |
B | C |
| wichitacounty.org Wichita County, KS |
B | C |
| boydcountyky.gov Boyd County, KY |
B | C |
| boylecountyky.gov Boyle County, KY |
B | C |
| lexingtonky.gov City of Lexington, KY |
B | B |
| clarkcoky.com Clark County, KY |
B | C |
| hardincountyky.gov Hardin County, KY |
B | B |
| hendersoncountyky.gov Henderson County, KY |
B | C |
| kentoncountyky.gov Kenton County, KY |
B | C |
| logancountyky.gov Logan County, KY |
B | C |
| marshallcountyky.gov Marshall County, KY |
B | C |
| meadeky.gov Meade County, KY |
B | C |
| woodfordcountyky.gov Woodford County, KY |
B | C |
| assumptionla.com Assumption Parish, LA |
B | C |
| brla.gov City of Baton Rouge, LA |
C | D |
| shreveportla.gov City of Shreveport, LA |
B | C |
| plaqueminesparish.gov Plaquemines Parish, LA |
B | C |
| sbpg.net Saint Bernard Parish, LA |
B | C |
| stjamesla.com St. James Parish, LA |
B | C |
| wpgov.org Washington Parish, LA |
B | C |
| wbrparish.org West Baton Rouge Parish, LA |
B | C |
| androscoggincountymaine.gov Androscoggin County, ME |
B | C |
| augustamaine.gov City of Augusta, ME |
B | C |
| waldocountyme.gov Waldo County, ME |
B | C |
| gov.allconet.org Allegany County, MD |
B | C |
| calvertcountymd.gov Calvert County, MD |
B | C |
| carolinemd.org Caroline County, MD |
B | C |
| cecilcountymd.gov Cecil County, MD |
B | C |
| annapolis.gov City of Annapolis, MD |
C | C |
| frederickcountymd.gov Frederick County, MD |
B | C |
| harfordcountymd.gov Harford County, MD |
B | C |
| qac.org Queen Anne's County, MD |
B | C |
| wicomicocounty.org Wicomico County, MD |
B | C |
| lowellma.gov City of Lowell, MA |
B | C |
| essexma.org Essex County, MA |
B | C |
| nantucket-ma.gov Nantucket, MA |
C | C |
| plymouthcountyma.gov Plymouth County, MA |
B | C |
| alpenacounty.org Alpena County, MI |
B | C |
| casscountymi.org Cass County, MI |
B | C |
| eastgrmi.gov City of Grand Rapids, MI |
B | C |
| walkermi.gov City of Grand Rapids, MI |
B | C |
| sterlingheights.gov City of Sterling Heights, MI |
C | C |
| clinton-county.org Clinton County, MI |
B | C |
| eatoncounty.org Eaton County, MI |
B | C |
| gtcountymi.gov Grand Traverse County, MI |
C | C |
| gratiotmi.com Gratiot County, MI |
B | C |
| mijackson.org Jackson County, MI |
B | C |
| lenawee.mi.us Lenawee County, MI |
C | C |
| manisteecountymi.gov Manistee County, MI |
B | C |
| co.monroe.mi.us Monroe County, MI |
B | C |
| montcalm.us Montcalm County, MI |
B | C |
| co.muskegon.mi.us Muskegon County, MI |
B | C |
| otsegocountymi.gov Otsego County, MI |
B | C |
| roscommoncounty.net Roscommon County, MI |
C | C |
| anokacountymn.gov Anoka County, MN |
B | C |
| bentoncountymn.gov Benton County, MN |
B | C |
| browncountymn.gov Brown County, MN |
B | C |
| carltoncountymn.gov Carlton County, MN |
B | C |
| chippewacountymn.gov Chippewa County, MN |
B | C |
| claycountymn.gov Clay County, MN |
B | C |
| crowwing.gov Crow Wing County, MN |
B | C |
| faribaultcountymn.gov Faribault County, MN |
B | C |
| freeborncountymn.gov Freeborn County, MN |
B | C |
| itascacountymn.gov Itasca County, MN |
B | C |
| koochiching.gov Koochiching County, MN |
B | C |
| lqpcomn.gov Lac qui Parle County, MN |
B | C |
| lakecountymn.gov Lake County, MN |
B | C |
| lesueurcounty.gov Le Sueur County, MN |
B | C |
| meekercountymn.gov Meeker County, MN |
B | C |
| millelacs.mn.gov Mille Lacs County, MN |
B | C |
| morrisoncountymn.gov Morrison County, MN |
B | C |
| co.mower.mn.us Mower County, MN |
C | C |
| nicolletcountymn.gov Nicollet County, MN |
B | C |
| polkcountymn.gov Polk County, MN |
C | C |
| co.sherburne.mn.us Sherburne County, MN |
B | C |
| sibleycounty.gov Sibley County, MN |
B | C |
| stevenscountymn.gov Stevens County, MN |
B | C |
| wadenacounty.gov Wadena County, MN |
B | C |
| wasecacounty.gov Waseca County, MN |
C | C |
| washingtoncountymn.gov Washington County, MN |
B | C |
| watonwancountymn.gov Watonwan County, MN |
B | C |
| winonacounty.gov Winona County, MN |
B | C |
| wrightcountymn.gov Wright County, MN |
B | C |
| desotocountyms.gov DeSoto County, MS |
B | C |
| co.jackson.ms.us Jackson County, MS |
B | C |
| lowndescountyms.com Lowndes County, MS |
B | C |
| oktibbeha.ms.gov Oktibbeha County, MS |
B | C |
| pearlrivercounty.net Pearl River County, MS |
B | C |
| casscounty.com Cass County, MO |
B | C |
| stlouis-mo.gov City of St. Louis, MO |
D | A |
| claycountymo.gov Clay County, MO |
B | C |
| colecounty.org Cole County, MO |
B | C |
| jeffcomo.gov Jefferson County, MO |
B | C |
| jocomo.gov Johnson County, MO |
B | C |
| mariescountymo.gov Maries County, MO |
B | D |
| new-madrid.mo.us New Madrid County, MO |
B | C |
| perrycountymo.us Perry County, MO |
B | C |
| sccmo.org Saint Charles County, MO |
B | C |
| bighorncountymt.gov Big Horn County, MT |
B | C |
| cascadecountymt.gov Cascade County, MT |
B | C |
| billingsmtpublicworks.gov City of Billings, MT |
B | C |
| billingsmt.gov City of Billings, MT |
B | C |
| adlc.us Deer Lodge County, MT |
B | C |
| gallatinmt.gov Gallatin County, MT |
B | C |
| lakemt.gov Lake County, MT |
B | C |
| madisoncountymt.gov Madison County, MT |
B | C |
| missoulacounty.gov Missoula County, MT |
C | B |
| ravallicounty.gov Ravalli County, MT |
B | C |
| co.sanders.mt.us Sanders County, MT |
B | C |
| stillwatercountymt.gov Stillwater County, MT |
B | C |
| valleycountymt.gov Valley County, MT |
B | C |
| greeleycounty.ne.gov Greeley County, NE |
C | B |
| sarpy.gov Sarpy County, NE |
B | C |
| waynecountyne.gov Wayne County, NE |
B | C |
| churchillcountynv.gov Churchill County, NV |
B | C |
| humboldtcountynv.gov Humboldt County, NV |
B | C |
| lyon-county.org Lyon County, NV |
B | C |
| nyecountynv.gov Nye County, NV |
B | C |
| storeycounty.org Storey County, NV |
B | C |
| whitepinecounty.net White Pine County, NV |
B | C |
| belknapcounty.gov Belknap County, NH |
B | C |
| carrollcountynh.gov Carroll County, NH |
B | C |
| concordnh.gov City of Concord, NH |
C | C |
| sullivancountynh.gov Sullivan County, NH |
B | C |
| burlingtoncountynj.gov Burlington County, NJ |
B | C |
| capemaycountynj.gov Cape May County, NJ |
B | C |
| essexcountynj.org Essex County, NJ |
F | D |
| gloucestercountynj.gov Gloucester County, NJ |
B | C |
| co.hunterdon.nj.us Hunterdon County, NJ |
B | C |
| sussex.nj.us Sussex County, NJ |
B | C |
| rrnm.gov City of Rio Rancho, NM |
B | C |
| eddycountynm.gov Eddy County, NM |
C | C |
| grantcountynm.gov Grant County, NM |
B | C |
| leacounty.gov Lea County, NM |
B | C |
| co.mckinley.nm.us McKinley County, NM |
B | C |
| co.otero.nm.us Otero County, NM |
B | C |
| rooseveltcounty.com Roosevelt County, NM |
B | C |
| taoscounty.org Taos County, NM |
B | C |
| unionnm.us Union County, NM |
B | C |
| co.valencia.nm.us Valencia County, NM |
B | C |
| broomecountyny.gov Broome County, NY |
B | B |
| chemungcountyny.gov Chemung County, NY |
B | C |
| albanyny.gov City of Albany, NY |
B | B |
| columbiacountyny.gov Columbia County, NY |
B | C |
| cortlandcountyny.gov Cortland County, NY |
C | C |
| livingstoncountyny.gov Livingston County, NY |
B | C |
| madisoncounty.ny.gov Madison County, NY |
B | C |
| rensco.com Rensselaer County, NY |
B | C |
| schuylercountyny.gov Schuyler County, NY |
B | C |
| senecacountyny.gov Seneca County, NY |
B | C |
| steubencountyny.gov Steuben County, NY |
B | C |
| waynecountyny.gov Wayne County, NY |
B | C |
| wyomingcountyny.gov Wyoming County, NY |
C | C |
| co.anson.nc.us Anson County, NC |
B | C |
| co.beaufort.nc.us Beaufort County, NC |
B | C |
| brunswickcountync.gov Brunswick County, NC |
B | C |
| burkenc.org Burke County, NC |
B | C |
| caldwellcountync.org Caldwell County, NC |
B | C |
| camdencountync.gov Camden County, NC |
B | C |
| carteretcountync.gov Carteret County, NC |
B | C |
| cherokeecounty-nc.gov Cherokee County, NC |
C | C |
| highpointnc.gov City of High Point, NC |
C | C |
| cravencountync.gov Craven County, NC |
B | C |
| co.davidson.nc.us Davidson County, NC |
B | C |
| daviecountync.gov Davie County, NC |
B | C |
| duplinnc.gov Duplin County, NC |
B | C |
| gastongov.com Gaston County, NC |
B | C |
| halifaxnc.com Halifax County, NC |
B | C |
| haywoodcountync.gov Haywood County, NC |
B | C |
| hokecounty.net Hoke County, NC |
B | C |
| iredellcountync.gov Iredell County, NC |
B | C |
| jacksonnc.org Jackson County, NC |
B | C |
| lincolncountync.gov Lincoln County, NC |
B | B |
| moorecountync.gov Moore County, NC |
B | C |
| nashcountync.gov Nash County, NC |
B | C |
| nhcgov.com New Hanover County, NC |
B | B |
| northamptonnc.com Northampton County, NC |
B | C |
| orangecountync.gov Orange County, NC |
B | C |
| pendercountync.gov Pender County, NC |
B | C |
| pittcountync.gov Pitt County, NC |
B | C |
| randolphcountync.gov Randolph County, NC |
B | C |
| richmondnc.com Richmond County, NC |
B | C |
| rowancountync.gov Rowan County, NC |
B | C |
| scotlandcounty.org Scotland County, NC |
B | C |
| stanlycountync.gov Stanly County, NC |
B | C |
| warrencountync.com Warren County, NC |
B | C |
| waynegov.com Wayne County, NC |
B | C |
| wilkescounty.net Wilkes County, NC |
B | C |
| yadkincountync.gov Yadkin County, NC |
B | C |
| yanceycountync.gov Yancey County, NC |
B | C |
| billingscountynd.gov Billings County, ND |
B | C |
| bismarcknd.gov City of Bismarck, ND |
B | C |
| ramseycountynd.gov Ramsey County, ND |
B | C |
| co.traill.nd.us Traill County, ND |
B | C |
| co.ward.nd.us Ward County, ND |
B | C |
| ashtabulacounty.gov Ashtabula County, OH |
B | C |
| daytonohio.gov City of Dayton, OH |
C | C |
| clarkcountyohio.gov Clark County, OH |
B | C |
| fayette-co-oh.com Fayette County, OH |
B | C |
| fultoncountyoh.com Fulton County, OH |
B | C |
| co.hancock.oh.us Hancock County, OH |
B | C |
| henrycountyohio.gov Henry County, OH |
B | C |
| lickingcounty.gov Licking County, OH |
C | C |
| loraincountyohio.gov Lorain County, OH |
B | C |
| co.lucas.oh.us Lucas County, OH |
B | C |
| mahoningcountyoh.gov Mahoning County, OH |
B | C |
| co.ottawa.oh.us Ottawa County, OH |
B | C |
| prebco.org Preble County, OH |
B | C |
| co.shelby.oh.us Shelby County, OH |
B | C |
| washingtongov.org Washington County, OH |
C | C |
| williamscountyoh.gov Williams County, OH |
B | C |
| co.wyandot.oh.us Wyandot County, OH |
B | C |
| canadiancounty.org Canadian County, OK |
B | C |
| clevelandcountyok.com Cleveland County, OK |
B | C |
| comanchecountyok.gov Comanche County, OK |
B | C |
| gradycountyok.com Grady County, OK |
B | C |
| courthouse.kay.ok.us Kay County, OK |
B | C |
| logancountyok.com Logan County, OK |
B | C |
| pottawatomiecountyok.gov Pottawatomie County, OK |
B | C |
| rogerscounty.org Rogers County, OK |
B | C |
| wagonercounty.ok.gov Wagoner County, OK |
B | C |
| clatsopcounty.gov Clatsop County, OR |
B | C |
| crookcountyor.gov Crook County, OR |
B | C |
| deschutescounty.gov Deschutes County, OR |
B | C |
| douglascountyor.gov Douglas County, OR |
B | C |
| grantcountyoregon.net Grant County, OR |
B | C |
| klamathcounty.org Klamath County, OR |
B | C |
| co.lincoln.or.us Lincoln County, OR |
B | C |
| polkcountyor.gov Polk County, OR |
B | C |
| shermancountyor.gov Sherman County, OR |
B | C |
| tillamookcounty.gov Tillamook County, OR |
B | C |
| yamhillcounty.gov Yamhill County, OR |
B | C |
| buckscounty.gov Bucks County, PA |
B | C |
| butlercountypa.gov Butler County, PA |
B | B |
| centrecountypa.gov Centre County, PA |
B | C |
| chesco.org Chester County, PA |
B | C |
| clearfieldcountypa.gov Clearfield County, PA |
B | C |
| fayettecountypa.org Fayette County, PA |
B | C |
| juniatacountypa.gov Juniata County, PA |
B | C |
| co.lancaster.pa.us Lancaster County, PA |
B | C |
| waynecountypa.gov Wayne County, PA |
B | C |
| yorkcountypa.gov York County, PA |
B | C |
| barnwellcountysc.us Barnwell County, SC |
B | C |
| cherokeecountysc.gov Cherokee County, SC |
B | B |
| charleston-sc.gov City of Charleston, SC |
B | B |
| gtcountysc.gov Georgetown County, SC |
B | C |
| hamptoncountysc.org Hampton County, SC |
B | C |
| lancastercountysc.gov Lancaster County, SC |
B | C |
| williamsburgcounty.sc.gov Williamsburg County, SC |
B | C |
| beadlesd.org Beadle County, SD |
B | C |
| brookingscountysd.gov Brookings County, SD |
B | C |
| brulecounty.gov Brule County, SD |
C | B |
| lawrence.sd.us Lawrence County, SD |
C | C |
| lincolncountysd.gov Lincoln County, SD |
B | C |
| blounttn.gov Blount County, TN |
C | C |
| clarksvilletn.gov City of Clarksville, TN |
B | C |
| murfreesborotn.gov City of Murfreesboro, TN |
B | C |
| coffeecountytn.gov Coffee County, TN |
B | C |
| madisoncountytn.gov Madison County, TN |
B | C |
| maurycounty-tn.gov Maury County, TN |
B | C |
| washingtoncountytn.org Washington County, TN |
B | C |
| williamsoncounty-tn.gov Williamson County, TN |
B | C |
| wilsoncountytn.gov Wilson County, TN |
B | C |
| co.andrews.tx.us Andrews County, TX |
B | C |
| beecounty.gov Bee County, TX |
B | C |
| bexar.org Bexar County, TX |
B | C |
| bosquecounty.gov Bosque County, TX |
C | C |
| bowiecounty.org Bowie County, TX |
B | C |
| burnetcountytexas.org Burnet County, TX |
B | C |
| chamberscountytx.gov Chambers County, TX |
B | B |
| abilenetx.gov City of Abilene, TX |
B | C |
| baytown.org City of Baytown, TX |
B | C |
| bellairetx.gov City of Bellaire, TX |
C | C |
| deerparktx.gov City of Deer Park, TX |
B | C |
| ci.friendswood.tx.us City of Friendswood, TX |
B | C |
| friscotexas.gov City of Frisco, TX |
B | C |
| cityofgalenapark-tx.gov City of Galena Park, TX |
B | C |
| garlandtx.gov City of Garland, TX |
B | C |
| southsideplacetx.gov City of Houston, TX |
B | C |
| killeentexas.gov City of Killeen, TX |
B | C |
| laportetx.gov City of LaPorte, TX |
B | C |
| leaguecitytx.gov City of League City, TX |
B | C |
| cityofmesquite.com City of Mesquite, TX |
B | C |
| midlandtexas.gov City of Midland, TX |
C | C |
| missouricitytx.gov City of Missouri, TX |
B | C |
| nassaubay.com City of Nassau Bay, TX |
B | C |
| newbraunfels.gov City of New Braunfels, TX |
B | C |
| odessa-tx.gov City of Odessa, TX |
C | C |
| pasadenatx.gov City of Pasadena, TX |
B | C |
| seabrooktx.gov City of Seabrook, TX |
B | C |
| southhoustontx.gov City of South Houston, TX |
B | C |
| springvalleytx.com City of Spring Valley Village, TX |
B | C |
| staffordtx.gov City of Stafford, TX |
B | C |
| sugarlandtx.gov City of Sugar Land, TX |
B | C |
| wallertexas.gov City of Waller, TX |
B | C |
| westutx.gov City of West University Place, TX |
B | C |
| comalcounty.gov Comal County, TX |
B | C |
| comanchecountytx.gov Comanche County, TX |
C | C |
| dentoncounty.gov Denton County, TX |
B | C |
| eastlandcounty.gov Eastland County, TX |
B | C |
| ectorcountytx.gov Ector County, TX |
B | C |
| epcounty.com El Paso County, TX |
B | C |
| elliscountytx.gov Ellis County, TX |
B | C |
| gillespiecounty.gov Gillespie County, TX |
B | C |
| henderson-county.com Henderson County, TX |
B | C |
| hidalgocounty.us Hidalgo County, TX |
B | C |
| co.howard.tx.us Howard County, TX |
B | C |
| co.irion.tx.us Irion County, TX |
B | C |
| kaufmancounty.net Kaufman County, TX |
B | C |
| kendallcountytx.gov Kendall County, TX |
B | C |
| co.martin.tx.us Martin County, TX |
B | C |
| co.midland.tx.us Midland County, TX |
C | C |
| nacogdochesco.gov Nacogdoches County, TX |
B | B |
| co.panola.tx.us Panola County, TX |
B | C |
| parkercountytx.gov Parker County, TX |
B | C |
| polktx.gov Polk County, TX |
B | C |
| randallcounty.gov Randall County, TX |
B | C |
| rockwallcountytexas.com Rockwall County, TX |
B | C |
| smith-county.com Smith County, TX |
B | C |
| somervell.co Somervell County, TX |
B | C |
| taylorcounty.texas.gov Taylor County, TX |
B | C |
| valverdecounty.texas.gov Val Verde County, TX |
B | C |
| webbcountytx.gov Webb County, TX |
B | C |
| co.winkler.tx.us Winkler County, TX |
B | C |
| co.wise.tx.us Wise County, TX |
B | C |
| beaver.utah.gov Beaver County, UT |
B | C |
| boxeldercountyut.gov Box Elder County, UT |
B | C |
| daggettcounty.gov Daggett County, UT |
B | C |
| grandcountyutah.net Grand County, UT |
B | C |
| kane.utah.gov Kane County, UT |
B | B |
| sanjuancountyut.gov San Juan County, UT |
B | C |
| sanpetecountyutah.gov Sanpete County, UT |
B | C |
| summitcountyutah.gov Summit County, UT |
B | B |
| uintah.gov Uintah County, UT |
B | C |
| wasatchcounty.gov Wasatch County, UT |
B | C |
| accomack.gov Accomack County, VA |
B | C |
| appomattoxcountyva.gov Appomattox County, VA |
B | C |
| botetourtva.gov Botetourt County, VA |
B | C |
| co.campbell.va.us Campbell County, VA |
B | C |
| co.caroline.va.us Caroline County, VA |
B | C |
| charlescityva.us Charles City County, VA |
B | C |
| chesterfield.gov Chesterfield County, VA |
C | C |
| cityofchesapeake.net City of Chesapeake, VA |
B | C |
| nnva.gov City of Newport News, VA |
B | C |
| norfolk.gov City of Norfolk, VA |
C | C |
| craigcountyva.gov Craig County, VA |
B | C |
| cumberlandcounty.virginia.gov Cumberland County, VA |
B | C |
| dickensonva.org Dickenson County, VA |
B | C |
| dinwiddieva.us Dinwiddie County, VA |
C | C |
| essexva.gov Essex County, VA |
B | C |
| franklincountyva.gov Franklin County, VA |
B | C |
| goochlandva.us Goochland County, VA |
B | C |
| graysoncountyva.gov Grayson County, VA |
B | C |
| greenecountyva.gov Greene County, VA |
B | C |
| halifaxcountyva.gov Halifax County, VA |
B | C |
| henrycountyva.gov Henry County, VA |
B | C |
| kinggeorgecountyva.gov King George County, VA |
B | C |
| kwc.gov King William County, VA |
B | C |
| loudoun.gov Loudoun County, VA |
B | C |
| louisacounty.gov Louisa County, VA |
B | C |
| madisonco.virginia.gov Madison County, VA |
B | C |
| mathewscountyva.gov Mathews County, VA |
C | C |
| mecklenburgva.com Mecklenburg County, VA |
C | C |
| co.middlesex.va.us Middlesex County, VA |
B | C |
| newkent-va.us New Kent County, VA |
B | C |
| co.northumberland.va.us Northumberland County, VA |
B | C |
| nortonva.gov Norton, VA |
B | C |
| orangecountyva.gov Orange County, VA |
B | C |
| pagecounty.virginia.gov Page County, VA |
B | C |
| co.patrick.va.us Patrick County, VA |
B | C |
| pittsylvaniacountyva.gov Pittsylvania County, VA |
B | C |
| powhatanva.gov Powhatan County, VA |
B | C |
| roanokecountyva.gov Roanoke County, VA |
B | C |
| co.rockbridge.va.us Rockbridge County, VA |
C | C |
| rockinghamcountyva.gov Rockingham County, VA |
B | C |
| russellcountyva.us Russell County, VA |
B | C |
| scottcountyva.gov Scott County, VA |
B | C |
| shenandoahcountyva.gov Shenandoah County, VA |
B | C |
| spotsylvania.va.us Spotsylvania County, VA |
B | C |
| surrycountyva.gov Surry County, VA |
B | C |
| warrencountyva.gov Warren County, VA |
B | C |
| westmoreland-county.org Westmoreland County, VA |
B | C |
| wisecounty.org Wise County, VA |
B | C |
| wytheco.org Wythe County, VA |
B | C |
| yorkcounty.gov York County, VA |
B | C |
| co.adams.wa.gov Adams County, WA |
D | C |
| asotincountywa.gov Asotin County, WA |
B | C |
| co.chelan.wa.us Chelan County, WA |
F | D |
| spokanevalleywa.gov City of Spokane Valley, WA |
C | C |
| clallamcountywa.gov Clallam County, WA |
B | C |
| columbiaco.com Columbia County, WA |
B | C |
| co.cowlitz.wa.us Cowlitz County, WA |
B | C |
| douglascountywa.gov Douglas County, WA |
B | C |
| franklincountywa.gov Franklin County, WA |
B | C |
| grantcountywa.gov Grant County, WA |
B | C |
| islandcountywa.gov Island County, WA |
B | C |
| co.jefferson.wa.us Jefferson County, WA |
B | C |
| co.kittitas.wa.us Kittitas County, WA |
F | C |
| klickitatcounty.gov Klickitat County, WA |
B | C |
| lincolncountywa.com Lincoln County, WA |
B | C |
| okanogancounty.gov Okanogan County, WA |
B | C |
| piercecountywa.gov Pierce County, WA |
B | C |
| sanjuancountywa.gov San Juan County, WA |
B | C |
| snohomishcountywa.gov Snohomish County, WA |
B | B |
| thurstoncountywa.gov Thurston County, WA |
C | C |
| co.wahkiakum.wa.us Wahkiakum County, WA |
B | C |
| whitmancounty.gov Whitman County, WA |
B | C |
| yakimacounty.us Yakima County, WA |
B | C |
| berkeleywv.org Berkeley County, WV |
B | C |
| prestoncountywv.gov Preston County, WV |
B | C |
| bayfieldcounty.wi.gov Bayfield County, WI |
B | C |
| burnettcountywi.gov Burnett County, WI |
B | C |
| co.calumet.wi.us Calumet County, WI |
C | D |
| chippewacountywi.gov Chippewa County, WI |
B | C |
| greenbaywi.gov City of Green Bay, WI |
B | C |
| co.door.wi.gov Door County, WI |
B | C |
| douglascountywi.gov Douglas County, WI |
B | C |
| greencountywi.org Green County, WI |
B | C |
| co.iron.wi.gov Iron County, WI |
B | C |
| ocontocountywi.gov Oconto County, WI |
B | C |
| ozaukeecounty.gov Ozaukee County, WI |
B | C |
| co.pierce.wi.us Pierce County, WI |
F | D |
| co.portage.wi.gov Portage County, WI |
C | C |
| co.price.wi.us Price County, WI |
B | C |
| sccwi.gov Saint Croix County, WI |
C | C |
| sawyercounty.gov Sawyer County, WI |
B | C |
| co.walworth.wi.us Walworth County, WI |
B | C |
| albanycountywy.gov Albany County, WY |
C | C |
| campbellcountywy.gov Campbell County, WY |
B | C |
| carboncountywy.gov Carbon County, WY |
B | C |
| conversecountywy.gov Converse County, WY |
B | C |
| goshencountywy.gov Goshen County, WY |
B | C |
| sublettecountywy.gov Sublette County, WY |
B | C |
| tetoncountywy.gov Teton County, WY |
B | C |
| uintacountywy.gov Uinta County, WY |
B | C |
| washakiecountywy.gov Washakie County, WY |
B | C |
About these checks
A robots.txt file tells search engines and AI agents which parts of a site they may crawl and where to find the sitemap. When a site has one, it has to follow the format and not block the whole site. These botability checks test each part of that. A site with no robots.txt file passes both checks, since nothing blocks crawling and there is no file to get wrong. Sites whose robots.txt could not be fetched are left out of the share.
- Robots valid. The site's robots.txt file is written correctly so search engines can follow it. Search engines ignore crawling rules they cannot read.
- Robots allowed. The robots.txt file allows search engines and AI agents to crawl the site. Your site is blocked from search results.
Guidance
- Robots valid standard, with how to fix it
- Robots allowed standard, with how to fix it
- The Web Robots Pages
- Botability rankings
About this data
ScanGov scans the homepage of each county site on our county list. Scores come from the checks in the ScanGov standards. Sites were scanned between 9/30/2026 and 11/8/2026. Each site's report shows its own scan date. A site counts as fully scanned when its homepage loaded and sent us its page content. Averages, grade shares, failure rates, and ranks use fully scanned sites only. We could not fully scan 949 sites. They did not respond, blocked the scanner, or sent no page content. They appear in the table without grades, with their scan status, and are left out of every figure. Grades follow the ScanGov grade scale. Rank follows the ScanGov ranking method. The full dataset, with every county's raw results, is in the ScanGov data repository.
167 fully scanned sites have no result for these checks and are left out of the share.