Rank 18


This idea is active.
Solution Architecture »

Use common digital formats for files

I feel like this is an obvious one but I just tried downloading some data set that has .dbf, .shp, and .shx files. I don't recognize any of these and could not open any on my computer.

Lets make this useful but allowing the data provided to be accessed.


Submitted by 5 years ago

Comments (6)

  1. As directed by objective 1.b in OMB M-10-06, "To the extent practicable and subject to valid restrictions, agencies should publish information online in an open format that can be retrieved, downloaded, indexed, and searched by commonly used web search applications." http://xml.gov/stratml/crane/OGD.xml

    XML is such a format.

    5 years ago
  2. The dataset you are describing is an ESRI Shapefile - it's proprietary to ESRI but in common usage throughout the GIS community and is supported by a wide variety of GIS tools including Open Source geospatial tools and libraries. I would not necessarily advocate getting rid of Shapefiles as they contain feature types of value to the geospatial community, such as geometry (points, lines, polygons) and a spatial index - however it would be useful and valuable for Data.gov to provide information for users on the filetypes hosted and how to use them.

    A shapefile uses a common tabular database format (.dbf) for attribute data, a .shp file for the feature geometry, and the .shx is a spatial index to improve performance - these files are all linked and used by the GIS software together.

    4 years ago
  3. I think the essence of this idea, "common digital formats for files" is good but would modify that statement and rather use "global standard format". Things like MP3, JPEG, PDF and such are examples of global standards. Personally, I would avoid creating something which is US centric. The US exchanges information with many, many countries. For example, I think I heard that the US has tax treaties with 140 countries. The US is not an island, it needs to be part of the global community.

    4 years ago
  4. OMB Circular A-119 directs agencies to participate in the development and use of voluntary consensus standards. http://www.whitehouse.gov/omb/rewrite/circulars/a119/a119.html

    With reference to Charlie's point, it says:

    "This policy does not establish a preference between domestic and international voluntary consensus standards. However, in the interests of promoting trade and implementing the provisions of international treaty agreements, your agency should consider international standards in procurement and regulatory applications."

    It would be good to strengthen that guidance, at least by deleting the first sentence.

    4 years ago
  5. As Dave Smith said, Shapefiles are a global and industry 'standard' in that most geospatial tools can utilize them.

    You can upload them to something like GeoCommons (disclaimer: I work on GeoCommons) to convert to spreadsheet or KML.

    I agree that it would be very useful to share out the data from Data.gov is as many formats as make sense - and there isn't really anything preventing the data from being shared in multiple formats.

    4 years ago
  6. tegoulet,

    David Smith does a good job of explaining what shapefiles are, etc. As a GIS (map oriented) professional, I use shapefiles every day.

    Shapefiles can be transformed into other formats, such as .kml (viewable in Google Earth) that are more accessible to non-mapping folks. That might be a project that a volunteer or Data.gov could take on.

    In the meantime, please, let's not throw away valuable data sets just because they are not in a format that some visitors to the portal can't use or don't understand.

    4 years ago

Vote Activity Show

(latest 20 votes)