Rank30

Idea#88

This idea is active.
Solution Architecture »

Improve ability to mashup and integrate data sets

It would be helpful if there were some standard formats for the data sets that are posted on data.gov. It would make it easier to create mashups and to integrate data across data sets. In addition, standards for key fields are needed.

Comment

Submitted by jenkins.lisa 4 years ago

Vote Activity Show

(latest 20 votes)

Comments (4)

  1. The ability to create mashups is fundamental in my view. The Semantic Web notion of "linked data" takes this even further. I would go futher than saying "...if there were some stanard formats for the data sets..." and say ALL data sets should be expressed in global standard formats. One example of this is XBRL. Another is RDF/OWL. Both of these formats (XBRL and RDF/OWL) use graphs to express information which are very flexible. These graphs would be hard for some to use, but if they were the low level formats with higher level "renderings" of these low level formats in easier to use forms then you have the best of both worlds.

    4 years ago
  2. Initially, Data.gov just provides the metadata catalog - the discovery component.

    Mashups are facilitated by how data is published. Toward being able to build mashups, data must be dynamically accessible, e.g. via XML or RESTful web services, et cetera.

    Data.gov can and should catalog service endpoints, APIs and so on which provide these types of "mashable" services.

    I would like to think that in the longterm, there is also a role to be played in Data.gov where these types of services can be directly hosted or generated by Data.gov via reusable infrastructure. This theme would tie in with two other associated ideas: Data Hosting for National Datasets http://datagov.ideascale.com/a/dtd/16754-6440 and Data Enhancement/Manipulation Capabilities http://datagov.ideascale.com/a/dtd/16764-6440 - those two would go far toward enabling mashups using Data.gov as part of the framework.

    4 years ago
  3. Mashups need to be hosted on a single server due to cross site scripting issues. It would be great to have a mashup server for gov data, but people will have to be able to bring in their own data.

    4 years ago
  4. Data don't necessarily need to all be hosted on the same server to build mashups - I regularly build mashups using data hosted on multiple servers.

    Regarding cross-site scripting issues, there are two pieces to this, functional and security.

    From a JavaScript functional standpoint, yes, XML pulled directly from external sites is blocked due to security concerns, however RESTful APIs, JSON et cetera can still be accessed easily and directly from remote servers. And there is a workaround for XML if one proxies the remote data locally or uses other means. Similarly, there are a variety of ways to mash up remote data using Flash, Silverlight and other technologies.

    From a security standpoint, the original concern which led to blocking of remote XMLHttpRequests from JavaScript is that there may be possibility of malicious information on the remote server which could have unintended or harmful impacts. While true, it broke a lot of previously-elegant functionality. This is also of course true of JSON and other technologies. Here, trust relationships and other approaches should come into play rather that breaking functionality outright and having to resort to kludgy workarounds.

    4 years ago