MN GIS/LIS NEWS
WINTER 2006-7
ISSUE 47

MN GIS/LIS NEWS
The Newsletter of the Minnesota GIS/LIS Consortium

Table of Contents

MN GIS/LIS Consortium
2006 Conference Wrap-up
From the Chair
Scholarship Competition Results
Conference Auction Results
What is GISP?
GIS Rules of Conduct
MN GIS Speakers Bureau
North Carolinian Attends Conference

State
2006 Air Photos
Geospatial Services Survey
Gopher State One Call

Governor's Council
Committee Priorities 

Regional
MetroGIS DataFinder Improved
Housing/Jobs Mapping Site
Metro Land Conversion

Local
GIS for Surveyors

Federal
2010 Census Address Program
2010 Census Options
National Land Cover Database 

Higher Education
St. Mary's Update CEU's

Non-Profits
GITA Announces Scholarship

Other Places
Benefits of Parcel WMS

 


 

Get Ready for Real Time:  Up-to-the Minute Data Exchange in Urban Information Systems Part II
By Greg Sanders, Web Projects Manager, Chicago Metropolitan Agency for Planning, Reprinted from URISA News September/October 2006 issue. With permission.

 

 

Major sections of this article:

Introduction

How it works

Why web services make sense for IT staffs

 

 

 

Service-Oriented Architecture (SOA) and Enterprise Architecture (EA)

A Chicago web services network

Web service networks at the federal level

Web service networks at the state level

Bridging the gap between federal, state and local agencies

Barriers to web service implementation

A note on GML and PMML

Conclusion

 

A Chicago web services network
Data sharing via XML has such potential that in Chicago we formed a partnership of government and non-profit agencies dedicated to exchanging data in real time via web services. The partnership includes the City of Chicago, the Cook County Assessor and CMAP as well as several non-profit planning and research organizations (see www.chidataexchange.net). We call the partnership Illinois Data Exchange Affiliates (IDEA), optimistically hoping that our Cook County coalition will expand into a statewide network of linked services in the near future.

 

Chicago is not the only metropolitan area investing in web services for tracking administrative data. The District of Columbia’s DCStats (see http://dcstat.octo.dc.gov/dcstat/cwp/view,a,1204,q,491690.asp) currently sets the standard for real-time web services in municipal data systems. The DCStats website calls this a growing trend: “The District of Columbia is joining the national trend of local governments by providing public access to city operational data directly through the Internet. The District will stream data that the agencies gather through normal operations. 


IDEA works on both technical and the policy questions related to data sharing. Many unanswered questions remain on the policy side, concerning security and confidentiality, obligations of public agencies to make data accessible via the web, and other thorny issues. On the technical side, IDEA recommends best practices and holds information sessions to encourage the deployment of web services.

 

IDEA’s core messages are:

  • Government works best when information is shared across divisions
  • Society works best when public information is widely available
  • Web technology gives unprecedented opportunities for making data available
  • Ensuring access to public data requires clear guidelines on how, when and with whom data is to be shared

 

As a result of the IDEA collaborative, CMAP’s partners from planning and development agencies can log in to our web site and see not only all the data that CMAP has on file, but also a batch of data fields retrieved in real time from the Cook County Assessor’s office. The latest data, straight from the source. Since the Assessor’s teams continuously sweep through various parts of the county updating assessments, this is an important feature. Other IDEA-enabled web sites include data applications developed by non-profit organizations to fetch building permits or property transactions matching user-entered criteria from city and county databases.

 

John Karnuth, Deputy Commissioner of the City’s department of Planning and Development, says “Generous sharing of information and its standardization contributes to better decision-making, efficient program development and delivery of services. The mission and purpose behind IDEA is to create policies and standards that will allow for the free flow of information between government agencies, not-for-profit organizations, community groups, average citizens, and even business interests.”

 

Derrick Thomas, IT Director for the Cook County Assessor’s office, says “sharing data with IDEA via highly efficient web services helps the Assessor's Office maintain a level of transparency, as we pride ourselves on providing accurate data and fair assessments.”

 

Cliff Wagner of Topiary Communications, who coded the Cook County Assessor’s web service, says the service was designed for agility and easy re-use. “My goal was to design something that was flexible. Right now it’s going to be feeding off a SQL Server and the primary consumer of the data will be CMAP. But [the Assessor’s office] already knew that there were other people who want access to the data. The database component I have completely separated, so that if they need to switch over to IBM or Oracle, they would be able to quickly port this over to another system.”

 

“The big thing was setting up an authentication system where based on the user, different things are possible,” Wagner says. “One thing they’ve suffered through recently is data miners who essentially screen-scrape their application. Their service gets hit on a regular basis by lawyers and real estate-related industries. They started cutting those people off, but they hope to offer the web service so that a lawyer in Evanston, for instance, could subscribe to the service--and for less than the cost of buying the full tables of all the county data, they would be able to get only Evanston data. That’s all that that they want so that’s all they would pay for. It would give the opportunity for more of an a la carte approach, and it would also allow for limiting the number of searches that an organization could pull in a day.”

 

Web service networks at the federal level

 

 

Niemann went on to chair the federal Chief Information Officers (CIO) Council’s XML Web Services Working Group. As the EPA’s self-described “web services evangelist,” Niemann knows both the power and the problems associated with broad web services implementations. Over the past few years web services have become widespread in the federal government, from EPA to the Departments of Justice and Homeland Security.

 

In 2002 the E-Government Act was signed into law, requiring, among other things, that the Office of Management and Budget establish “policies which shall set the framework for information technology standards for the Federal Government developed by the National Institute of Standards and Technology ... maximizing the use of commercial standards as appropriate, including the following:

(A) Standards and guidelines for interconnectivity and interoperability as described under section 3504 (italics added). 

(B) Consistent with the process under section 207(d) of the E-Government Act of 2002, standards and guidelines for categorizing Federal Government electronic information to enable efficient use of technologies, such as through the use of extensible markup language (italics added). 

(C) Standards and guidelines for Federal Government computer system efficiency and security” (italics added).

 For federal project administrators, some of the biggest challenges have been (a) motivating data managers to adopt standard practices, and (b) achieving “semantic interoperability” across governmental divisions. As for motivating data managers to participate, Niemann says, “You’ve got to have a people process to make that happen. We call that ‘communities of practice.’ You bring them together, show them this new way of doing things--and that you have a mandate. Carry it out collectively: everybody steps up to do a little piece of it. You begin to see win-win situations. People discover the benefit of it through networking.”

 

But with widespread participation come new headaches. “We knew our own vocabulary, but we weren’t working with anyone else’s data. So we had to solve the problem of what we call semantic interoperability.” Semantic interoperability refers to the mapping of terminologies from one “namespace” (department, community, discipline, etc.) to another namespace. Data applications that cross organizational divisions usually encounter differences in terminology, ontology and taxonomy. Semantic interoperability is what allows these applications to structure such diverse information in meaningful ways.

 

Today, EPA manages the National Environmental Information Exchange Network (www.epa.gov/onestop/index.html) for exchanging data from the federal to the state and local levels.

Meanwhile, in 2002 the Department of Justice (DOJ) put together a task force of 32 federal, state, local and international organizations to design an XML standard specifically for criminal justice. The result was the Global Justice XML Data Model (JXDM). The purpose of the Global JXDM was “to provide a consistent, extensible, maintainable XML schema reference specification for data elements and types that represent the data requirements of the general justice and public safety communities.” The JXDM was specifically intended to link the federal DOJ with their state partners.

In 2005 the U.S. Department of Homeland Security (DHS) joined DOJ in announcing the first release of the National Information Exchange Model (NIEM). NIEM "establishes a single standard XML foundation for exchanging information between DHS, DOJ, and supporting domains, such as Justice, Emergency Management, and Intelligence." The NIEM was designed to expand the Global JXDM data model, adding new data components while maintaining compatibility with JXDM.

Web service networks at the state level

 

 

In 2005, the National Association of State CIOs (NASCIO) issued a “call to action,” advocating information sharing within and between state governments. Eric Sweden, editor of the NASCIO publication PERSPECTIVES - Government Information Sharing: Calls to Action says NASCIO’s position speaks to “the need to share information across government lines of business, and also to share information across various levels of government: from federal to state to local government. Some of the circumstances that are driving that need include public health and the war on terrorism. In pandemics, epidemics, terrorist events, we have to have fast, effective communication across jurisdictions to prevent, mitigate or respond to these kinds of events. That’s what NIEM is all about.” 

 

Examples of web service implementations here in Illinois include the Illinois workNet web system. Several departments each owned a subset of the data pertaining to federal employment training programs, according to Illinois workNet Coordinator Jeanne Kitchens. By adopting services, the various data sources could continue to be maintained by their current stewards, but accessed by all the partners.

 

Bridging the gap between federal, state and local agencies

 

 

Barriers to web service implementation

Three big challenges must be overcome before a vibrant web services network can be created. First, security and confidentiality must be managed; second, data definitions and terminology must be agreed upon and standardized; and third, a general reluctance to share data must be overcome.

 

Managing security is always a concern, Sweden says. “That is a requirement when we talk about information sharing—maintaining security, privacy, authentication…. Along with that information going from agency to agency, we’ve got to be sending information about authentication and authorization: the metadata regarding security and privacy.”

 

Still, there is always the risk that data will fall into the wrong hands whenever it leaves the home office. This is not unique to web services, but is inherent in all data sharing. “Opening up information carries risk,” says NASCIO’s Eric Sweden. “You will never get to zero risk.” But, Sweden adds, “Information that is not shared cannot be used. Think of information as an asset. Are we getting full value out of the asset?”

 

Agreeing on data definitions can be as daunting as wrestling with confidentiality concerns. If you think concepts like “address” are simple and straightforward, read the Federal Geographic Data Committee’s “Street Address Data Standard” (http://urisa.org/node/227). Be sure to read and memorize all four parts.

 

Finally, many public agencies are unsure of their obligations to provide or withhold access to data. The Freedom of Information Act (the federal statute and its state-level counterparts) is generally vague about the obligation to provide web access to digital records. Some agencies struggle with HIPAA (Health Insurance Portability and Accountability Act of 1996) and counter-terrorism guidelines that restrict the release of some data.

 

A note on GML and PMML

XML’s great strength—it’s almost infinite extensibility—also dictates that to be useful, XML schemas must be agreed upon by all parties in an exchange network. That’s why major IT industry players have banded together to hammer out specific XML standards for specific purposes. Two of these are Geospatial Markup Language (GML) and Predictive Model Markup Language (PMML). GML allows geospatial data to be transported as XML, while PMML does the same thing for statistical and data mining models. Both these technologies are maturing to the point where we are starting to see applications in production. Either could be the subject of dedicated articles, but I at least wanted to mention them here.

 

Conclusion

The time is now for real-time data linkages among public entities, and real-time data feeds to the public. As long as sufficient attention is paid to issues of standards and security, data sharing becomes relatively simple and sustainable. Government efficiency, accountability and performance can be enhanced, while the public’s role in deriving meaning from the available data can be encouraged.  

 

There is a growing realization that no government data system is complete unless it connects with all tiers from federal to local. The NIEM project is a good example: state CIOs have gotten involved with this federal effort in order to link their own networks with those of their counterparts at other levels. Sweden says, “The NIEM project is actively seeking participation from state and local governments in this initiative. The interests of state and local government are extremely important to the NIEM initiative. NIEM has named a director of outreach and he is specifically devoted to state and local government, to gain their active participation.” NIEM Outreach Director Tom O’Reilly can be reached at toreilly@ncja or 202-204-6026.
While serving as the state of Utah’s CIO, Phil Windley published a “Web Services Manifesto” (see www.windley.com/archives/2002/07/03.shtml), encouraging state agencies to post information via web services whenever possible: “Let's face it, if we're going to build an application that lets someone query a database, it’s a shame not to return XML since we can do it for little additional cost and the potential benefits are huge.” The policy implications were clear: “All queries for data from a web server should produce at least XML. If human readability is required, post process the XML with XSLT.  As an example, if I go to the professional licensing division and query about doctors, the application should, at a minimum, produce XML.” In 2001, the EPA’s Brand Niemann had a small problem with an expensive solution. “We had a need to upgrade a web site for the Local Emergency Planning Committee or LEPC to put this database on the web and eventually to make it accessible by telephone,” Niemann says. “Like many software projects, we asked the contractor to estimate the project. But EPA needed to do it for less money and deliver it faster. At about the same time, I became aware of VoiceXML…. I said, ‘we can do this for $2000 with in-house stuff in a few weeks…. The result was that you could dial an 800 number, enter your ZIP code, and it would read the database.”Why web services make for better governmentWhat is XML?