Data Harmonization

UNCEFACT defines data harmonization as an iterative process of capturing, defining, analysing and reconciling government information requirements, and data standardization as the mapping of this simplified data to international standards (*).

Relevance to trade facilitation

Throughout a cross-border trade transaction, traders and other transport and logistics service providers have to provide information to many regulatory agencies and other partners along the supply chain. The information needed concerns goods, their packaging, weight and height, and means of transportation, and details of the importer and exporter. Hence, a multitude of different trade documents, still mainly paper format, are exchanged. Parties spend much time preparing the data, filling in the documents and submitting them. When data requirements are not harmonized and standardized, each agency and each document may have different requirements. Data is understood in various ways, and may not have the same representation. This increases the complexity of managing information requirements and increases the likelihood of errors. Differences in data sets also inhibit exchange across agencies, so that the same information often has to be submitted several times.

Benefits

Data harmonization and standardization facilitates the submission and processing of trade information (documents and data). It can help to:

  • reduce information requirements by eliminating redundancies and duplications, thus making the submission easier,
  • improve the quality of the data and therefore reduce errors,
  • facilitate receiving, processing and checking of information, and
  • facilitate exchange of data and improve automation as this ensures inter-operability.

Data harmonization is an important aspect of any automation project, in particular for a Single Window for Trade, for the migration to paperless trade, and for document alignment.

Data harmonization process

Data harmonization "involves a set of activities that improve the consistency in the use of data elements in terms of their meaning and representation format" (*). It is usually undertaken at the semantic level before considering document structures. Later the message syntax can be created from standard naming rules that may be part of standard Technical Specifications. This ensures the message syntax is also harmonized when derived from the semantics using a naming and design rule (NDR) as part of standards-based Technical Specifications.

There are currently two guides that explain the different activities and tools that can be used for data harmonization (see Solutions and Tools). The process usually starts with establishing an inventory of the current data requirements, definition of the data collected, analysis of the information requirements and data elements, and reconciliation of the data (i.e. consolidation of the defined and analysed trade data and alignment to international standards). The result of these steps is a simplified, standardized national data set so that e-documents can be developed.

Solutions and tools

A data harmonization process as summarized above ideally uses Business Process Analysis in the capture phase, to identify and map information and document requirements, and document and data flows. This analysis reveals a clear picture of the current situation and enables identification of redundancies.
The process of defining and analysing the data should integrate international standards, such as UNTDED and recommended code lists. UNTDED is a dictionary of trade data elements that provides clear and unambiguous identification of data elements (a data element name, description and four-digit number) in the analysed trade documents. Referencing the UNTDED ensures consistency. In the reconciliation phase and for the development of e-documents, other reference data models can be used to map the trade data. In cases where standards exist for messages within the project domain, the process should consider any harmonization that has already taken place. It is often better to reuse existing semantic elements than to develop a new set. Therefore the data should be represented in a Data Model such as the UN Core Component Library or WCO Data Model. This not only helps to validate the process but assures better inter-operability with systems already using the standards.

Practical guides

UNNExT, UNCEFACT and WCO have developed practical guides that can guide and assist in data harmonization projects:
UNNExT Data Harmonization and Modelling Guide
UNECE/UNCEFACT Recommendation No. 34
WCO Data Model, SINGLE WINDOW DATA HARMONIZATION