BPM process for geocoding with OSM

Basics

A C0 process for geocoding address data in CRM is delivered as standard. This uses "System Start (Webservice, Timer, ProcessUtils.startProcess)" as the start event and can therefore be executed regularly via a timer.

If the geocoding process is used in conjunction with geoanalysis, it must be ensured that the runtimes of the timers for geocoding and updating the geoanalysis tables are coordinated.
It should be ensured that the timer for geocoding the addresses runs before the timer for updating the geoanalysis tables.

The main logic of the process is relocated in the script library SC0Geocoding.library.

The process is executed in the CRM system, all address datasets are first used for geocoding, provided they are complete. By default, the process encodes all active addresses when the following fields are filled:

ZIP (ZIPKey.Address)
City (City.Address)
Street (Street.Address)
Street Number (StreetNumber.Address)

If available, the house number supplement (StreetNumberAddition.Address) is taken into account. In the event that no coordinates could be determined for a transferred house number supplement via the service, an attempt is made to geocode the address including the house number, but without the house number supplement, in a new call to the geocoding service.

If, for example, addresses that do not contain a house number or house number supplement but only the street are to be geocoded, the search for determining the data to be geocoded must be overridden accordingly. Furthermore, the SC0GeocodeUtils delivered in the script library must be adapted accordingly in this case, since in the standard implementation responses of the geocoding service, which only provide coordinates for a street, but not explicitly for a building or the like, are excluded.

If in your system the house numbers are not stored in the house number field, but together with the street in the street field, you can change the condition for StreetNumber.Address in the search to NO_CONDITION in the script library SC0Geocoding in the method createAddressSearch.

If an error occurs when attempting to geocode an address, a distinction is made within the process between

The service cannot communicate outwards (IO-Exeption)
Possible reasons for this are
- The URL is not unlocked on your used proxy
- The invocation is blocked by the firewall because the URL is not unlocked there.
- There is no internet connection
- The SSL certificate is expired or incorrectly configured
- The geocoding service is not available (e.g. because of maintenance works)
The geocoding service returns the information for the transferred address data that the coordinates cannot be found.

In case of an error the "Geocoding State" field (GeocodeState.Address) can be used to dissipate which of the two cases mentioned above applies to an address. If the service could not communicate externally, the address receives the"Geocoding status" = 0 or null, but if it is an address for which the OSM geocoding service has not found any coordinates, the value of "Geocoding status" is set to'11'.

Successfully geocoded addresses received the value '100' for the field "Geocode State" in addition to the coordinates for the line of longitude and latitude.

The process reacts to saving an address and clears the fields Lng (longitude), Lat (latitude) and GeocodeState (Geocode State) when a relevant address field is changed, so that the address is geocoded again the next time the process is run.

The geocoding process was implemented as a mass data process. This means that if required - i.e. at least one address must be geocoded - the process inserts a mass data task that performs the geocoding. If such a process is already running, no further processes are added.

Figure: Geocoding process in the job queue

An entry in the mass data tasks provides information on the status of the process within the processing duration.

Figure: Logging of the process flow in the mass data tasks

Here, for example, you can see how many results were processed by the process. The detailed view of a mass data task shows whether the process was successfully executed or whether an error occurred during processing. If the geocoding process runs on an error, no further process is started until the failed instance has been deleted.

How often addresses are geocoded using the process in your system depends on how many addresses are created or changed daily in your system and how quickly you want to enrich, use and evaluate them with coordinates (for example in geoanalysis). To find the most current coordinates, we recommend you to run the process at least once a day using a timer call.

Used global variables

C0Geokodierung_isSecureHttp

<empty>: Access to geocoding by https

<empty>: Access to geocoding by http (default value)

Specifies whether geocoding is accessed by an http or https connection.

C0Geokodierung_NominatimBaseUrl

<empty>: Uses //nominatim.openstreetmap.org (default value)
Free-of-charge, public Nominatim service.
This is limited in the number of daily geocoding calls.

If you have licensed Geoanalysis, please contact your project manager.
Here, the global variable must be overloaded and the value adjusted in order to use the CURSOR geocoding service.

If necessary, another activation of a URL on the proxy/firewall must take place here after the customization of the global variable.

The number of daily geocoding calls is not limited.

Base URL for geocoding with Nominatim. Specifies the Nominatim service by which the geocoding of data successes is to be performed.

Geoanalysis_MapProvider

<empty>: Map material about public OSM map server

<empty>: Map material about the private CURSOR-OSM map server at Sylphen

Specifies from which service provider the map tiles are obtained for layout in geoanalysis.

Technical information

Activation of URLs

To use geocoding, the following URL must be accessible from the system and, if necessary, be enabled for the proxy and firewall by the customer:

https://nominatim.openstreetmap.org/*

URL for the public Nominatim geocoding service

SSL certificate

The business partner must have a valid SSL certificate installed.

Interface Overview

When using interfaces that edit or create new entries in the address table, please note:

The interface/Web service must be adapted to empty the coordinates (Lng.Address, Lat. Address) and GeocodeState.Address in case of address changes, so that changed addresses are taken into account during the update run of the geocoding process.

There must be no (database-side) write protection in the fields Lat.Address, Lng.Address, GeocodeState.Adress, UpdateDate.Adress and UpdateUser.Adress.
The geocoding process must be allowed to write to these fields of the address table.

You have to be aware that the geocoding process changes the update date as well as the update user of address datasets

When starting the process for address geocoding via timer, make sure that it does not run simultaneously with interfaces that also work on the address table.
The geocoding process should only start after the interface run has been completed.

Customization options

If the street number is in your system together with the street name in the Street field of the address instead of separately in the Street Numberfield, the process can be customized to suit the customer so that the address is also geocoded. However, in the best case the coding service recognizes the street, the house number is then ignored, so that the coding is only valid for the street.