DatMap
is run by double clicking on the DatMap icon or on its file name in the
same way as any Windows application. It does not need to be installed.
DatMap is a Windows application which enables the production of distribution maps from raw data. The data can be provided in a variety of formats and the mapping can be based on a wide range of parameters defined by the user.
DatMap is a single Windows executable file (DatMap.exe) that
calls on standard Windows facilities to control file opening, printing,
etc.
The executable is provided as part of a single Zip file (DatMap.zip)
together with its help text (DatMapHelp.html), style sheet (DatMapSS.css)
and image files (DatMap*.gif).
The application is provided as Freeware with no warranty. You are free to
use DatMap, but I would ask that a) its use is acknowledged, and b) if
distributed further, it is distibuted as the complete Zip file which
includes the Help files. Copies of the Zip file and an email link for
comments or suggestions can be found on my web site at www.middlea.freeserve.co.uk.
A registration facility has been built into DatMap to enable the author to
inform users of any changes. A registration key can be obtained by emailing
the author (see email link). Registration will
remove the nag screen shown when the application is opened. No facilities
are denied to un-registered users!
Mapping is based on an orthogonal grid such as the UK National Grid Reference (**), and the data can be statistical data relating to grid squares (e.g. land use, observations per grid square, population, etc.), or can be specific data for 'exact' locations (e.g. sightings of particular species). A limited amount of data aggregation (Sum, Average, Max, Min, Count) can also be done within DatMap with the results being plotted by grid square.
(** Throughout these notes, the terms Eastings and Northings are used to denote the axes of the grid - i.e. as used in UK NGR. In practice they can refer to any such grid, such as Latitude/Longitude or another local grid system.)
Data can be provided in a number of text formats - in a fixed format, or delimited by special characters. Each record should contain a 'Tag' of up to 10 characters indicting the type of data, though this can be defined separately for the whole file. The record should provide two values defining the grid coordinates for the grid square or location, and a numeric data value. Grid references and data values can be of any appropriate scale or precision. If data values are omitted, they are defaulted as 0.
Grid Coordinates can alternatively be provided using the UK Ordnance Survey NGR format, i.e. using the 2-letter 'prefix', e.g. SK123456.
The data needs to be imported into a DatMap database file before producing a map. Several sets can be built into a single database to allow maps to be produced easily using related data and sets of parameters.
The data can be plotted using a variety of symbols of specified sizes and colours in up to 10 bands or categories. The bands are defined by the data values held within or aggregated from the data (e.g. these may be statistical values, but could be treated as codes for specific types of observations).
For 'grid' mapping, the grid squares are of a user defined size which also defines the symbol size range available. For 'exact' mapping, the parameter only defines the maximum symbol size.
The user can also define the area covered by the map, text for map titles and subtitles, the spacing of grid lines, fonts, colours, etc.
A background image can also be defined to provide either a background logo, colour field or base map. The 'background' can optionally be placed over the mapped data as a transparent overlay - white pixels in the image will be treated as transparent.
Parameters for the data and mapping can be saved in a variety of groups in separate settings files and these sets can also be re-loaded in any combination as required.
DatMap was written to provide flexible mapping of data using a stand-alone Windows application. It is based on an earlier facility (DisMap) written using Visual Basic within MS Access. DisMap was limited by its dependence on Access, though Access did provide all of the data handling required, and DisMap took its data directly from Access tables. Though not maintained, DisMap is still available from the author's web site.
Note that DATMAP is also a registered name used by DatMap Corporation (a US Company) for their (professional) GIS based mapping software - 'Computer software for use in the field of sales forecasting that analyzes and displays demographic, sales, market, and market research data for use in site selection'. Their president has been in touch, and I have gladly agreed to include this note highlighting the clash of names and providing a link to their web site www.datmap.com for more information (the web site is currently being redesigned 11/2007).
DatMap has been written using REALbasic, an object oriented integrated development environment (IDE) available from REALsoftware.com. This includes its own relational database facilities based on SQLite. REALbasic produces self contained applications that call upon standard system resources for file opening, printing, etc. DatMap has been developed using these facilities to include its own data handling and storage, as well as its mapping functions.
Though REALbasic can be used to compile for Linux and Mac platforms, DatMap has currently only been built for Windows.
Version details and a list of changes are given here. The current version is shown in the Help>About menu option as 'Major version.minor version.bug version [stage (build)]' (stage and build are only given on test versions).
DatMap
is run by double clicking on the DatMap icon or on its file name in the
same way as any Windows application. It does not need to be installed.
If DatMap has not been registered, a 'nag' screen will appear before the main window is opened.
DatMap opens with its Main Window which shows a tabbed dialog box through which the user can control its operation. The various tabs are described below, and hovering the cursor over a tab will display a brief explanatory message. In general, the tabs step from left to right through the process of defining a map, but they can be selected in any order.
The window also has a menu bar through which the application can be closed (File>Exit) or the Help file can be accessed. A new registration key can also be entered from the Help menu.
Control keys can also be used to close the applocation (Ctrl/X) and to open the Help Browser (Ctrl/H).
DatMap is best run on a monitor with 800x600 pixel resolution or higher.
When entering data, numeric fields are validated as they are typed. If the characters typed are not valid they are ignored. If the number falls outside the expected range it will be highlighted in red while typing and set to the upper or lower limit of the range once focus moves to another control. If the field is left empty, it is set to a default value which will be an appropriate value (or may itself be blank!).
The use of the Settings Tab is perhaps better explained after the user has an understanding of the other tabs and their data fields. The new user may therefore wish to skip this section and return to it later. In general use, the Settings Tab may be the first tab required to load settings saved from previous DatMap sessions.
Settings can be saved in a variety of 'groups' in a series of settings files (*.dms). The 'groups' relate to the main window tabs ...
Any combination of 'Select setting groups to be saved' check boxes can be ticked, and clicking the Save Settings button will save the settings to a '.dms' file defined by the user. As well as being written to the .dms file, the saved settings will also be available for reloading later in the session without reselecting the file.
The .dms file is an XML file that can be read as text by the user. Though it can be editted, this is done at your own risk!
To reload saved settings, an existing .dms file is first selected by clicking the Select Saved Settings File button. The settings it contains are shown in the text box below the button, and the appropriate 'load' check boxes are enabled. DatMap will also ask whether all of these settings should be loaded immediately. Alternatively (or later in the session), some or all of the check boxes can be deselected/reselected, and clicking the Load Settings button will load only the selected groups. Settings groups can therefore be saved and reloaded in any combination.
DatMap uses data held in a DatMap database file. These are created or selected through the Database Tab, and are populated with data through the Import Tab
A new database (.dmd) file can be created by clicking the Create Database File button - its path and name will then be shown in the window.
An existing database file can be selected using the Select Database File button (or via a Settings file) - its name and contents will then be shown in the window.
These files contain two SQL tables - Tags and Data. Short alpha-numeric 'Tags' are used to identify sets of data within the database (e.g. for different statistics) and provide some information that effectively documents the Tag and thus the Data. This information also provides defaults for use in the mapping, i.e. ...
The 'data' comprises a set of records including ...
As data is imported to the database, the tags are listed in this window and the tag details can be editted in situ - NOTE that changes are applied directly to the database. Grid/Exact can be changed by entering the full or part word (an invalid entry is changed to Grid), and the grid size, title and sub-title can be changed by entering new values in the appropriate cell.
Once a Tag is selected, the user can also Browse the data for that Tag (see below re. the Browse Window).
The Tag details and Data can also be Exported to a text file (a '.dme' file). This may be of use if the original source has been lost, but it also provides a simple way to transfer data between databases (via the Export Format in the Import Tab). The text file consists of a comma separated header record showing the Tag details, and a series of comma separated data records. To avoid confusion any commas within the default title and sub-title will be changed to semi-colons!
The selected Tag and its associated data can also be Deleted if no longer required.
Data must be imported into the current database created or selected in the Database Tab before it can be used by DatMap.
The user should first select the required source data file. This will generally be a '.txt' or '.csv' file, but can be a '.dme' export file created from DatMap, or any other file type. The contents of the file will be shown in the list box.
Data can be imported from a variety of text file formats ...
Each record will provide the four fields required for the database ...
The Tag (of up to 10 characters) can alternatively be set for the entire file and omitted from the records. If set within the records, several different tags can be included in the single file.
Eastings and Northings can be provided as a single field using the UK OS NGR format (e.g. SK123456). Internal spaces will be ignored (i.e. SK 123 456 will be accepted). The NGR can be at any precision from 100km (SK) to 1m (SK1234567890), though the four digit (1km) or six digit (100m) are most commonly used. The Import therefore expects an even number of digits, but if necessary will assume the Northings portion is missing a final '0' (i.e. SK12345 will be imported as SK123450). Whatever the precision of the incoming data, the NGR will be held within DatMap in 1km units (e.g. SK123456 will be stored as 412.3,345.6). The NGR letters can be re-generated in the final maps if required (see Map Details Tab).
Grid coordinates and data values can be of any appropriate scale or precision.
For a delimited file, the fields will normally be provided in the order shown above (e.g. 'Tag, E, N, Value' or 'Tag, NGR, Value') but this can be overridden using the field order boxes. Leaving these as all zeros will use the default order. Using appropriate values can also force fields to be skipped (e.g. 1,3,5).
For a fixed format file, the fields cannot overlap, but can be in any order and do not need to fill the record.
If a numeric field is missing (blank) it will be defaulted to zero. If an Eastings or Northings field is missing or zero, a warning will be given and the import can continue (optionally suppressing further warnings) or can be stopped. If any zero data values are found, the user will be asked whether such zero values should be loaded to the database - if they are included they can still be omitted from the mapping in the Map Details Tab.
If the Tag included in data check box is un-checked, the tag string should be entered in the box which will then be enabled.
The Coordinate Import Format can be set to Eastings/Northings (default) or OS NGR.
Delimited, Fixed or Export format can be selected using the radio buttons, and the appropriate parameter boxes will be enabled. For Delimited format the Field boxes can be left as zero to use the default order, but the default values given for Fixed format are purely for illustration, they should be amended as required.
Having entered the required parameters, the user should click on the Parse Data before Import button so that the parameters and data can be validated. The parsed data records will be shown in the list box and can be checked before being imported.
Having successfully parsed the source data, and if a database file is open, the Import Parsed Data button will be enabled. Clicking the button will first check for existing data for each tag - the data can optionally be deleted before the import. When a new tag is added to the database it is added with default details, unless the source was a DatMap Export, when the exported details will be used.
On completion of the Import, the Database Tab is opened to allow the default Tag details to be amended as required.
Before a map can be created, some basic details are required via the Map Details Tab.
The tab is split into two main parts - Map Data and Map Area - which are also reflected in the settings groups that can be saved and loaded using the Settings Tab.
Map Data defines the database tag (i.e. data) on which the map will be based, and the basic format of the map.
Select Tag is a pull down list of the tags in the current open database. On opening a database any previous selection is cleared and the list is refreshed. If a tag is already selected, the list is not updated automatically as data is imported or deleted, but it does includes a Refresh List option to force a refresh. The required tag has to be selected before a map can be created.
Having selected a tag, the Set Defaults from Database button will populate the Map Title, Sub-title and other boxes. It will also show the range of Eastings and Northings values held in the database in brackets in the Map Area part of the tab - this is for information only, but if no Map Area has been defined previously, these values will also be used to define a default Map Area.
A suitable Map Title and Subtitle can be defined or the default can be used.
The mapping mode 'Grid/Exact' can be defined as required. In Grid mode, data will be mapped in a grid defined by the Grid/Symbol Size, each symbol will be centred within the grid square and will reflect the data for the square. The data may already have been imported with Eastings/Northings values reflecting this grid (i.e. the south-west corner of grid squares), but DatMap does not make this assumption. Data records will be mapped to the appropriate grid square and multiple records will be aggregated - the aggregation being defined by the Grid Aggregation box - i.e. Sum, Average, Minimum, Maximum, or Count. For example, even if a data value was missing from the import source data and the data values loaded were all zero, the mapping could be based on the number of records (e.g. observations) falling into each grid square.
In Exact mode, each data record is mapped by a symbol centred at the specified location. The Grid/Symbol Size is then used to define the maximum symbol size in terms of map grid units.
Where the data value to be mapped in a grid square (after any defined aggregation), or at a specified location, is Zero, the data can optionally be included or (by default) omitted from the mapping.
Reference grid values can be included on the map margins at a spacing defined by Reference Grid Size. By default, grid lines will also be drawn at the same spacing, but can optionally be omitted (e.g. if a background image is added in the Background Tab which itself includes grid lines).
If
the coordinate data is held in DatMap as 1km units on the UK OS grid,
the marginal numbers can be modified and OS NGR letters added to the body
of the map. By selecting OS NGR labels, the 100km units will be
shown in a reduced font (cf. at the corners of OS maps) (e.g.
345), and the 100km OS Grid leters will be added
in the body of the map in the SW corner of the map and of each 100km square
(see example on right). Negative values are outside the normal OS NGR
range and will use the normal font.
The Map Area part of the window defines the range of grid coordinates for the map. This also defines the selection of data from the database. The range of values given by 'set defaults' can be used as a reference. In 'grid' mode the upper limits are omitted from the selection (logically these values would be mapped into squares outside the map area), while in 'exact' mode they are included. In 'exact' mode these values are used for the data selection, but the map area will be slightly larger to accommodate symbols for data points at the very edge of the area.
A Background image for the map can be specified in this tab. This could be a logo, copyright message, colour field, or a base map. If not defined, the map is drawn on the plain white background of the page
The image file is selected using the Select Background Image button. The dialog will initially select from files of types *.bmp, *.jpg, *.jpeg and *.gif. Other image types (e.g. TIFF) can also be opened if Quicktime is installed on the computer - an 'All Files' option is therefore available in the dialog, but the selected file may not open correctly.
The selected image will then be shown in the Preview box.
Though by default, the map symbols will be drawn over the background, the user can choose to Treat image as a transparent overlay. Only white pixels in the image are treated as transparent, therefore this is only suitable (for example) for a line drawing or sketch map drawn on a white background. In the preview, the transparent areas will then be shown as a pale blue.
The image can be fitted to the map in a number of ways which will be reflected in the preview. Fit to Map will fit the entire image to the selected map area and will place the image in the Centre, Top/Left or Bottom/Right of the area. Alternatively, DatMap can take the image and Stretch to Map Area - note that this will distort the image.
In some cases - e.g. where the image is a background map for a larger (or smaller) area - DatMap can Crop to Map Area. To do this, the user must define the grid area that the image represents. The min and max Eastings and Northings ranges can be defined by the user and the image will be effectively stretched to these coordinates before being cropped to the Map Area. Alternatively, one value can be left blank and it will be defaulted using the aspect ratio of the source image - i.e. the image will not be stretched. If these values are changed, the preview can be refreshed using the Preview button.
If the map area is defined, its outline is shown in green on the preview.
If the background image is no longer required, it can be removed using the Remove Background button.
TIP - For UK data on the NGR grid, outline and administrative maps for the UK are available free on the OS Web Site. If the data is held as Latitude/Longitude, the appropriate World Map required as a background is the Plate Carrée or Equirectangular projection (see Wikipedia).
Data values are mapped using a series of bands and symbols defined in the Data Bands Tab.
The Data Bands Tab first defines the Number of Bands Required - this can be from 1 to 10 bands. A single band could be used to simply plot the presence of data in each grid square or at each location. If this value is changed, band details not required will be disabled, and default key text will be adjested.
Having defined a required number of bands, the band minima can be defaulted using the Initialise Band Values from Data button. The values are then calculated using the data selected in the Map Details Tab. Where 'grid' data is aggregated, this initialisation is based on the results of the aggregation.
The first band minimum is defined as the lowest data value. If this is negative, all negative values are placed in the first band with the second band minimum being set to 0.0. Positive data values are spread across the remaining bands such that an approximately equal number of symbols will be mapped into each band. These can then be amended as required.
The rest of the tab if divided into two areas for the Bands - i.e. the band minima and key text - and Symbols - i.e. the symbols to be used in the mapping for each band. These are reflected in the settings groups that can be saved and loaded using the Settings Tab.
Minimum Values define the bands into which the data is mapped. By default, these are reflected in the Key Text which will be used in the 'key' area of the map. This shows that, in general, the minimum value is included within the band. Where a band minimum is zero, it will be excluded or included as defined on the Map Details Tab. The final band will include all values above the defined minimum. If the highest data value falls into a lower band, the key shown on the map will only include the range of bands actually used. The maximum value will be shown as the top of the last used band unless the text was marked for manual editing (see below).
The Key Text can be overridden by ticking the Manual check box - the text can then be editted as required - e.g. if the data values relate to types of data rather than numeric ranges, or if the automatic updating of the text is not required.
The user can define the shape, size and colour of the symbol used to map data falling into each band. Symbol shapes include Round, Square, Diamond, Triangle (or Delta), and Nabla (**) which can optionally be Filled or drawn as coloured outlines. If Filled, an additional black hairline can be drawn around the shape by setting the Line option. Available shapes also include Cross, Plus, Slash and Back-slash for which the Fill and Line options does not apply.
(** 'Nabla' is an inverted Delta. The name is used in vector calculus and derives from the Greek name for the shape of a Hebrew harp!)
The symbol Size is defined as a fraction of the 'grid size' set in the Map Details Tab and can be any value between 0.0 and 1.0.
Symbol Colour can be changed by clicking on the colour block - this will open the standard Windows Colour Picker. As changes are made in the symbol settings, they are reflected in the example symbols shown to the right of the row.
Though the user can define the symbol parameters individually, a related set of symbols can be defined using the Symbol Schemes Tab (below).
A related set of map symbols can be created using the Symbol Schemes Tab.
All of the symbols in a scheme will be of one Shape and all will be Filled or drawn as a coloured outline. When Filled, an optional Black Outline can also be added.
The symbols can be defined in Stepped Sizes up to 1.0 (i.e. the full grid size), or can all be Equal and the full grid size (e.g. if the symbols are square, filled, of equal size, and of different colours (see below) the map will consist of solid coloured areas representing the data bands).
The colour(s) of the symbols will be created from a Base Colour defined by the colour block on the right of the window. This can be changed by clicking in the colour block to open the Windows colour picker.
All of the symbols can be the same as the Specified Colour. Alternatively, they can be Shades or Tints of the specified colour as selected from the pull-down list - these can vary between White or Black and the specified colour (in either order as required). If the specified colour is too near to Black or White, the range will be adjusted to provide a better spread.
The final colour alternative is a set of different Colours based on specified colour. The 'hues' of the selected colours will vary through part of the colour spectrum from that of the base colour, while their saturation and lightness will be based directly on the selected colour. Again the base colour may be adjusted if it is too near to the extremes of black or white.
Clicking the Apply Symbol Scheme to Data Bands button will ask for confirmation before changing the settings in the Data Bands Tab.
Having defined the database, map details, map area, data bands, and symbols for the mappings, the final details of the page layout and graphic formatting are defined and the finished map created from the Create Map Tab
The user can select a Font that will be used for all text in the finished map, and separate Font Sizes for the Title, Sub Title, Key, Grid Numbers, and (if applicable) NGR Letters. Font sizes are defined in 'points' (1/72 inch) and inter-relate with the image resolution chosen below. A zero font size can be used to suppress the related text.
The user can also define Line Widths for the outer Frame of the map and for Grid Lines (and the inner frame).
The maximum number of Decimal Places can also be defined for data values shown in the Key and for Grid Numbers shown in the map margins.
The Position of the Key can also be defined as the Bottom of the page (by default), at the Top below the titles, or to the Left or Right of the map. The map size will be adjusted to fit in the rest of the defined image area. If the specified fonts are not sensible, the map area may become too small and a warning may be given. The pull-down list also provides the option to 'Omit' the key if required.
These elements of the map will, by default, be printed in Black, but the colour of each can be specified by clicking on the appropriate colour block.
The image Resolution and its size in Pixels are then defined. These are used in creating the map and by the Save and Copy options in the Map Window. If the map is printed from the Map Window, it will be fitted to the page as required.
The map is created by clicking the Create Map button. After checking for the required data, DatMap will warn about any inconsistencies in the data (e.g. band minima in the wrong order) and the finished map will be opened in a separate Map Window.
When a map has been created, it is opened in a separate Map Window.
On opening, the map image is sized to fit within the window. The window can be resized by dragging its margins, and the view can be zoomed using the slider bar.
When the image is zoomed, horizontal and vertical scroll bars are added to the image.
If the cursor is placed over the map, its location in Grid units will be displayed in the top left of the window. If the mapping is in Grid mode, this will show the southwest corner of the grid square. In Exact mode, it will be based more exactly on the cursor position.
Clicking the left mouse button
will add a data value to the display. For Grid mode, this will be the
aggregate used in the mapping of the square. For Exact mode, it will show the
location and value for the nearest data record (within +/- 1xGrid Size)
(see example on right). The data value will disappear when the mouse is moved.
The page can be printed, copied or saved from the File Menu.
File>Copy All to Clipboard will copy the full image to the clipboard from which it can be used in other applications.
File>Copy Map Only to Clipboard will copy the main map area omitting marginal text, titles and the key.
File>Save/Export All will open an image export dialog where the full image can be saved in various formats (e.g. JPEG, GIF, etc.). Further parameters (e.g. quality) will be requested as necessary. Note that on some machines it may take some time to bring up this dialog - it is a known issue.
File>Save/Export Map Only will save the main map area omitting marginal text, titles and the key. This enables a map created using one set of data to be used as a Background for a second map showing a second set of data. TIP - Omit grid lines from one of the maps!
File>Page Setup opens the standard Page Setup dialog allowing the user to select page orientation, margins, a printer and printer options.
File>Print (or Ctrl/P) will open a standard print dialog with default settings based on the Page Setup dialog - if Page Setup has not already been called it will be called automatically by DatMap before calling the Print dialog.
The Map Window can be closed from the menu (File>Close Map Window), or Ctrl/C, or by using the Windows 'Close' icon (top/right). It does not need to be closed if the map is to be redrawn - the new map will be shown in the existing window.
The Browse option in the Database Tab opens a separate Browse window.
The Browse window simply shows a list of the data records for the currently selected Tag in the Database tab. The data is sorted into Eastings, then Northings order, and can be scrolled using the scroll bar. The heading shows the current database file, the selected Tag and the number of data records. Data can be re-sorted by clicking on the column headings.
The Help window is a simplified HTML browser that can be called directly from the Help Menus in each DatMap window. The various Help options open the file DatMapHelp.html at an appropriate point within the text. The Go To Top button will move to the top of the text where you will find links to the different help sections.
The browser window can be resized, and text can be scrolled using the scroll bar.
The browser expects DatMapHelp.html and various other files to be in the same folder as the DatMap.exe application file.
The file DatMapHelp.html can also be opened in any other browser (e.g. Internet Explorer).