Processing NMEA0183 logfiles
In this tutorial, you will learn how to convert NMEA0183 data recorded using a datalogger carried by a drone or USV to final products, like a depth matrix, depth contours or volume calculations. `The NMEA0183 logfile should contain position data as well as depth measurements, otherwise it could not be used to generate soundings.
$GNGGA,075359.718,5653.5709,N,02419.3519,E,1,15,71.0,0,M,0,M,,*4D $YXXDR,A,-0.9,D,PTCH,A,-0.3,D,ROLL*57 $SDDBT,8.045,f,2.452,M,,*66 $GNGGA,075400.249,5653.5709,N,02419.3519,E,1,16,66.0,0,M,0,M,,*42 $YXXDR,A,-1.1,D,PTCH,A,-0.0,D,ROLL*5D $SDDBT,7.429,f,2.264,M,,*64 $GNGGA,075400.809,5653.5709,N,02419.3519,E,1,16,65.0,0,M,0,M,,*4F $YXXDR,A,-0.9,D,PTCH,A,1.0,D,ROLL*78 $SDDBT,7.364,f,2.245,M,,*69Example of a valid NMEA0183 logile recorded using a drone
The NMEA0183 logfile should contain at least the 'GGA' sentence containing position information and GPS quality and a depth sentence, for instance 'DBT', 'DPT', 'DBS' or 'DBK'. Optional sentences include 'XDR', which can contain dual frequency depth information or motion sensor data, and sentences containing speed or heading information like 'VTG'.
In this tutorial we will cover the following steps to convert NMEA0183 logfiles into final products:
- Create a new project and select the coordinate system;
- Importing the NMEA0183 data into the project;
- Configuring a color schema;
- Downloading a background map;
- Creating a boundary for data clipping or shoreline purposes;
- Cleansing of the imported data and sounding generation;
- Generating a matrix;
- Check the data using the terrain viewer;
- Exporting data.
1. Create a new project and select the coordinate system
To create a new project, make sure the current project (if any) is saved, and select the "New Project..." option from the "File" menu. The following dialog should appear:
Start by creating a new hydromagic project from scratch
Enter a descriptive name for the project which also will be used as folder name for the project. At this point it is important to know which coordinate system (or map projection) will be used, since we have to select it now. Please note that this cannot be changed later (you actually can, but it is not easy to do!). When unsure ask your client in which coordinate system he wishes to receive the final products. The example NMEA0183 files we will use in this tutorial are provides by SPH Engineering and were recorded on a lake near Riga (Latvia). In this example we will use the "LKS92 / Latvia TM" coordinate system. Click the "Select..." button to select the coordinate system of your choice.
Select a suitable coordinate system for your new project
You can now enter the name of the surveyor or a description of the project. Please note that these fields are not required and can be changed later. Finally, click the "Create" button to generate the new project.
Click here to read more detailed information on Hydromagic projects in the online Hydromagic manual.
2. Importing the NMEA0183 data into the project
An empty project is now loaded and we can now import the data so we know which area is covered by the data, making it easier to download a background map for this project. First make sure the "Project Explorer" is displayed at the left side of the screen. If the "Project Explorer" isn't visible, show it by selecting the "Project Explorer" option from the "View" menu.
In the "Project Explorer", locate and right-click the "Raw Data" to open the popup menu, and select the "Import Data Wizard..." option. This wizard allows you to import data as raw data files from different file formats. Since this tutorial is about importing NMEA0183 data, select the "NMEA0183 ASCII Log Files (*.*)" option on the first page of the wizard and click "Next" to select the source folder.
Select the "NMEA0183 ASCII Log Files (*.*)" option from the drop-down list.
Put all NMEA0183 logfiles (you can import an unlimited number of logfiles into the same project) in the same folder and select the folder by clicking the "Browse..." button. After selecting the folder which contains the logfiles, the files will be listed, and the files you wish to import can be selecting by using the check boxes. When done, click "Next".
Select the "Select the files to import by clicking check boxes.
The "Advanced Settings" page can be used to set the sounding date. Since only the UTC time is encoded in NMEA0183 position sentences, you can use this field to assign a date to the raw data records. This is optional, but highly recommended, and even required in cases where you do want to apply a tide file. It is also required to select the NMEA0183 sentence used to report the echo sounder depth. You can find the correct sentence in the logfile. In the example which is displayed in the introduction, 'DBT' is used.
Another important settings is the "Sounder Offset", also called static draft. This is the distance between the waterline and the bottom of the echo sounder transducer. This value must be entered as a positive value. When unsure, contact the supplier of your USV or drone for more information. When this value is already applied to the NMEA0183 data generated by the datalogger, enter '0.000'.
Select the "Select the files to import by clicking check boxes.
Now click "Next" and the final page contains some information on the generation process. Click "Finish" to start the import and the import raw data files will be displayed shortly.
The imported data displayed as raw data files in the map view.
3. Configuring a color schema
This step is optional, however it is recommended to configure color ranges for the depths, so you can easily detect outliers. To configure colors, locate the "Depth Colors" button in the tool bar, or select the "Depth Colors..." item from the "Options" menu. When there are no color sets defined in the "Manage Color Sets" dialog, click the "..." button to add a new set. Click the "Add..." button and enter a descriptive name for the new color set, for instance "Depths".
Click the "Add..." button to add a new color set to your Hydromagic project.
After naming the new color set, click "Close" and select the newly created color set using the drop down box at the top of the "Manage Color Sets" dialog. You can now configure color ranges by hand, or let the software create them automatically, which is the best option for most cases. To do so, click the "Auto..." button and select the minimum and maximum depths in the surveyed area. For this example we will use the range from zero to ten meters with a one meter interval. Please note that a color set can only contain 30 colors.
Use the "Generate Color Ranges" dialog to generate a color set automatically.
After generating a color set automatically, you can always modify some of the ranges by hand by double-clicking the range you want to alter. When you are happy with the color set, click the "OK" button to store the color set.
The imported data displayed as raw data files in the map view (with the new color set applied).
4. Downloading a background map
In order to check the correct location of the sounding, or to use as guidance to draw boundaries, it is highly recommended to download a background map for your project (in cases where you do not have a CAD drawing of the area). To download a map, please make sure the "Project Explorer" is visible, and right click the "Maps" folder. From the pop up menu which appears, select the "Download Map..." option.
Before downloading and storing the map, you have to select the boundaries of the map. Since we already improted the raw data, Hydromagic can detect the boundaries of the map automatically. In the "Download Background Map" tool, click the "Calculate..." button to select the method used to calculate the boundaries:
Click the "Calculate..." button to calculate the boundaries for the map to be downloaded.
Select the "Calculate map boundaries from raw data file in the "Select Download Area" dialog, and select one of the RAW data files which was imported in step #2 of this tutorial. Please make sure you do not select a RAW data file with none or little position information. It is recommended to select a large raw data file which is located near the center of the survey area.
Select a RAW data file to detect the boundaries of the background map.
Use a boundary to define the survey area. This boundary is used when generating matrices.
5. Creating a boundary
It is recommended to draw a boundary around the area to be surveyed. A boundary can be used to clip matrix data, filter position data or to generate cross sections. Since boundary creation is already discussed in detail in the manual, please click the link below to read more on this subject, or you can watch the YouTube video:
Use a boundary to define the survey area. This boundary is used when generating matrices.
6. Sounding generation
In Hydromagic, the "Sounding Wizard" is used to convert raw data into filtered, time synchronized and tide corrected soundings which can be used to generated matrices, depth contours, volume reports and much more. To start the "Sounding Wizard", make sure the "Project Explorer" is visible and right click the "Raw Data" folder to select the "Process Raw Data Files..." option.
The sounding wizard will guide you in a couple of steps through data selection, tide correction, draft and sound velocity correction, data filtering and much more. Please visit the link below to view all it's details in the online Hydromagic user manual:
Click here to read more on using the Sounding Wizard in the online Hydromagic manual.
7. Generating a matrix
After generating the sounding(s) for the imported raw data file(s), the next step is to generate a matrix for the survey area. A matrix consists out of cell's in X- and Y direction showing depth values as text or coded as the color ranges defined in step #3 of this tutorial. Using a matrix you can see if the data has been filtered and processed correctly in the blink of an eye.
To generate a matrix, make sure the "Project Explorer" is visible and right click the "Matrix" folder to select the "Generate Matrix..." option from the menu. In the "Generate Matrix" dialog, please select the generated soundings, the boundary and a cell spacing of your choice and click "OK".
Click here to read more on generating matrices in the online Hydromagic manual.
8. Check the data using the terrain viewer
Click here to read more on using the 3D Terrain Viewer in the online Hydromagic manual.
9. Exporting Data