Note
Go to the end to download the full example code.
Magnetic Raster Dataset
These magnetic data channels were pulled from the Wisconsin SkyTEM example in this repository to demonstrate the relatively simple case of gridded raster files.
Source Reference: Minsley, B.J, Bloss, B.R., Hart, D.J., Fitzpatrick, W., Muldoon, M.A., Stewart, E.K., Hunt, R.J., James, S.R., Foks, N.L., and Komiskey, M.J., 2022, Airborne electromagnetic and magnetic survey data, northeast Wisconsin (ver. 1.1, June 2022): U.S. Geological Survey data release, https://doi.org/10.5066/P93SY9LI.
import matplotlib.pyplot as plt
from os.path import join
import gspy
from gspy import Survey
import warnings
warnings.filterwarnings('ignore')
Initialize the Survey
# Path to example files
data_path = '..//data_files//magnetics'
# Survey metadata file
metadata = join(data_path, "WI_Magnetics_survey_md.yml")
# Establish the Survey
survey = Survey.from_dict(metadata)
Survey YAML file
1dataset_attrs:
2 title: Magnetic data from SkyTEM Airborne Electromagnetic (AEM) Survey, Northeast Wisconsin Bedrock Mapping
3 institution: USGS Geology, Geophysics, and Geochemistry Science Center
4 source: SkyTEM raw data, USGS processed data
5 history: (1) Data acquisition 01/2021 - 02/2021 by SkyTEM Canada Inc.; (2) AEM and magnetic data processing by SkyTEM Canada Inc. 02/2021 - 03/2021; raw and minimally processed AEM data, and processed magnetic data, received by USGS from SkyTEM Canada Inc 03/2021; Minimally processed AEM data exported to netCDF /tabular/0 group 11/2021; (3) Minimally processed binary data and system response information received from the contractor were imported into the Aarhus Workbench software (v 6.0.1.0) where data were processed by USGS 03/2021 - 06/2021. Processed AEM data exported to netCDF /tabular/1 group 11/2021; (4) Processed data were inverted in Aarhus Workbench software using laterally constrained inversion to recover 40-layer fixed depth blocky resistivity models by USGS 03/2021 - 06/2021; Inverted resistivity models exported to netCDF /tabular/2 group 11/2021. (5) Resistivity models were imported into the Geoscene3D software (v. 12.0.0.680) and points were generated at the first depth where resistivity exceeded 325 ohm-meters. These points were visually inspected and manually adjusted in selected areas to produce an AEM-derived estmiate of the elevation of the top of bedrock by USGS together with WGNHS 06/2021 - 07/2021. Points were exported to netCDF /tabular/3 group 11/2021. (6) Bedrock elevation points were interpolated using kriging in Geoscene3D software to produce a regular bedrock elevation grid 07/2021. (7) A bedrock depth grid was calculated in QGIS software (v. 3.14.1-Pi) by subtracting the bedrock elevation from land surface elevation. (8) Bedrock elevation, bedrock depth, and SkyTEM-provided magnetic grids were aligned to a common 100m x 100m grid and exported to netCDF /raster/0 group 11/2021.
6 references: Minsley, Burke J., B.R. Bloss, D.J. Hart, W. Fitzpatrick, M.A. Muldoon, E.K. Stewart, R.J. Hunt, S.R. James, N.L. Foks, and M.J. Komiskey, 2021, Airborne electromagnetic and magnetic survey data, northeast Wisconsin, 2021, U.S. Geological Survey data release, https://doi.org/10.5066/P93SY9LI.
7 comment: This dataset includes minimally processed (raw) AEM and raw/processed magnetic data provided by SkyTEM
8 summary: Magnetic survey data were collected during January and February 2021 over a distance of 3,170 line kilometers in northeast Wisconsin. These data were collected in support of an effort to improve estimates of depth to bedrock through a collaborative project between the U.S. Geological Survey (USGS), Wisconsin Department of Agriculture, Trade, and Consumer Protection (DATCP), and Wisconsin Geological and Natural History Survey (WGNHS). Data were acquired by SkyTEM Canada Inc. with the SkyTEM 304M time-domain helicopter-borne electromagnetic system together with a Geometrics G822A cesium vapor magnetometer. The survey was acquired at a nominal flight height of 30 - 40 m above terrain along parallel flight lines oriented northwest-southeast with nominal line spacing of 0.5 miles (800 m). AEM data were inverted to produce models of electrical resistivity along flight paths, with typical depth of investigation up to about 300 m and 1 - 2 m near-surface resolution. Shallow resistivity transitions were used to estimate depth to bedrock across the survey area.
9 content: Wisconsin SkyTEM survey information
10
11survey_information:
12 contractor_project_number: 20022
13 contractor: SkyTEM Canada Inc
14 client: U.S. Geological Survey
15 survey_type: EM/Mag
16 survey_area_name: Northeast Wisconsin Bedrock Mapping
17 state: WI
18 country: USA
19 acquisition_start: 20210117
20 acquisition_end: 20210207
21 survey_attributes_units: SI
22
23spatial_ref:
24 wkid: 3071
25 authority: EPSG
26 vertical_crs: NAVD88
27
28flightline_information:
29 traverse_line_spacing: 800 m
30 traverse_line_direction: nw-se
31 tie_line_spacing: n/a
32 tie_line_direction: n/a
33 nominal_terrain_clearance: 30 m
34 final_line_kilometers: 3170 km
35 traverse_line_numbers: 100101 - 115201
36 repeat_line_numbers: 920001 - 920006
37 pre_zero_line_numbers: n/a
38 post_zero_line_numbers: n/a
39
40survey_equipment:
41 aircraft: Eurocopter Astar 350 B3
42 magnetometer: Geometrics G822A, Kroum KMAG4 counter
43 magnetometer_installation: Front of transmitter frame
44 radar_altimeter_system: n/a
45 radar_altimeter_sample_rate: n/a
46 laser_altimeter_system: MDL ILM 300R (2)
47 laser_altimeter_sample_rate: 0.033 s
48 navigation_system: Real-time differential GPS Trimble Bullet III
49 navigation_sample_rate: 1.0 s
50 acquisition_system: skytem
Create a Data Branch
data_container = survey.gs.add_container('magnetic_data', **dict(content = "raw flightline and gridded magnetic data", comment = "grids were contractor-derived"))
Add the raw magnetic data to the branch
# Import raw magnetic data from CSV-format.
d_data1 = join(data_path, 'WI_Magnetics.csv')
d_supp1 = join(data_path, 'WI_Magnetics_raw_data_md.yml')
rd = data_container.gs.add(key='raw_data', data_filename=d_data1, metadata_file=d_supp1)
Raw Magnetic Data YAML file
1dataset_attrs:
2 content: raw data
3 comment: Contains flightline magnetic data
4 type: data
5 structure: tabular
6 mode: airborne
7 method: magnetic, total magnetic field
8 instrument: cesium vapour
9
10magnetic_system:
11 type: system
12 mode: airborne
13 method: magnetic
14 instrument: Geometrics G-822A cesium‑vapor magnetometer
15
16 prefixes: ['base_magnetometer']
17
18 dimensions:
19 base_mag_locations:
20 standard_name: base_mag_locations
21 long_name: Base Magnetometer Location Index Numbers
22 units: not_defined
23 missing_value: not_defined
24 centers: [1, 2]
25 discrete: True
26
27 variables:
28
29 transmitter:
30 label: passive
31 description: No artificial magnetic transmitter was used. The system measures the scalar Larmor precession frequency induced by the Earth's magnetic field.
32
33 receiver:
34 label: scalar_magnetometer
35 sensor_type: cesium-vapor split-beam
36 sensor_model: G-822A
37 sensor_manufacturer: Geometrics
38 description: Scalar cesium-vapor magnetometer mounted in the aircraft tail stinger. Measures total magnetic field through Larmor precession frequency.
39 orientation: Tail-stinger mounted; scalar measurement independent of orientation.
40 coordinates: not_reported
41 lag_correction: Lag was negligible and no lag correction was applied
42 heading_correction: Heading was negligible and no heading correction was applied
43
44 couplet:
45 passive_scalar_pair:
46 transmitters: [passive]
47 receivers: [scalar_magnetometer]
48 description: The magnetic measurement system consists of the Earth's field as a passive transmitter and a single scalar magnetometer mounted in the tail stinger.
49
50 base_magnetometer:
51 label: base_magnetometer
52 description: The base station magnetometer was placed in a location of low magnetic gradient, away from electrical transmission lines and moving metallic objects, such as motor vehicles and aircrafts.
53
54 location_names:
55 values: ["Door County", "Manitowoc"]
56 dimensions: 'base_mag_locations'
57
58 values:
59 values: [54538, 54194.7]
60 units: nT
61 dimensions: 'base_mag_locations'
62
63 latitude:
64 values: [44.849335, 44.127998]
65 long_name: Latitude in WGS84
66 units: decial degrees
67 dimensions: 'base_mag_locations'
68
69 longitude:
70 values: [87.422440, 87.685524]
71 long_name: Longitude in WGS84
72 units: decial degrees
73 dimensions: 'base_mag_locations'
74
75 elevation:
76 values: [178.1, 164.4]
77 long_name: Elevation
78 datum: WGS84
79 units: m
80 dimensions: 'base_mag_locations'
81
82 diurnal_correction: Diurnal signal removed using 3 second Fraser low-pass filter and subtracting base-station magnetometer values.
83 tieline_levelling: No tie line-leveling were applied
84 microlevelling: No micro-levelling were applied
85 igrf_model_date: "2015, 15th generation"
86 igrf_model_location: variable according to GPS WGS84 longitude and latitude
87 igrf_model_height: variable according to magnetic sensor altitude derived from DGPS data
88
89coordinates:
90 x: E_Nad83
91 y: N_Nad83
92 z: DEM
93
94variables:
95 E_Nad83:
96 standard_name: easting_nad83
97 long_name: Easting, Wisconsin Transverse Mercator (WTM), North American Datum of 1983 (NAD83)
98 units: meter
99 missing_value: not_defined
100 axis: x
101
102 N_Nad83:
103 standard_name: northing_nad83
104 long_name: Northing, Wisconsin Transverse Mercator (WTM), North American Datum of 1983 (NAD83)
105 units: meter
106 missing_value: not_defined
107 axis: y
108
109 Alt:
110 standard_name: altitude
111 long_name: DGPS instrument altitude
112 units: meter
113 missing_value: not_defined
114
115 Base_Mag:
116 standard_name: base_magnetometer
117 long_name: Raw magnetic base station data
118 units: nanoTesla
119 missing_value: -9999.99
120
121 Date:
122 standard_name: date
123 long_name: Date, yyyy/mm/dd
124 units: not_defined
125 missing_value: not_defined
126
127 dec:
128 standard_name: decliniation
129 long_name: IGRF declination
130 units: degree
131 missing_value: not_defined
132
133 DEM:
134 standard_name: dem
135 long_name: Digital elevation model
136 units: meter
137 missing_value: not_defined
138 axis: Z
139 positive: up
140 datum: North American Vertical Datum of 1988 (NAVD88)
141
142 Diurnal:
143 standard_name: magnetic_diurnal
144 long_name: Magnetic base station diurnal variation
145 units: nanoTesla
146 missing_value: not_defined
147
148 Fid:
149 standard_name: fiducial
150 long_name: Unique fiducial number
151 units: not_defined
152 missing_value: not_defined
153
154 Height:
155 standard_name: instrument_height
156 long_name: Instrument height above ground
157 units: meter
158 missing_value: not_defined
159
160 igrf:
161 standard_name: magnetic_igrf_model
162 long_name: Magnetic IGRF value from 2015 model
163 units: nanoTesla
164 missing_value: not_defined
165
166 inc:
167 standard_name: inclination
168 long_name: IGRF inclination
169 units: degree
170 missing_value: not_defined
171
172 Lat:
173 standard_name: latitude
174 long_name: Latitude, World Geodetic System of 1984 (WGS84)
175 units: degree
176 missing_value: not_defined
177
178 Line:
179 standard_name: line
180 long_name: Line number
181 units: not_defined
182 missing_value: not_defined
183
184 Lon:
185 standard_name: longitude
186 long_name: Longitude, World Geodetic System of 1984 (WGS84)
187 units: degree
188 missing_value: not_defined
189
190 Mag_Filt:
191 standard_name: total_magnetic_intensity_filtered
192 long_name: Filtered total magnetic intensity data
193 units: nanoTesla
194 missing_value: not_defined
195
196 Mag_Raw:
197 standard_name: total_magnetic_intensity_raw
198 long_name: Raw total magnetic intensity data
199 units: nanoTesla
200 missing_value: not_defined
201
202 RMF:
203 standard_name: residual_magnetic_field
204 long_name: Residual magnetic field, IGRF corrected based on 2015 model
205 units: nanoTesla
206 missing_value: not_defined
207
208 Time:
209 standard_name: time_utc
210 long_name: Universal coordinated time (UTC), hh:mm:ss
211 units: not_defined
212 missing_value: not_defined
213
214 TMI:
215 standard_name: total_magnetic_intensity
216 long_name: Diurnally corrected filtered total magnetic intensity
217 units: nanoTesla
218 missing_value: not_defined
Add the Gridded Data
Import a tif of gridded mag data. The metadata file for raster datasets should contain paths to the raster files.
d_supp1 = join(data_path, 'WI_Magnetics_grids_md.yml')
gd = data_container.gs.add(key='grids', metadata_file=d_supp1)
Gridded (Raster) Magnetic YAML file
1dataset_attrs :
2 content : gridded magnetic maps
3 comment: contractor-derived maps
4 type: data
5 structure: raster
6 mode: airborne
7 method: magnetic, total magnetic field
8 instrument: cesium vapour
9 property: magnetic
10
11coordinates:
12 x: E_Nad83
13 y: N_Nad83
14
15dimensions:
16 x: E_Nad83
17 y: N_Nad83
18
19variables:
20 magnetic_tmi:
21 standard_name: total_magnetic_intensity
22 long_name: Total magnetic intensity, diurnally corrected and filtered
23 units: nanoTesla
24 missing_value: -9999.99
25 files : Midwest_Core_MAG_TMI_NAD83.tif
26 dimensions: [x, y]
27
28 magnetic_rmf:
29 standard_name: residual_magnetic_field
30 long_name: Residual magnetic field, IGRF corrected from 2015 model
31 units: nanoTesla
32 missing_value: -9999.99
33 files : Midwest_Core_MAG_RMF_NAD83.tif
34 dimensions: [x, y]
35
36 E_Nad83:
37 standard_name: easting_nad83
38 long_name: Easting, Wisconsin Transverse Mercator (WTM), North American Datum of 1983 (NAD83)
39 units: meter
40 missing_value: not_defined
41 axis : x
42
43 N_Nad83:
44 standard_name: northing_nad83
45 long_name: Northing, Wisconsin Transverse Mercator (WTM), North American Datum of 1983 (NAD83)
46 units: meter
47 missing_value: not_defined
48 axis : y
Save to NetCDF file
d_out = join(data_path, 'magnetics.nc')
survey.gs.to_netcdf(d_out)
Opening a GS NetCDF
new_survey = gspy.open_datatree(d_out)['survey']
View the Data Tree
print(new_survey)
<xarray.DataTree 'survey'>
Group: /survey
│ Dimensions: ()
│ Coordinates:
│ spatial_ref float64 8B ...
│ Data variables:
│ survey_information float64 8B ...
│ flightline_information float64 8B ...
│ survey_equipment float64 8B ...
│ Attributes:
│ type: survey
│ title: Magnetic data from SkyTEM Airborne Electromagnetic (AEM) S...
│ institution: USGS Geology, Geophysics, and Geochemistry Science Center
│ source: SkyTEM raw data, USGS processed data
│ history: (1) Data acquisition 01/2021 - 02/2021 by SkyTEM Canada In...
│ references: Minsley, Burke J., B.R. Bloss, D.J. Hart, W. Fitzpatrick, ...
│ comment: This dataset includes minimally processed (raw) AEM and ra...
│ summary: Magnetic survey data were collected during January and Feb...
│ content: Wisconsin SkyTEM survey information /survey; raw flightlin...
│ gspy_version: 2.2.4
│ conventions: GS-2.0, CF-1.13
└── Group: /survey/magnetic_data
│ Dimensions: ()
│ Data variables:
│ spatial_ref float64 8B ...
│ Attributes:
│ content: raw flightline and gridded magnetic data
│ comment: grids were contractor-derived
│ type: container
├── Group: /survey/magnetic_data/raw_data
│ │ Dimensions: (index: 6785)
│ │ Coordinates:
│ │ * index (index) float64 54kB 0.0 1.0 2.0 ... 6.783e+03 6.784e+03
│ │ spatial_ref float64 8B ...
│ │ x (index) float64 54kB ...
│ │ y (index) float64 54kB ...
│ │ z (index) float64 54kB ...
│ │ Data variables: (12/17)
│ │ fid (index) float64 54kB ...
│ │ line (index) float64 54kB ...
│ │ lon (index) float64 54kB ...
│ │ lat (index) float64 54kB ...
│ │ alt (index) float64 54kB ...
│ │ height (index) float64 54kB ...
│ │ ... ...
│ │ mag_raw (index) float64 54kB ...
│ │ tmi (index) float64 54kB ...
│ │ rmf (index) float64 54kB ...
│ │ igrf (index) float64 54kB ...
│ │ inc (index) float64 54kB ...
│ │ dec (index) float64 54kB ...
│ │ Attributes:
│ │ content: raw data
│ │ comment: Contains flightline magnetic data
│ │ type: data
│ │ structure: tabular
│ │ mode: airborne
│ │ method: magnetic, total magnetic field
│ │ instrument: cesium vapour
│ └── Group: /survey/magnetic_data/raw_data/magnetic_system
│ Dimensions: (n_transmitter: 1, n_receiver: 1,
│ n_couplet: 1, n_base_magnetometer: 1,
│ base_mag_locations: 2)
│ Coordinates:
│ * n_transmitter (n_transmitter) float64 8B 0.0
│ * n_receiver (n_receiver) float64 8B 0.0
│ * n_couplet (n_couplet) float64 8B 0.0
│ * n_base_magnetometer (n_base_magnetometer) float64 8B 0.0
│ * base_mag_locations (base_mag_locations) float64 16B 1.0 2.0
│ Data variables: (12/29)
│ transmitter_label (n_transmitter) <U7 28B ...
│ transmitter_description (n_transmitter) <U142 568B ...
│ receiver_label (n_receiver) <U19 76B ...
│ receiver_sensor_type (n_receiver) <U23 92B ...
│ receiver_sensor_model (n_receiver) <U6 24B ...
│ receiver_sensor_manufacturer (n_receiver) <U10 40B ...
│ ... ...
│ diurnal_correction <U110 440B ...
│ tieline_levelling <U33 132B ...
│ microlevelling <U31 124B ...
│ igrf_model_date <U21 84B ...
│ igrf_model_location <U54 216B ...
│ igrf_model_height <U69 276B ...
│ Attributes:
│ type: system
│ mode: airborne
│ method: magnetic
│ instrument: Geometrics G-822A cesium‑vapor magnetometer
│ name: magnetic_system
└── Group: /survey/magnetic_data/grids
Dimensions: (x: 799, nv: 2, y: 1155)
Coordinates:
* x (x) float64 6kB 6.551e+05 6.552e+05 ... 7.348e+05 7.349e+05
* nv (nv) float64 16B 0.0 1.0
* y (y) float64 9kB 4.953e+05 4.952e+05 ... 3.8e+05 3.799e+05
spatial_ref float64 8B ...
Data variables:
x_bnds (x, nv) float64 13kB ...
y_bnds (y, nv) float64 18kB ...
magnetic_tmi (y, x) float64 7MB ...
magnetic_rmf (y, x) float64 7MB ...
Attributes:
content: gridded magnetic maps
comment: contractor-derived maps
type: data
structure: raster
mode: airborne
method: magnetic, total magnetic field
instrument: cesium vapour
property: magnetic
Plotting
plt.figure()
new_survey['magnetic_data/raw_data']['height'].plot()
plt.tight_layout()
pd = new_survey['magnetic_data/raw_data']['tmi']
plt.figure()
pd.plot()
plt.tight_layout()
m = new_survey['magnetic_data/grids/magnetic_tmi']
plt.figure()
m.plot(cmap='jet')
plt.tight_layout()
plt.show()
Total running time of the script: (0 minutes 0.531 seconds)