LiDAR Peak Analysis: What It Takes

Items that do not fit the categories above.
Forum rules
  • This is a mountaineering forum, so please keep your posts on-topic. Posts do not all have to be related to the 14ers but should at least be mountaineering-related.
  • Personal attacks and confrontational behavior will result in removal from the forum at the discretion of the administrators.
  • Do not use this forum to advertise, sell photos or other products or promote a commercial website.
  • Posts will be removed at the discretion of the site administrator or moderator(s), including: Troll posts, posts pushing political views or religious beliefs, and posts with the purpose of instigating conflict within the forum.
For more details, please see the Terms of Use you agreed to when joining the forum.
User avatar
Eli Boardman
Posts: 660
Joined: 6/23/2016
14ers: 58  1  15 
13ers: 18 1
Trip Reports (16)
 
Contact:

Re: LiDAR Peak Analysis: What It Takes

Post by Eli Boardman »

RadioJay wrote: Sun Jan 02, 2022 8:31 am
seano wrote: Sun Jan 02, 2022 6:44 am Thanks for the tutorial! QGIS is pretty overwhelming and clumsy at first, as one might expect from an expert tool. Unfortunately LiDAR data does not seem to be available for the Sierra yet. When it is, it's very possible Mount Barnard (between Whitney and Williamson, currently 13,990') will become another 14er.
Can Eli or someone address the accuracy of the aircraft’s elevation? I imagine it uses a combination of GPS and INS but super accurate elevation estimates usually require a long observation time and the airplane is moving and is subject to turbulence.
Ah yes, good question. As I understand it, based on working with a few people who do their own LiDAR flights, the airplane has 2 positioning components: a GPS, and an IMU (inertial measurement unit). The IMU is a super accurate system of multiple gyroscopes that is fixed to the same solid plate as the observation equipment and constantly monitors the transverse and rotational acceleration of the instrument. This isn't just used for XYZ positioning--the IMU data are also critical for figuring out the look-direction of the scanner at any given moment so that shots can be accurately located on the ground. (For instance, what happens when the plane is slightly banking? The shots are skewed more to one side.) The general position of the aircraft is updated based on onboard GPS, BUT there's a trick: they simultaneously use a base station GPS unit positioned at well-known coordinates (say, a benchmark) to track the actual effect of the atmosphere and other noise on the GPS signal at a certain time in a certain general area. This is called Differential GPS, and combining the two GPS records with the IMU measurements yields extremely accurate positioning.

Read more:
IMU and GPS positioning for remote sensing
Differential GPS

By the way, here's what a LiDAR scanner looks like from inside and outside the airplane.
Riegel-LiDAR.jpg
Riegel-LiDAR.jpg (136.24 KiB) Viewed 3013 times
Updated: looks like Ben was answering the same way while I was typing haha!
User avatar
Eli Boardman
Posts: 660
Joined: 6/23/2016
14ers: 58  1  15 
13ers: 18 1
Trip Reports (16)
 
Contact:

Re: LiDAR Peak Analysis: What It Takes

Post by Eli Boardman »

glenmiz wrote: Sat Jan 01, 2022 9:05 pm Thanks for all of the information. This is pretty interesting and I'm looking at Jeffco soft-ranked peaks. I came pretty close to your values for Ralston Buttes' elevation and prominence but I'm not sure how to convert from UTM to lat/long coordinates.
Brief lat/lon tutorial for QGIS:

Note this is not the only way to do it, even within QGIS, but it's convenient when you already have a long spreadsheet of XY coordinates. Also, this doesn't take as long as it looks, but I'm still erring on the side of being overly detailed.

Step 1: identify the coordinate reference system (CRS) of your original data. THIS IS THE MOST IMPORTANT STEP. Your data are not necessarily in the same CRS as someone else's data from a different time, agency, or part of the country (CRS is, however, constant within a given survey). A given CRS is identified uniquely by a number called its EPSG code. In QGIS, right-click on one of your pointcloud layers and scroll over "Layer CRS" in the menu that pops up. This should show a secondary menu with the name of the CRS at the top. In my case, it says "EPSG:6350 - NAD83(2011) / ConusAlbers," but yours might be different. In any case, write down the EPSG code (mine is 6350) for future reference.

Step 2: get all your coordinates into a single .csv file. You can just copy-past columns of XY coordinates into a new Excel sheet and save it as type "CSV UTF-8 (Comma Delimited) .csv" (title each column "X" and "Y" since QGIS will default to using the first line as a header).

Step 3: import coordinates into QGIS and save as shapefile. In the top toolbar in QGIS, click "Layer-->Data Source Manager," click the comma-shaped item on the left for "Delimited Text," then browse for the file you created in Step 2. Expand the "Geometry Definition" submenu if it isn't already expanded, then set the Geometry CRS to the same CRS you identified in Step 1. (It's likely the same as your Project CRS, but check the EPSG code to be sure--if need be, you can click the map-globe icon on the left and search for the numeric code from Step 1.) Click "Add" to add the data. Right-click the new layer and select "Export-->Save Features As," then choose "Format-->ESRI Shapefile" and an appropriate filename. The new shapefile should automatically load into the map.

Step 4: transform the coordinates. In the top toolbar in QGIS, click "Processing-->Toolbox," then search for "Reproject Layer" in the search bar at the top of the Processing Toolbox. Double-click the name of this tool, then make sure that the Input Layer is the shapefile you created in Step 3, and set the Target CRS to "EPSG:4326 - WGS 84." You can leave the default option to save as a temporary file, run the tool, and the "Reprojected" shapefile should appear in your Layers.

Step 5: export the coordinates. Now, this is a little annoying, but to re-export it as a spreadsheet, you have to right-click the new layer and click "Open Attribute Table." In this window, click the icon that looks like an abbacus (or press Ctrl-i) to open the Field Calculator. Set the Output Field Name to "longitude," change the Output Field Type to "Decimal number (real)," increase the precision to at least 7, and type "$x" into the Expression box. Then you can click OK, and repeat the process for "latitude" and "$y." Finally, click the save icon then left-most pencil icon in the Attribute Table to save and exit edit mode (or press Ctrl-s then Ctrl-e) and close the Attribute Table. Finally, right-click the Reprojected layer and choose "Export-->Save Features As." Select the "Comma Separated Value [CSV]" option for Format, give it a filename, and click OK. Congrats, you've survived one of QGIS's rather clumsy procedures! You can open the new .csv in Excel, and the lat/lon coordinates should be in the same rows next to the original XY coordinates.

Cheat code: if you're only converting a handful of points, you can just use THIS online automatic tool with the appropriate EPSG codes.
User avatar
RyanSchilling
Posts: 171
Joined: 1/18/2005
14ers: 58 
13ers: 249
Trip Reports (1)
 

Re: LiDAR Peak Analysis: What It Takes

Post by RyanSchilling »

Eli, really appreciate the tutorial you put together here, thank you!
User avatar
bdloftin77
Posts: 1090
Joined: 9/23/2013
14ers: 58  1 
13ers: 58
Trip Reports (2)
 

Re: LiDAR Peak Analysis: What It Takes

Post by bdloftin77 »

Eli Boardman wrote: Sun Jan 02, 2022 9:35 am Updated: looks like Ben was answering the same way while I was typing haha!
No worries, you had a much more detailed answer! Thanks for all your lidar contributions so far.

Do you have a GIS background? I really like your lidar-derived 7.5' WY quad!
User avatar
Salient
Posts: 178
Joined: 2/19/2021
Trip Reports (0)
 
Contact:

Re: LiDAR Peak Analysis: What It Takes

Post by Salient »

Good to see What It Takes threads coming back :)
Be the best you that you can be.
User avatar
Eli Boardman
Posts: 660
Joined: 6/23/2016
14ers: 58  1  15 
13ers: 18 1
Trip Reports (16)
 
Contact:

Re: LiDAR Peak Analysis: What It Takes

Post by Eli Boardman »

bdloftin77 wrote: Tue Jan 04, 2022 1:18 pm
Eli Boardman wrote: Sun Jan 02, 2022 9:35 am Updated: looks like Ben was answering the same way while I was typing haha!
No worries, you had a much more detailed answer! Thanks for all your lidar contributions so far.

Do you have a GIS background? I really like your lidar-derived 7.5' WY quad!
Thanks! It depends on what we mean by background. I never had formal GIS training, but I use QGIS a lot for my hydrology research (I'm a grad student), and I've made a lot of simpler maps with it. Catchment delineation and channel mapping is my favorite since it makes such cool-looking images. The LiDAR part is new to me though, and the almost-total lack of QGIS documentation for the new native pointcloud utility is frustrating. For anyone looking to make DEMs from LiDAR for free, try this:

http://forsys.cfr.washington.edu/fusion ... atest.html
https://gis.stackexchange.com/questions ... g-lastools
klopp
Posts: 3
Joined: 7/4/2010
14ers: 58 
13ers: 58
Trip Reports (0)
 

Re: LiDAR Peak Analysis: What It Takes

Post by klopp »

Thank you Eli for the helpful tutorial! I wish this was available a few months ago when I started looking at Lidar data in QGIS.
Another useful and fun tool I have used in QGIS is the 3D Map function. It can be used to identify odd points, locate summits/saddles, and generate cool images. The pic below is a 3D visualization of the Maroon and N Maroon Lidar data that I generated while taking a look at the prominence of N Maroon. It doesn't quite look like a picture, but it is at least a reminder that you are looking at data from mountains.

Image

Once you have imported and formatted the Lidar data as you described in your tutorial, 3D Maps are easy to generate from the toolbar (View>New 3D Map View), though you may need to zoom out to find the point cloud.

Keep up the good work!
User avatar
bdloftin77
Posts: 1090
Joined: 9/23/2013
14ers: 58  1 
13ers: 58
Trip Reports (2)
 

Re: LiDAR Peak Analysis: What It Takes

Post by bdloftin77 »

klopp wrote: Thu Jan 06, 2022 1:51 pm Thank you Eli for the helpful tutorial! I wish this was available a few months ago when I started looking at Lidar data in QGIS.
Another useful and fun tool I have used in QGIS is the 3D Map function. It can be used to identify odd points, locate summits/saddles, and generate cool images. The pic below is a 3D visualization of the Maroon and N Maroon Lidar data that I generated while taking a look at the prominence of N Maroon. It doesn't quite look like a picture, but it is at least a reminder that you are looking at data from mountains.

Image

Once you have imported and formatted the Lidar data as you described in your tutorial, 3D Maps are easy to generate from the toolbar (View>New 3D Map View), though you may need to zoom out to find the point cloud.

Keep up the good work!
Pretty cool! Thanks, Adam.
User avatar
JoeGrim
Posts: 25
Joined: 5/2/2017
14ers: 3 
13ers: 8
Trip Reports (0)
 
Contact:

Re: LiDAR Peak Analysis: What It Takes

Post by JoeGrim »

Hi. Thanks for this discussion!

I'm starting to work on this too. Would someone be willing to explain how we can only view the "Ground" (classification 2) pixels in QGIS? Or describe another way that you can differentiate between the ground and vegetation (or other) elevations in the "Attribute by ramp" Z-coordinate display?

Thanks!
User avatar
jkirk
Posts: 65
Joined: 7/19/2005
Trip Reports (0)
 
Contact:

Re: LiDAR Peak Analysis: What It Takes

Post by jkirk »

JoeGrim wrote: Thu Jan 20, 2022 1:57 pm Hi. Thanks for this discussion!

I'm starting to work on this too. Would someone be willing to explain how we can only view the "Ground" (classification 2) pixels in QGIS? Or describe another way that you can differentiate between the ground and vegetation (or other) elevations in the "Attribute by ramp" Z-coordinate display?

Thanks!
Duplicate the layer, make it visible, set to class instead of elevation ramp, then delete out everything but the class you want (class 2). Enlarge the point size to 2mm and color it black. Then class 2 (or whichever class) will have a black border if it's size is 1mm. May work better for you if point size is 2mm in elevation ramp and 3mm in class.
Example:
thumbnail_image.png
thumbnail_image.png (12.45 KiB) Viewed 2190 times
User avatar
Eli Boardman
Posts: 660
Joined: 6/23/2016
14ers: 58  1  15 
13ers: 18 1
Trip Reports (16)
 
Contact:

Re: LiDAR Peak Analysis: What It Takes

Post by Eli Boardman »

Thanks John for pioneering the color-rim method for visualizing classification and elevation at the same time! QGIS should have that functionality built in, but since it's a beta version I can hope they're still working on a more flexible symbology menu.

Anyway, I decided to try my hand at some forested peaks today and figured it would be worth sharing some screenshots as an addendum to the tutorial.

Step 1: Figure out where the trees are (in general). Just like when you're first looking at all the terrain around a peak to understand the topography and identify potential summit areas, I think it's helpful to play with the color ramp a little bit before zooming in too much. If you set the Min/Max elevations to about 20-40 feet above and below the expected summit or col elevation, you should be able to see individual trees show up as vaguely circular clusters of high-elevation returns in front of a smoother-looking background slope. In this view, the symbology ramp spans 20 m, and the trees are clearly visible as the bright pink spots. (Recall that I like to change my highest elevation color swatch to pink for easier identification of points exceeding my upper threshold.)
LiDAR_trees.JPG
LiDAR_trees.JPG (189.19 KiB) Viewed 2048 times
Step 2: Make sure you understand the topography. Now that we know roughly where the trees are and how they look on our screen with different color ramps, we can choose a color ramp that makes sense for the underlying topography: perhaps plus or minus a few feet once we narrow in on the saddle or summit. Here, my color ramp spans a couple meters, and you can see what looks like a pretty clear saddle (with a notable road cut). The trees are once again the bright pink areas, but by narrowing our threshold enough, most of the color ramp is spread over the actual ground elevations. At this stage, it's important to get a reasonable idea of what the summit/saddle elevation will be: if the topography is clearly visible and your thresholds are only a few feet or meters apart, the candidate points must lie in the range between your Min/Max elevations (so if we accidentally select a tree point at a later stage, we should be surprised that it's so much higher and we can thus rule it out). Note that we haven't done anything with the classification yet--in my opinion, getting a feel for the actual shape and elevation range of the underlying topography is crucial.
LiDAR_saddle.JPG
LiDAR_saddle.JPG (163.5 KiB) Viewed 2048 times
Step 3: Pick final points based on classification. Now, we can use John's method to easily identify ground-class points. Right-click your layer and select "Duplicate Layer," then open the symbology menu on your new layer. (N.B.: duplicating usually places the layer directly "under" the original layer, which is what we want. You can also play around with putting the duplicated layer "on top" of the original layer by clicking and dragging the layers in the "Layers" panel.) In the duplicate layer's symbology menu, change it from "Attribute by Ramp" to "Classification" in the top-most drop-down menu. Un-check the box next to "Unclassified," and double-click on the color swatch next to the "Ground" class. Change its color to something distinctive (black works well), then change the "Point size" to 1 mm larger than the point size in your original layer. (I used 2 mm and 3 mm in these screenshots.) Click OK, and you should now see black borders around the ground-class points. You can now proceed as you normally would to identify summit or saddle points, but only choose those points which have black borders.

Final point to note: when a road cut interferes with the natural saddle, it seems to be customary to choose a "col" point from the lowest natural ground on the original slope. This seems a bit subjective to me, since it's unclear where exactly the road cut ends and the "undisturbed" ground starts, but hopefully in most cases it won't make a difference in the peak's ranking. It seems like this is one of the areas that a new type of "LiDAR soft ranked" peak type could emerge, say if the lowest natural ground above the road cut gave a peak 298 ft. of prominence.
LiDAR_col&road.JPG
LiDAR_col&road.JPG (164.18 KiB) Viewed 2048 times
P.S. The example screenshots are from the saddle analysis of THIS originally soft-ranked Nevada peak, which turned out to be ranked with 322 ft. of prominence. :)
User avatar
Eli Boardman
Posts: 660
Joined: 6/23/2016
14ers: 58  1  15 
13ers: 18 1
Trip Reports (16)
 
Contact:

Re: LiDAR Peak Analysis: What It Takes

Post by Eli Boardman »

Addendum: be careful what you believe! The classification is not perfect, and in general, the classification process involves some assumptions of "smoothness" for identifying the ground. In a sense, this is the exact opposite of what we're trying to achieve--the usefulness of LiDAR is its ability to show the sharp points, not just the average surface.

It is absolutely critical that you inspect each summit or saddle area in Google Earth, in person, or with other high-resolution imagery to understand what's going on. On completely vegetation-free summits, I don't think that using the classification is particularly useful--barring weird cases with radio towers or people, what else would it reflect off, if not the ground? One thing to consider is the number of returns. In general, we should only be using points that are the "first and only" return, meaning that the laser encountered something opaque (like solid ground, as opposed to semi-transparent tree branches). In the below example, I'm looking at THIS peak near Lake Tahoe. Note that there appears to be a bit of talus right at the summit, and the highest return is likely off one of these blocks. In the QGIS window, note the circled "Number of Returns" bar, showing that this is the first-and-only return. I have no reason to believe that this point and the other similarly high points surrounding it are anything other than ground returns off a sharp talus block, and selecting the highest ground-classified point in this case would likely underestimate the summit elevation by ~2 ft.
TrickySummit.JPG
TrickySummit.JPG (113.03 KiB) Viewed 2008 times
TrickySummit_GoogleEarth.JPG
TrickySummit_GoogleEarth.JPG (223.71 KiB) Viewed 2008 times
Post Reply