Crafting GCPs

Introduction

  This lab consisted of creating GCPs which will be used in the field for future labs. The materials used to create the GCPs include a heavy hard plastic, spray paint, canvas cloths, a plywood stencil, and a table saw. To make the GCPs, first the heavy plastic material was cut into about 2 ft by 2 ft squares using the table saw. Then, the plywood stencil was used to create a bright pink "X" on the GCP so it easily identified when processing the imagery. After the "X" was painted on, or when at least half of the "X" was painted on, then a number was painted on the GCP using a bright yellow spray paint. Lastly, time was allotted to allow the paint to dry.
  Below are some pictures taken while creating the GCPs. The GCPs were created in Professor Joe Hupy's garage. In total, 16 GCPs were created, and about one hour was taken to make them.

Fig 10.0: Example of a GCP before painting

Fig 10.1: GCP with half of the "X" painted

Fig 10.2: Painting on the numbers 

Fig 10.3: Allowing the GCPs to dry when finished

Mission Planning Essentials

Introduction

  The goal of this lab is to become familiar with the C³P mission planning software and other mission planning essentials. These essentials will be covered, altitude settings will be discussed, and an overview of the C³P software will be given. A few main missions will be created, two in the Bramor Test Field area, and one in downtown Minneapolis. Issues with the C³P mission planning software will also be addressed. Then, an overall review of the C³P software will be given.

Mission Planning Essentials

Prior to Departing
  Before departing, it is good to learn about the study site. Will there be hilly terrain, radio towers, cell towers, thick vegetation, crowds of people or other obstacles? These obstacles must be taken into account before starting the mission. Some of these obstacles may prevent the mission from taking place. For example, it is illegal to flight UAS platforms over large crowds of people. Before departing, it is also important to check to make sure that batteries are charged, that equipment is is working condition, and that all necessary equipment is accounted for. Lastly, one should check the weather before departing to make sure that mission day is a fair weather day. When checking the weather it is most important to look at precipitation and wind forecasts.

In the Field
  Before deciding on a home, takeoff, rally, and landing point, the weather conditions must be checked. These include, wind speed, wind direction, temperature, and dew point. It is important to take off into the wind and to land with the wind. The vegetation, terrain, and man made features must also be assessed. The elevation of the launch site is important to know, because often UAS platforms will be flown at an absolute height above the launch site. Because GPS units are used for the flying of the UAS platforms frequently, it is important to make sure to stay away from objects which could emit electromagnetic waves. These could include power lines, power boxes, and underground cables. The units used should be standardized. Mixing between the English and Metric system increases the chance for error to occur. Lastly,  one needs to make sure that the study area has a good GPS and cellular signal.

C³P Mission Planning Software

  

Working with the Software
    Creating a mission in C³P begins with moving the home, takeoff, rally, and land locations depending on the study area and the weather conditions. Then, a flight path is drawn using the Draw feature. This gives one the option to draw the path point by point, by area, or by line. The Draw by Area feature is the most commonly used. The missions settings then should be changed, which is discussed below. The mission can then be viewed in the 2D map which the C³P software provides or in in 3D through ArcGIS Earth or Google Earth.
  For this lab, three main missions were created. The first mission took place in the Bramer Test Field. This is where the C³P software defaults the user to. the C³P allows one to control the mission settings which can be seen below in figure 9.0. It also allows the user to move the takeoff, home, rally, and landing locations. These are represented by T, H, R, and I  orange circles respectively. The mission settings allow the user to alter the altitude, speed, overlap, sidelap, GSD, overshoot, camera type used, and the altitude mode of the mission. Generally, the altitude should be a comfortable distance above the highest object one is expected to encounter, the speed should be around 16 m/s to 18 m/s, the overlap should be at least 80%, the sidelap should be at least 70%, the GSD (pixel resolution) is usually left to the default, and the overshoot ( space for the UAS platform to correct itself when turning around) can be chosen based on the study area.

Fig 9.0: Mission Settings

Critical Altitude Settings: Height, Orientation, and Mode
  To show the difference between altitude height, altitude mode, and flight orientation, 6 mini missions were created roughly covering the same areas. Relative altitude mode means that the UAS platform will always flight a certain height relative to the surface. Absolute altitude mode means that the UAS platform will always fly at the same altitude no matter how the terrain changes. Flight orientation refers to the direction the UAS platform is flown relative to obstructing terrain. All of the six mini missions use the draw area points feature. The T, H, R, and I circles are not shown in any of these missions because the point of these figures is to show the differences between certain settings. Besides terrain, altitude settings will need to be chosen based off of what anthropogenic features there are in the study area such as radio towers, cell towers, buildings, and other infrastructure.
  Figures 9.1 and 9.2 show the difference between using different absolute altitudes. Figure 9.1 has an flight altitude of 200 m, and figure 9.2 has a flight altitude of 175 m. All of the the other mission settings remained the same. Notice how figure 9.2 has red circles in the flight zone. This indicates that the UAS platform will crash if it is flown at that height. The red circles indicate that the UAS platform needs to flown at a higher altitude.

Fig 9.1: 200 Meter Flight

Fig 9.2: 175 Meter Flight

  Orientation also affects flight planning. The difference between parallel and perpendicular orientation is shown below in figures 9.3 and 9.4. Figure 9.3 uses parallel orientation which goes with the hilly terrain and figure 9.4 uses perpendicular orientation which goes against the hilly terrain. Both areas roughly cover the same area, but the flight in figure 9.3 would be successful and the flight in figure 9.4 would not. Simply put, the UAS platform would hit something in the flight path in figure 9.4. This is because the UAS platform is instructed to turn around on the large hill. The flight in figure 9.3 would be successful because the flight path doesn't make the UAS platform turn around on the hill. It goes parallel with it instead.

Fig 9.3: Parallel Orientation

Fig 9.4: Perpendicular Orientation

 Altitude mode also affects mission and flight planning. The difference between using relative and absolute altitude mode can be seen between figure 9.5 and 9.6, the altitude set in the mission settings for both is 140 m. Because the relative altitude mode changes the absolute altitude of the UAS platform in the flight in figure 9.5, it would be a successful flight. Figure 9.6's mission would not be successful because the absolute height of the UAS platform wouldn't change throughout the flight depending on the terrain. Therefore, when the UAS platform encounters a hill it would crash right into.

Fig 9.4: Relative Altitude Mode

Fig 9.5: Absolute Altitude Mode

Planning Missions
 Figure 9.6 shows a mission created with the software. It uses the Draw Street Points feature to plan out the route along a road near the Bramer Test Field. The wind in this mission is coming from the east at about 3.6 m/s. The takeoff and landing zones are placed so that the UAS platform will takeoff into the wind and land with it. The takeoff and landing zones should be located in safe areas. They should not be located in the same area! If they are, the UAS operator and other spectators run the risk of  getting injured because often the UAS is controlled from the home and launch site. One should also be careful to locate the landing area away from expensive objects such as cars. C³P, missions can also be displayed in 3D in ArcGIS Earth or Google Earch. The 3D mission for this one is shown in figure 9.7 using ArcGis Earth. The 3D view allows the user to see the surrounding terrain and vegetation near the mission site.

Fig 9.7: 3D  Road Sample Mission

 

Fig 9.6: 2D Road Sample Mission

  Another mission was created in downtown Minneapolis. This is shown below in 2D in figure 9.8 and in 3D in figure 9.9. The altitude for this flight is an absolute 140 m. This flight is completely illegal. The home and launch site is located in the outfield at Target Field, the mission flight area is all around very tall buildings in downtown, and the landing site is located on the roof of Target Center. The wind speed an direction is the same as in the previous sample mission.

Fig 9.8: Downtown Minneapolis Mission

Fig 9.9: Downtown Minneapolis Mission

 An error with the C³P software has been discovered. In North America, the software will not tell the user if the flight will be successful or not. Many of the buildings in downtown Minneapolis are taller than 140 meters, yet there are no red circles in the 2D map indicating that objects will be hit. Also, the 3D map isn't 3D at all. The downtown buildings appear flat with the surface. This causes the software to think that the missions will be successful when it would be a complete failure. This kind of mission planning could potentially be misleading and dangerous.

Review of C³P Mission Planning Software

  Overall, I found the C³P mission planning software to be very useful for planning missions. It even allows for simulation. I did a simulation which really helped me to understand all of the way points (home, takeoff, rally, land, and navigation) and how a mission works. In the beginning, I had to use the help quite a bit which was really useful. The help provides a whole tutorial to setting up a mission. The amount of information which the software is capable of providing is very nice. Being able to account for weather conditions, battery life, altitude and other information makes this very valuable software package. The measure tool is also handy because its a quick way to measure the distance between certain points.
  The downside to this mission planning software is that missions planned in North America cannot be totally depended on. This was shown in the mission located in downtown Minneapolis. This is a good reminder that technology isn't always to be trusted. People must do their own planning as well as using the mission planning software.

Processing Oblique UAS Imagery Using Image Annotation

Introduction

  There are two main types of UAS imagery: nadir and oblique. Nadir UAS imagery is taken perpendicular from an object. This is how most aerial images are taken. In UAS, these photos are most commonly used to create orthomosaics and DSMs. This is the kind of imagery which has been used for previous Geog390 UAS labs. In this lab, oblique imagery will be used. Oblique imagery consists of photos taken at an angle that is not perpendicular from an object. The most common angle is 45°. Oblique imagery allows for 3D modeling where the sides of objects can be measured because oblique pictures area taken all around the object. Common market uses of oblique imagery include emergency management, community planning, and property assessment situations (AIMS).

Methods

  For this lab, there are three sets of oblique imagery which will be used to create 3D models of objects. In previous labs, the 3D maps template was used, but because there is oblique imagery in this lab, no maps with orthomosaics or DSMs can be created. Thus, the 3D Model template will be used. The first study area is at the Litchfield frac sand mine located just southwest of Eau Claire along the Chippewa river. This imagery captured a bulldozer. Other lab data has been collected at this location. The other two sets of oblique imagery are from South middle school located in the southern part of Eau Claire. The second set of imagery captures a storage shed near the athletic fields, and the third set captures a pickup truck in the parking lot. All three oblique imagery sets were taken in a corkscrew like fashion starting from the ground and working up in altitude.
  Annotation will also be used in this lab. Annotation is a tool used in Pix4D after initial processing to remove unnecessary objects in each individual photo. There are three types of annotation: mask, carve, and global mask. Mask is used to rid of unwanted background objects and objects which occur in a few photos which overlap the main object which the 3D model is being created for. Carve, is used to remove the sky. Lastly, global mask is used to delete overlapping objects on the main object which occurs in almost all the photos. For this lab, only the mask tool will be used, even though the carve tool is specifically to remove the sky, the mask tool can do this as well. The data for this lab will also be processed without any annotation to draw comparisons between using annotation and not using annotation.

Image Set One: Bulldozer at the Litchfield Frac Sand Mine
  The first step is to do the initial processing. After that, a copy of the Pix4D file was created so that the data could be processed with and without annotation. Then, images are ready for annotation. Below, figure 8.0 is one of the 5 images that were annotated using the mask template. To annotated the image, the pencil was clicked on in the upper right part of the figure. The template can then be changed in the lower right of the figure if necessary. The down arrow to the right of the mask template provides the options of mask, carve, and global mask.

Fig 8.0: Annotated Bulldozer Photo

  The data was then re-optimized using the annotation and then was further processed using the 5 annotated images with Point Cloud and Mesh processing. This created the 3D model using all of the images. Lastly, the file copy, created before annotating, was used for Point Cloud and Mesh processing.

Image Set Two: Storage Shed at South Middle School
  The same process used for the bulldozer is used for the storage shed. First. the initial processing was done, then a copy of the Pix4D file was created. After that, annotation was used to highlight the areas not wanted in the 3D model which can be seen below in figure 8.1. Lastly, a re-optimize was done using the annotation and the Point Cloud and Mesh processing was ran using annotation on the main file and on the copy file without using annotation.

Fig 8.1: Annotated Storage Shed Photo

Image Set Three: Pickup Truck at South Middle School
  The same process used for the bulldozer and storage shed is used for the pickup truck. First. the initial processing was done, then a copy of the Pix4D file was created. After that, annotation was used to highlight the areas not wanted in the pickup truck's 3D model which can be seen below in figure 8.2. Lastly, a re-optimize was done on using the annotation and the Point Cloud and Mesh processing was ran using annotation on the main file and on the copy file without using annotation.

Figure 8.2: Annotated Pickup Truck

 Results / Discussion

  A fly by video for all three 3D models using annotation were created. The bulldozer video can be seen in figure 8.3, the storage shed video can be seen in figure 8.4, and the pickup truck video can be seen in figure 8.5. These videos are a good visual of the 3D models.

Fig 8.3: Bulldozer Flyby Video

Figure 8.4: Storage Shed Flyby Video

Figure 8.5: Pickup Flyby Video

Bulldozer
 To create comparisons between using annotation and not using annotation, .png files were created with roughly the same resolution. These were of the the final 3D model produced with and without annotation. The bulldozer image without annotation can be seen in figure 8.6, and the bulldozer image with annotation can be seen in figure 8.7.

Fig 8.6: Bulldozer Model Using Annotation

Fig 8.7: Bulldozer Model Without Using Annotation

  Unfortunately, there isn't ant significant difference seen here between using annotation and not using annotation. This is likely because the images taken were already fairly clean in that there were no unwanted objects overlapping the bulldozer. As seen in the figures above, both models had a very poor data quality area in the scoop of the bulldozer. This is probably because this area is enclosed from three directions and the camera can only capture it in the front. Overall, the bulldozer's 3D models produced with and without annotation was of good quality and didn't have any major issues.

Storage Shed

  Next, a comparison was done between the 3D model of the storage shed with and without annotation. These can be seen below in figures 8.8 and 8.9 respectively.

Fig 8.8: 3D Model of Storage Shed Using Annotation

Fig 8.9: 3D Model of Storage Shed Without Using Annotation

  Much like the bulldozer, there is really no difference between the models because there were no unwanted overlapping objects on the storage shed. There is a little difference on the top of the shed where poor quality is present. On the actual shed by South Middle School, there is not random plume on the top of the shed. The reason for this poor quality is unknown. However, both models experienced this issue. The other main issue is the random discolored pixels present in both models. The reason for this is also unknown, but could be because there weren't enough images taken of the shed.

Pickup Truck

  Lastly, a comparison is done between the 3D models of the pickup truck with and without annotation. An image of the the model with annotation is displayed in figure 8.10, and an image of the model without annotation is displayed in figure 8.11.

Fig 8.10: Pickup Truck Model With Annotation

Fig 8.10: Pickup Truck Model Without Annotation

  Just like the bulldozer and storage shed comparisons, there are really no major differences between using annotation and not using annotation present in the pickup truck models. Not shown in the images, but present in both models was poor representation of underneath the tailgate of the pickup truck. This can be seen in the flyby video in figure 8.5. It is a similar to the error that occurred in the scoop of the bulldozer. It happened to both models because the imagery taken in these spots was at a very sharp angle which doesn't allow for much depth perception from the camera.

Conclusion

  Although annotation wasn't necessarily needed for the oblique data sets used in this lab, many times it is. Oblique imagery can be used to create 3D models of objects as demonstrated in this lab. Based off the results from the pickup truck and the bulldozer, it seems that oblique imagery is most difficult to process when there is an object overhanging the desired model area. This happened in the bulldoze and pickup truck models where the tailgate was overhanging the ground and the scoop was overhanging itself. The flight path for taking oblique imagery should be decided based on the area surrounding the object. If there are no objects in the way of the desired object, then a corkscrew patterns starting from the ground and working up should be used just like the image pattern for the 3 oblique image sets for this lab. If there are many objects in the way, a different image aquisition pattern will have to be used.

Sources

AIMS (Automated Information Mapping System), Oblique Imagery
  http://aims.jocogov.org/AIMSData/Oblique.aspx
Pix4D Help, How to Annotate Images in the Ray Cloud
  https://support.pix4d.com/hc/en-us/articles/202560549-How-to-Annotate-Images-in-the-rayCloud#gsc.tab=0

Calculating Volumes of Sand Piles Using Pix4D and ESRI Tools

Introduction

  Volumetric analysis is the process of calculating the volume of objects using software. It can be used to calculate the volume of buildings, mine piles. river valleys, canyons, and more. To calculate the volume of an object using volumetric analysis x,y, and z values are needed. They do not need to be coordinates and elevation. They can be cartesian values. UAS data is a great source to perform volumetric analysis on. Processing imagery through Pix4D creates an orthomosaic and a DSM. The DSM contains elevation values which can be used to calculate volume. Volume measurements can be very accurate and precise when using UAS imagery, especially if GCPs were used.
  In this lab the volume of three sand piles chosen from the Litchfield sand mine will be calculated using Pix4D, 3D analyst tools, and TIN tools. The tools used for the 3D analyst method include Extract by Mask and Surface Volume. For the TINs, the tools used include Raster to TIN, Add Surface Information, and Polygon Volume. After calculating the volumes of the sand piles, a table and a map of the average volume values will be created. The difference between the methods and values will then be discussed.

Methods

Use Pix4D to Calculate Volumes
  To do this, the Litchfield Mine GCP project was opened. Then, the volumes tab was used to create a polygon around the three sand piles. After that, the volume was calculated. Piles two and three are shown below in figure 7.0. Their calculated volumes can be seen in the left part of the image. Pile one's calculated volume is 1086.17 m³, pile two's calculated volume is 18.59 m³, and pile three's calculated volume is 32.20 m³.

Fig 7.0: Using Pix4D to Calculate Volumes for Sand Piles 2 and 3.

Use a DSM Clip to Calculate Volumes

  First, three different feature classes needed to be created in ArcCatalog and edited in ArcMap so that the DSM could be clipped to only the area needed to calculate the volume. These were name Pile1, Pile2, and Pile3. Then, the Extract by Mask tool was used to clip the DSM to the corresponding feature class. The Extract by Mask tool is used to clip a raster (the DSM) to a specific feature class (Pile1, Pile2, or Pile3)  or a different raster dataset. After this, the Surface Volume tool was used to calculate the volume of the three sand piles. The Surface Volume tool calculates the volume based off a specific surface. In this case, the volume wanted is the area above the lowest value of the DSM clip for each sand pile. Using this method, the calculated volumes were 1191.85 m³ for pile 1, 23.76 m³ for pile 2, and 48.04 m³ for pile 3. 

Use a TIN to Calculate Volumes

  For this, the three DSM clips were converted to TINs using the Raster to TIN tool. This tool creates a TIN based on an input raster. Then, the Add Surface Information tool was used to add the minimum elevation value (Z-Min) to the TINs. These minimum elevations values were imported from the DSM clip features. In general, the Add Surface Information creates a new feature class which imports information into its attribute table. After the Z-Min value feature class was created for each pile, the Polygon Volume tool was used to calculate the volumes. This tool used both the TINs and the Z-min feature classes. The Polygon Volume tool is ordinarily used for calculating the volume and surface area between a polygon feature class and a TIN. After doing this for each pile, the calculated volumes were 1202.46 m³ for pile 1, 24.31 m³ for pile 2, and 48.96 m³ for pile 3.

Results / Discussion

  While calculating the volumes of the sand piles using the different methods, the results were entered into an Excel spreadsheet. This can be seen below in figure 7.1. An average and standard deviation field were also added. Overall, all three methods seemed to be within reasonable accuracy of each other. The TIN and 3D analyst methods churned out extremely similar results with the Pix4D volumes having lower values across all three piles. Looking at the difference in the calculated volumes for each pile, it seems that the difference between the methods grow as the size of the sand pile increases. For example, there is a greater difference between the volumes of pile 1 across the three methods than there is between the volumes of pile 2. This is probably because each method is consistent in the way it calculates the volume so larger pile volumes are going to differ more than smaller pile volumes.

Fig 7.1: Sand Pile Volume Table

  A map was created showing the locations of the sand piles and the average calculated volumes using the three different methods. This map is shown below in figure 7.2. Because of the table in figure 7.1, there aren't any surprises in the map. By far, pile 1 is the largest pile by both area and volume. Piles 2 and 3 are similar in surface area so they have more similar volumes.

Fig 7.2: Average Sand Pile Volume Map

Conclusion

   In conclusion, UAS imagery can be used to calculate volumes in at least three different ways. It is interesting to see how similar the 3D analyst and the TIN calculated volumes area. These values are very close to each other because the TINs were derived from the DSM which was used to calculate the 3D analyst volume. The volumes calculated in this lab could be used by the mining company to see how much sand is left in the piles and to see how much time they have before they need to replace the pile with more sand. Volumetric analysis is a much more efficient and cheaper way to get this estimate than actually measuring the pile.  Based off of the three methods in this lab, no conclusion can be made seeing which method is the most accurate. Even though the 3D analyst and TIN methods were similar, that doesn't mean that those methods are more accurate than using Pix4D. A comparison between the actual volume of the sand piles and the calculated volumes using each method would need to be done to test the accuracy.