Predicting high-altitude vehicle launch opportunities using machine learning: a preliminary investigation

Introduction

Weather is far from a new concept. Since as early as 650 B.C., humans have been attempting to predict the weather for events as important as farming or natural disasters to events as mundane as picnic weather (1). Initially, these systems were based on viewing clouds and attempting to use astronomical observations to determine future weather. Since then, advancements have been made to the point where artificial intelligence (AI) and machine learning (ML) have begun to be implemented in modern weather prediction systems.

Advancements in the capabilities and use of AI and ML have found their way into many aspects of engineering and operations. The art of weather forecasting has long been a struggle of identifying patterns in the Earth’s atmosphere with numerous factors and variables, including both the Earth’s atmosphere and external elements such as solar flares and radiation. While there is a large effort to improve existing weather models and to create standalone models based on AI, there is a gap in specialized ML models dedicated to specific operations involving the weather. Aerospace endeavors are often locked into timeframes and dates depending on weather and environmental conditions and require copious amounts of planning and forethought to avoid wasted money, potential damages, and potential delays. Due to the complexity of Earth’s atmosphere and environmental conditions, a step-by-step process must be taken in order to identify the most optimal way to tackle such a broad range of problems.

This research project is a preliminary investigation into determining which independent variable(s) have the largest impact on accurately predicting future weather and subsequent events. This is accomplished via the use of AI algorithms trained on altitude-based wind speeds. Future endeavors will be carried out with additional training on various weather elements, including wind direction, temperature, pressure, and humidity. While other researchers have sought to develop algorithms and training methods to predict specific weather conditions on given days, this research seeks to determine the most impactful variables and elements to focus on for the specific application of predicting high-altitude vehicle launch opportunities. Such opportunities include rocket launches, high-altitude aircraft, and the particular focus of this project’s training, high-altitude balloons.

High-altitude ballooning, along with other aerospace endeavors, requires extensive preplanning and preparation for vehicle launching. In ballooning specifically, weather conditions are especially effective and driving whether or not a launch can occur. As most flights must be shaped around the flight path, both for safety and recovery reasons, it is imperative that any acceptable flight path and day may be considered. The goal of this project is to minimize the complexity and manpower requirements for determining if a launch can occur to better open up launch day possibilities. When considering a balloon launch possibility, several factors must be taken into consideration, namely, ground wind speed and direction, wind speeds and direction throughout the altitudes the balloon will be present in, gust speeds and directions, temperature, and humidity. Monitoring all of these factors requires a host of equipment as well as multiple skilled personnel to monitor this equipment and provide feedback to the launch director. A balloon launch also takes a large amount of time from initial equipment setup through to vehicle launch. Reducing any burden on operators’ preflight or during launch would increase safety and reliability in these systems.

A typical time frame can see equipment set up at 00:00 lasting through 08:00. During this time, systems must be monitored by a specialized team. If conditions appear to be in line with the predictions, vehicle and balloon hardware will begin to be rolled out at 07:30 and completely set up around 10:30. System testing will occur next, spanning from 10:00 to 11:30. Balloon fill will occur following this, from 11:30 to 12:30. Balloon launch procedures will follow, ending with vehicle and balloon release around 13:15. This large amount of time requires stable, predictable conditions while also requiring trained staff monitoring and setting up systems.

The rest of this paper is organized as follows. Section 2 presents the related works. Section 3 presents the methods. Section 4 presents the training data. Section 5 presents the weather models. Section 6 presents the data fields. Section 7 presents the results. Section 8 presents a discussion of the results. Section 9 presents some limitations of this work. Section 10 presents the conclusions, and finally Section 11 presents future directions.

Related works

Haupt et al. (2) sought to predict solar power production and efficiencies based on the weather in the area and available sunlight. In contrast to that, this project is centered on the weather conditions and whether or not they provide suitable conditions for high-altitude vehicle launch. Haupt et al. (2) utilized a neural network with some additional preprocessing to form separate regimes. Although this project is not intending on performing this same preprocessing, it will follow suit with the usage of an ML model being used for weather prediction.

Dewitte et al. (3) sought to explain the need for, and importance of, using AI and ML in the fields of weather prediction and climate change to minimize the challenges of insufficient scalability of traditional methods. They compare various AI/ML architectures, including 4D-Var, back-propagated deep learning (DL), convolutional neural networks, and recurrent neural networks. Based on the resulting accuracy, hardware performance requirements, and development complexity, Dewitte et al. (3) determined that any form of the neural network provides better accuracy than traditional non-AI methods while not requiring too much additional development effort. They also suggested that pure AI is not understood enough, and as such, numerical weather prediction models should be used in conjunction with AI for the best performance.

Anandharajan et al. (4) discussed the need for using ML to help predict weather forecasting while including additional variables like maximum and minimum temperature and rainfall. Their team determined that a linear regression model was the best fit for their problem but noted an issue about the need to manually update weather parameters. A particularly interesting part of their research was about how they split up existing data: 20 for cross-validation, 20 for testing, and 60% for training. This allowed them to learn about their hypothesis’ bias and/or variance to be adjusted.

McGovern et al. (5) covered a wide variety of ML algorithms present in other systems and research papers, as well as several they attempted to use too. This paper serves as a wonderful baseline to direct research to many other systems that are trying to accomplish similar goals to this project, such as predicting if hail will occur. McGovern et al. (5) particularly tried out three new ML algorithms that our team was unfamiliar with gradient boosting random tree, random forest, and elastic nets. In addition, their team also used multiple pre- and post-processing algorithms to further attempt to refine their predictions.

Research methods

Research design

This research was designed with an iterative approach through increasingly more complex ML algorithms. Due to time limitations, the scope of the project was limited to a preliminary investigation and the viability of more extensive research. This was done by both accepting the limited capabilities of the available premade ML algorithms outlined below and reducing the number of data sources and types collected and corroborated. While this would prove to impact the accuracy of the final ML model used, it would serve to outline some of the key features future researchers should make note of and improve upon.

Binary perceptron

The project begins with a simple ML algorithm in the form of a binary perceptron to predict a simple yes or no response for a single hour of wind speeds. This algorithm was implemented using SciKit Learn’s perceptron model in Python and was trained and tested on single-hour, single-altitude wind speeds to predict if those conditions were suitable for a launch attempt. This simple implementation and solution could be utilized to prescreen incoming data and highlight days of interest. For this research, this model served to validate the methods of data collection, training, and testing but was quickly transitioned out to more capable models.

Linear regression

From this, the project was expanded to a single-variable linear regression model to predict the percentage chance of being able to launch, again using just wind speeds as the independent variable. SciKit Learn was used once more for the implementation of their linear regression model. This component of software was used to begin narrowing in on predicting future conditions as opposed to the perceptron’s classifying of a current condition. This model was fed in a single averaged low-altitude wind speed (LAWS) on an hour-by-hour basis to predict the chance percentage that the next time step (hour, day, week) would provide a potentially suitable launch day. This model would serve to be a stepping stone for more comprehensive models. The next model utilized was a single independent variable linear regression model used to verify that a correlation in the data could be calculated. A SciKit Learn model was again used as a stepping stone to a multiple linear regression model that could utilize more data fields in the data correlation. The linear regression model was not used for any results data.

Multiple linear regression

To increase the number of independent variables to encompass all that are taken into account when looking at potential launch days, a multiple linear regression model was utilized. This model was implemented with Statsmodels’ Python algorithm. Due to limitations with Statsmodels’ multiple linear regression model, only three dependent variables were used at a time. This led to the use of average wind data in the form of three weather bands, LAWS, middle-altitude wind speed (MAWS), and high-altitude wind speed (HAWS). The LAWS band consisted of altitudes ranging from 0 feet above ground level to 20,000 feet above ground level. The MAWS band contains an altitude range of 20,000–51,000 feet. HAWS completes the altitude bands with a range from 51,000 to 155,500 feet at a lower resolution. These altitude bands were used to identify if an ML algorithm could predict future wind speeds with knowledge of the averaged atmospheric conditions.

Training data

Training data for the ML algorithm were gathered from a single location, the National Oceanic and Atmospheric Administration (NOAA). Different weather models were combined, namely, OP40 and Global Forecast System (GFS), to produce the highest vertical (altitude-based) resolution possible from this data source. Data were collected daily, with some data collected in the morning and some collected in the evening. The decision to collect at different points in the day was directed by the distinct difference in weather patterns in the morning versus afternoon and evening. These differences allowed for testing the model’s ability to predict across all possible launch windows. Future research would benefit from the utilization of additional models, described below, as well as measured actual data in the location being studied.

The collected data were then analyzed and labeled with expected percentage launch chances in the following manner: data from 1 h in the future were used to identify the likelihood of a potential launch in the 1 h time slot. This would essentially train the model to identify launch chances in 1 h increments based on the given independent data. It should be noted that these collected data were analyzed and classified by a human who has experience in the industry. This means that although the classification is not random, there is some invariability due to the human element. However, this is an invariability that will exist in the true application of an ML model being utilized to solve the problem expressed in this project.

Weather models

The existing weather forecasting models were utilized to mirror how modern meteorologists predict launch conditions. By using data readily available to most companies and researchers, this project sought to emulate how such an application would operate in a commercial environment. As other AI weather models grow in popularity, it would not be unexpected to have their output fed into programs such as this project as inputs for prediction.

Global forecast system

The GFS is a weather forecasting model that offers moderately accurate forecasts with the advantage of being able to forecast weeks in advance. In addition, this model covers forecasting to >45,000 m altitude. Where this model suffers is in its update speed, being updated only once every 3–6 h, as well as its horizontal gridding, of 28 km within 16 days and 70 km beyond that.

OP40

OP40 is a model from the National Center for Environmental Prediction and is a low-altitude weather predictor that typically generates forecasts up to 3 days in the future. This model has been found to be highly accurate and can serve as a strong baseline for predicting launch weather conditions 24 h in advance. These data were combined with the GFS model to achieve the desired altitude range of greater than 120,000 feet.

European center for medium-range weather forecasting

The European Center for Medium-Range Weather Forecasting is a source of high-accuracy weather forecasting and reanalysis data. This service is offered as a paid subscription but can offer better accuracy and data than more publicly available sources. For this project, these data were not used due to the cost, but this service is often used for operations involving high-altitude balloon launches.

Radiosonde

There also exists a source of training data that is not determined by existing weather models. Radiosonde devices are weather monitoring systems typically flown under small 1 kg balloons to 42,500 m to gather information about wind speed, wind direction, temperature, pressure, and humidity. Hundreds of these devices are launched in the United States alone daily to collect actual measurements of weather data throughout the day. These data are then collected and made publicly available. This project will utilize these data in conjunction with predictive data to train the AI and verify its accuracy.

During high-altitude balloon launches and vehicle recovery, radiosondes can be launched multiple times preflight as well as pre-splash to verify predictive weather accuracy. These data are then incorporated into the trajectory prediction model to influence the predicted flight path for increased splash location accuracy.

Data fields

Wind speed

Wind speed is at the core of potential launch weather. This is largely due to how the balloon will act when it is being filled and being “stood up.” High wind speed will cause the balloon to “lean” in one direction, causing it to potentially impact equipment or even the ground if the wind speed is high enough.

Wind direction

Wind direction is the counterpart to wind speed as it determines which direction the launch system will need to move in order to reduce the load on the balloon at launch. If a launch system does not move “with the wind,” it runs the risk of the payload being damaged via a pendulum swing in the direction the balloon is leaning.

Temperature

Temperature is another factor that can affect the quality and effectiveness of a balloon launch as the temperature of the lift gas in the balloon highly affects its lift capacity and ascent rate. These factors must be brought into consideration in the flight trajectory prediction algorithms and are typically assumed preflight. On launch day, cloud cover and ambient temperature must be monitored to ensure that they are within the bounds set preflight or else there is the risk that a balloon will be overfilled or underfilled.

Pressure

Pressure is an element that in theory should help an ML algorithm “learn” to predict low surface wind timings and improve accuracy in predicting launch conditions. High- and low-pressure waves or ridges can give indications of upcoming, or just passing, low surface winds. Typically, in the Southeast United States, areas in the path of a high-pressure ridge will experience low surface winds in the mornings and can give timing windows for meteorologists to search for.

Results

Perceptron

From inception, it was known that a perceptron model would be incapable of providing the level of detail required for actual operational use. Although only used on a small data set, it was quickly determined that the model would be horribly inaccurate, most likely due to the lack of independent variables. In addition, its output is not entirely helpful as it does not provide much information on how the decision was made and how confident the model was in that decision.

Linear regression

The linear regression model quickly showed its shortcomings and was moved away in favor of the multiple linear regression model provided by Statsmodels. The shortcomings appear to be primarily due to the single independent variable being unable to produce a comprehensive enough function for future weather prediction. However, this model could perhaps serve as a low-complexity method of determining the relationship between wind speed and wind direction. Generally, lower wind speeds promote higher variability in wind direction and a simple linear regression model could serve to identify the likelihood of variable wind direction.

Multiple linear regression

The multiple linear regression model, while it had only three independent variables, was able to provide interesting results that could be analyzed. The complexity involved with weather prediction and forecasting quickly shows itself in these results, and as such, the raw values were disregarded and trends became the more analyzed pieces.

The model struggled to predict the extreme changes in launch probabilities as the wind speeds changed dramatically. This is demonstrated in Figure 1, where the model kept a much tighter range in its launch probability calculations compared to the actual launch chances. The model also heavily favored not launching in any conditions despite several of the testing launch days having above a 70% chance of launching.

An alternate result that was not being sought after but became apparent in the graphed results is the appearance of correlation between high launch probability and low wind speeds in the lowest altitude bands. Figure 1 shows a distinct downward trend in launch probabilities as average wind speeds increase. However, it should be noted that some of the lowest launch percentages also exist in the lower wind speed sections including as low as 8 knots.

Low-altitude winds (LAWS)

Low-altitude winds are some of the most important factors in determining potential launch times due to their impact on the equipment during sensitive time periods, such as vehicle delivery, checkouts, and final closeouts. The training data, and subsequently the testing data, largely favored wind speeds below 13 knots as having a high probability of launching. Wind speeds greater than 20 knots were almost entirely below 10% with variations to this only existing if mid-level winds seemed extremely favorable. As such, it would be expected that the model would predict extremely high values for slow low-altitude winds of less than 13 knots. Figure 2 shows that this assumption is incorrect as there were multiple examples of the high likelihood of launching at averaged speeds in excess of 15 knots.

In practice, low-altitude winds show a combination of high inaccuracy and high variability in time segments with low wind speeds but a high accuracy with low variability in time segments with high wind speeds. As demonstrated by the graph in Figure 2, the high inaccuracy and variability are mainly captured between wind speeds of 8.5 and 12.5 knots. This high variability is most likely due to the majority of testing data containing LAWSs in this range. As expected, most of the inaccuracy is in an over-evaluation of the wind speeds as the training data heavily favored these lower wind speeds. However, there is an unusual jump between overestimates and underestimates. It is probable that this is caused by the data labeling taking place with the insight of specific wind speeds at all altitudes, whereas the training and testing data are only capable of viewing the averages of these altitudes. This is a flaw in labeling the data with more information than is available to the ML model and is covered later in the paper in the future research section.

Mid-altitude winds (MAWS)

The MAWS band was used in an attempt to identify the impact of these wind speeds on future LAWSs. As demonstrated in Figure 3, this band proved to have the most inconsistent result across the entire band when compared to the low- and high-altitude bands. Quite frequently, the actual values jumped between near-perfect launch conditions and appallingly poor launch conditions. The model struggled to keep up with these sharp transitions in the lower range of the band, but it quickly matched them at the higher range. On the surface, it would be expected that the higher speeds in this band should imply poor launch conditions, but this does not appear to be the case as the launch percentage values seem to match lower wind speeds in this band. This leaves a question as to why the model was able to predict strongly at higher speeds and poorly at lower speeds.

It should be noted that the sharp increase in inaccuracy occurs at much higher wind speeds than in the low-altitude speed bracket. Interestingly, although the model is unable to consistently and accurately predict in the lower wind speeds in this altitude bracket, it is able to accurately predict the higher wind speeds, even with their similar inconsistency. Given that the shape of the graph in Figure 4 so closely resembles the shape of the LAWS graphs, the inaccuracy can actually be taken as a measure of consistency in the model. With further improvements, the inaccuracy can theoretically be removed while maintaining the accuracy between altitude bands. Another reason for the interest in these figures is that the location of the inaccuracies is clumped, similar to those in the low-altitude band. This would imply that there is a correlation between the two bands that warrants further investigation and explanation.

High-altitude winds (HAWS)

The HAWS band demonstrates a new viewpoint where the shape in the model prediction closely resembles the actual values. However, as seen throughout the figures, the model consistently underpredicts the actual value throughout most of the data. Oddly enough, there is a large lack of data points found between 25 and 50 knots. In this section, the model severely underestimated the launch percentages. This gap, shown in Figure 5, serves as a bridge between the highly variable low and high ranges of this altitude band. It would be difficult to identify a trend or relationship between launch possibilities and these lower and upper ranges. However, as Figure 6 demonstrates, this altitude band sees some of the best prediction accuracies of the three bands. Outside of the high variability of inaccuracy in the lowest portion of the altitude band, it maintains a relatively smooth and highly clumped accuracy when compared to the other two altitude bands. Much like in the mid-altitude band, it would be assumed that higher speeds would imply lower chances of launch opportunities, but Figure 5 shows this is again not the case. Most of the data points in the highest speed section are above a 50% chance of favorable launch conditions with the model consistently predicting just shy of this number.

Combined wind speeds

The combined wind speeds section is where the most interesting results appear. An obvious inconsistency and inaccuracy can be identified in the LAWS band below 13 knots. Interestingly, this same bracket can be seen moving up the next chart in the MAWS band before falling back to the lower end of the high-altitude band. As identified earlier in the actual versus predicted figure, Figure 7, the model appeared to struggle with identifying how to manage the low speeds in the lower wind band. This would indicate that LAWSs are frequently accompanied by higher MAWSs and lower HAWSs.

Additionally, Figure 7 demonstrates the potential for ML models to be able to provide meaningful results. In this case, the model would be able to provide negative results in that it would determine when days were most likely not going to be a strong launch opportunity. With the model consistently underpredicting the true value, it would provide conservative estimates on when good launch days would be available.

Discussion of results

Although the results produced by the ML model appear to be lackluster in terms of usefulness and capability for real-world use, the patterns identified in the results of the model’s predictions offer a direction to pursue further research endeavors. With improvements outlined below, and the shortcomings identified, a more in-depth research project could be conducted to identify if the patterns theorized were accurate over a greater span of time and with greater resolution. If this were the case, a model could be developed that could serve to reduce the burden on meteorologists and operations managers while also reducing the number of delays associated with unacceptable launch conditions.

The appearance of a correlation between slow wind speeds at lower altitudes and high launch probabilities, though expected, points to additional research opportunities. While the downward trend itself can be explained by lower overall wind speeds being favorable, the existence of low launch probabilities at the lowest speeds indicates that there is potential for more complexity in this assumption. Although potentially simply outliers, these data points demonstrate why weather prediction and launch are not as simple as identifying generalized altitude bands as this project attempted to do.

It should also be noted that this model was both trained and tested on real and not simulated or generated data. Because of this, and the limited timespan that data were collected in, large swings in accuracy are not completely unexpected. It is quite possible that there exist differences in accuracy in year-by-year or even season-by-season research and testing. Such long-term testing would open up avenues for further demonstrating ML’s capabilities in this field as well as demonstrating the existing weather models’ ability, or inability, to identify localized unusual weather conditions. Such conditions would be natural disasters such as hurricanes or weather extremes such as blizzards and droughts.

Limitations

The limitations of this project can be mitigated using more advanced models and a greater range of data captured. As outlined below in the future research section, mistakes such as model limitations in independent variable counts can be avoided entirely. Some limitations, however, such as data gathering, are largely limited by only being able to gather data day by day and as such are subject to time. While historical and reanalysis data can be used, if a research team or implementor of such a model wanted to use the most recent and accurate weather prediction models, they would be subject to this time constraint.

Conclusion

The results of this research demonstrate that there is strong potential for AI to improve launch window predictions, both quantitatively and qualitatively. Although a multiple linear regression model proved to be incapable of properly predicting launch opportunities on the data set collected, it did highlight the complexities and potential patterns developing within the data. The use of more advanced models, such as DL neural networks, opens up a new avenue of research and potential commercial application in solving these complex patterns. In conjunction with other research, it is likely that such a system could prove to expand beyond high-altitude launch prediction and into the world of hazard assessment or other weather-driven features. In conclusion, there exists greater complexity to weather prediction that places a barrier on even lower resolution aspects, such as predicting windows of opportunity, that might be solvable with the use of AI.

Future directions

Although seemingly inconclusive, the data and information found throughout this research reveal that there is additional information to be uncovered. Through improvements in the ML algorithm or a switch to more advanced AI, deeper patterns may be found. Following this, improved data labeling and classification along with an increase in independent variables would allow for a more accurate and complete picture of weather patterns and their effects on launch conditions. A transition to DL or neural networks would allow for a large increase in independent variables and more thorough training. Additionally, these algorithms would allow for additional dependent and result variables that could show estimated wind speeds, gusting, and even direction.

With an increase in resolution created by a change in the ML algorithm, the data would see an equivalent increase in data labeling resolution. As mentioned above, a potential error arose from the data labeling being done with a better understanding of all of the weather conditions for a given day whereas the ML algorithm would only have access to the averaged data. This error would steadily decline as the resolution on independent variables increased for the ML model.

If a new ML model was not selected, improvements could still be made to the data labeling and classification by adjusting the averaging of wind speed bands. Ideally, additional levels of wind data would be collected below 1,000 feet and these data would be averaged to create the LAWS band. This is largely because wind speeds below 10 knots are common below 1,000 feet and are highly

desirable for launch conditions. When these low-speed conditions are averaged with conditions up to 20,000 feet, this level of information is quickly overcome by the larger amount of data found between 1,000 and 20,000 feet (22 data points).

Also, an increase in independent variables would open up many doors for pattern recognition and development. Understanding minute changes in pressure waves would allow AI to detect pressure waves and with them, expect low or high wind speeds accordingly. An ability to learn wind directions would also greatly improve the usability of the model as it would help greatly with estimating wind gust chances.

Author contributions

JH conceptualized the research and carried out most of the research. SB was provided by the direction of the analysis. JH did most of the writing under the supervision of SB, who also heavily edited the paper to bring it to its final form.

References