A case study in weather prediction using logistic regression, unbalanced datasets, and l1-normalization
I recently had an interview with a consulting company, and they posed a fascinating scenario:
Let’s say it’s your second day on the job, and your supervisor calls in sick. They ask you to fill in for them at a meeting with top officials from the Department of Energy. What would you tell these people if they asked you how your background in machine learning could help them with their objectives?
Well, thinking on my feet, I came up with what I felt was a pretty good answer, but in the week since then, I’ve pondered that question a lot. I personally believe in using machine learning as a powerful tool to understand real-world phenomena, rather than as a black box to just get answers. But how does that perspective inform real-world energy policies? As I’ve listened to podcasts and read articles over the past week, I’m starting to get some thoughts.
My answer at the interview was informed primarily from a podcast that I listened to recently from Wood Mackenzie about energy policy in the EU in the face of the recent Ukraine crisis. Take a listen, it’s fascinating. One of the points that resonated with me is that energy policy is usually centered around climate change arguments. And that’s not a bad thing. The issue with that is that climate change is always a long-term approach; we’re always thinking 50 years down the road, and while it is urgent to do things now, the sense of impending doom is too far away to really spur serious action. However, the recent Ukraine crisis has highlighted that energy policy isn’t only about long-term climate change, but also about very immediate national security issues, especially in the EU. As the EU attempts to wean itself from Russian gas in a short timeframe, policies and technologies must be implemented fast.
How does this relate to machine learning? One of the most common refrains that I saw in my recent graduate experience is that machine learning is fast. One professor I interacted with published a paper on the computational efficiency of predicting reaction rate constants with machine learning compared to classical, semi-classical and quantum mechanical approaches. The computational savings were extensive. Or, consider the immensely complex field of computational fluid dynamics (CFD), which is an important field in airplane design or weather forecasting. This paper talks about another innovative way that neural networks are used to improve the computational time of fluid flow simulations compared to standard CFD. A quick Google Scholar search will yield countless papers in fascinating fields that all show this concept.
So, in this world where suddenly our timeline for informing energy policy is becoming more urgent, machine learning helps us get answers fast. Rather than hire a team of PhDs to work on a problem for 5 years, we can ideally get answers in less than a year by using the copious amounts of raw data available and a team of machine learning engineers.
Fast results are great, but we always run into the question:
Do we actually understand why we’re getting the results that we get?
Another professor I interacted with had a fantastic way of using machine learning to inform real-world solutions. In developing perovskite solar cells, his team and collaborators performed some complex physical models relating to water-accelerated photooxidation, which is important to understand the lifetime of a cell. But they didn’t stop there. After developing the model, they fed the results as a feature into a previous ML-based degradation model, then used LASSO regression to obtain feature importance. Turns out, the model selected the results of their physics-based model as a significant feature, validating the work. The lesson from this? Machine learning is actually a powerful validator for real-world understanding. If an objective computational algorithm says that a result is important, without the confounding complexities of politics or research agendas, we can move forward with greater confidence in our physical understanding.
The third benefit I’ve considered is the ability of machine learning principles to be used across multiple disciplines. Any time I go to a machine learning conference or poster session, I’m amazed by the breadth and depth of the engineering topics covered. What gets me really excited, though, is that I actually can follow most of the presentations! Even though I may not be an expert in something like cell biology, I can go to a presentation on that topic that is using machine learning, and I can see exactly what they’re trying to do and understand why their results are so significant. There’s certainly a reason that machine learning has permeated almost every field in the past decade.
This is particularly important in the field of renewable energy. Renewables are a vast field, with many technologies that need to be integrated seamlessly. We have concerns in materials science when manufacturing PV panels; invested interest in electrochemistry to understand best-performing batteries; a need to know about electrical engineering to design inverters and battery charge controllers. Then we have to think about grid interconnections and energy policies and everything in between. Now, by no means am I saying that machine learning allows us to become an expert in all of these areas. In fact, I’m a huge proponent of using machine learning while always having a foot in the real, physical world. But the advantage that machine learning gives us is in collaboration. Experts in a single domain can have discussions with experts from another domain that they may have very little understanding of, and yet still speak a common language, and even get insights from the other on how to do their own work more effectively! Now that’s a pretty great synergy.
So those are a few of my thoughts. In summary, machine learning provides three distinct advantages in helping us tackle the challenges around energy better:
- We can get preliminary results fast, rather than spending years on something that might not even work.
- We can leverage machine learning to gain additional insights or to verify intuition using an unbiased mechanism.
- We can facilitate interdisciplinary collaboration between domain experts with a common language and methodology.
I’m curious what your experience has been. Is a machine learning approach faster? Does it give you more insight into the real world that you might have missed? How would you have answered that question? Feel free to connect with me on LinkedIn!