Aerial analytics: Laying the tracks of a new intelligence

This article first appeared in Global Railway Review

Remember your first train ride? Depending on where in the world you grew up and when the memory differs. Mine is from the 1970s in India: a trip back to my ancestral home in Kerala. From the minute the engine blows its resoundingly loud whistle, there’s magic in the air. You scramble to your seat alongside strangers soon to be friends. And while you gaze out through the window into the expanse as the train rolls along, you can’t help but think that this box on wheels is a slice of home. As warm as it gets, and just as safe.

That’s the romantic in me talking. The pragmatist pulls me in another direction: to peek closer at the 123,542 kilometers of rail tracks in India. When you’re dealing with such a large network, the statistics are overwhelming, and you’re bound to have room for error. Despite being the world’s eighth-largest employer with 1.4 million workers, the Indian Railways just can’t man every crossing or regularly inspect every inch of every track, can they?

Gauging the state of the safety art

Most accidents are avoidable, the common causes being derailments, level crossing accidents, collisions, and other hazards such as fire. From 2013 to 2018, the Indian Railways saw an average of 110 accidents per year. Great Britain witnessed 517 train accidents in 2018-19, while the US reported 2,200 incidents in 2018. Before I paint too grim a picture, on the bright side, all these numbers are the lowest they’ve ever been since the inception of the railroad over two centuries ago. In fact, the US numbers in 2018 were 92% lower than the 12,000 incidents in 1972. But maybe we’ve come as far as humanly possible and need a little help as we pull into the last station, the final frontier of efficiency.

Keeping an AI on the (rail)road

Could there possibly be a day where we report zero incidents in light and heavy rail transport? It’s a question we often mull over at Ignitarium’s R&D lab, “The Crucible” as it is called. A couple of years ago, we sensed the opportunity to put a higher sense in the sky: aerial analytics. Imagine being able to spot every fault in the line. Then building a large enough data set to overtime, predict the next fault. Do it without disrupting train schedules. And do it all with limited operator intervention. That’s a tall order. And the sheer challenge made it a use case worth exploring. But before we hopped onto this video analytics wagon, which was set to grow at 42% annually by 2022, the question was: we’ll cross the viability barrier, but how do we make it scalable?

Innovation, from the yard to the junction

Iron Man may have Jarvis, but we generally have to work with more science than science fiction. Here’s the ‘stark’ reality: hours in the lab are long and prototype building is an endlessly iterative process of elimination for which one must toil tirelessly. As we tinkered with beta versions in our lab, we tried to ace the first test of any AI solution: running it without human supervision. Aerial analytics obviously meant mounting a camera on a drone or a fixed-wing aircraft, but how would we train it to be highly accurate irrespective of variations in movement?

The first step was to achieve automatic control of the mounted gimbal to focus on the center of the track at all times. Once that was done, we ensured no pilot would be required to control or maneuver the aerial vehicle. This was achieved by adding the captain’s wings to our AI software implemented on an Nvidia Edge device. Once the vehicle and gimbal were up in the air, the rail tracks were streamed in 4K from as far as 500 meters away. And complex image processing algorithms were written and rewritten on the video stream for image registration, detection, and classification, all working in parallel. At first, the AI was trained to look for obvious faults in the track: cracked or skewed ties and missing spikes or fish plates. Then, as it learned more, it was time to go deeper: see and tag foreign objects and even vegetation of all kinds. After multiple beta builds, it was time to set our AI free and let it fly.

Gathering steam: tunnel vision, sounding the alarm, and more

With our solution put to work for a leading aerial imaging company in the US, we felt a sense of pride. And yet, the journey was only beginning. We switched tracks to other obstacle detection approaches more work for us, and more processing for Matlab. Think of the combined powers of aerial analytics with cameras mounted on trains, trained to spot disruptions even in the dimmest of conditions: night, rain, snow. Or an AI hammer that analyzes the sound of the train passing through a tunnel to detect abnormalities. And what if the AI could study the historical footage of the visual landscape around a train’s regular route to identify disparities with the current footage? It’s not a what-if: we’ve made it possible to implement economically, so the cost of predictive video analytics isn’t even a consideration when weighing it against human safety.

That, I believe, would make my 21-year old self really proud. As a newly minted electronics engineer from the Bombay University setting out on my rail travels in the 1990s, I rolled up the window from my side berth in the coach. With a giddy optimism for the future, I gazed ahead at the tracks as the train snaked along a steep curve bend. Little did I know that a couple of decades later, I would have sent a drone up ahead to gaze at it with me.