Sergey Nivens - Fotolia
Big data recognition technology the next frontier for machine learning
It's one thing to have big data, but it's another to be able to understand it. That's why big data recognition technology is so important to the world of machine learning.
Simply having big data isn't enough if there's no way to make that big data useful. Solving that challenge often boils down to the power of recognition technology. If human subjects are presented with a type of fruit they've never seen before, they can instantly identify an image of that fruit—even if it's a different color, in motion, larger or smaller, against a new background, or with a variety of other variables thrown in. Teaching a machine to recognize an image of a fruit is a lot more difficult since the machine needs to see hundreds or thousands of different images to start distinguishing the relevant patterns.
Telling apples from bananas is not a life or death situation. But being able to tell the difference between a landmine and a can of soda could be. Twenty years ago, when Dr. Meltem Ballan, a Data Scientist with Clarity Insights, was working on a project to clean up the great junkyards of the military, ground penetrating radar was certainly advanced enough to pick up the location of objects underground and tell the difference between metal and non-metal objects. But processing the big data samples collected based on the shape of the object detected wasn't as easy. There were a lot of false positives and false negatives. Today, with better ways to prep the data and more sophisticated analytics tools, it's possible to determine the nature of the object being detected with a high degree of confidence.
Unlocking the power of big data recognition technology
In today's world, big data recognition technology is driving many processes. In Europe, eye scans are commonly used to accurately verify the identity of account users at ATMs. There's even a retail solution that uses sensors to "sniff" produce and identify what's being placed on the scanner. But there's still a long way to go in making the most of image analysis. One area of innovation is consumer-sourced images like those collected as part of a secret shopper program. Meltem was surprised when she started investigating this particular use case. "I always thought it was an urban legend, but it is a very lucrative job." She described the process such shoppers go through of visiting a specific store to capture an image of the current pricing for an item, then taking it to a competing store to share the data.
"You might go to Burger King and take a picture, then take it to McDonalds where they put the photo in their database." The obvious reason to require submission of a photo is to verify that the secret shopper was actually at the location in question and deserves to be paid. But this image-heavy data could also be used to analyze the price points of a huge variety of competitor items on a large scale. Of course to do this, the images would have to be processed in a way that made the text in the photo recognizable as such—regardless of photo quality, lighting, font, size, and other issues.
Image analysis and big data recognition
Taking the raw data, preparing it, extracting the relevant features, and classifying the results is the basis of image analytics. Deciding on the relevant features determines the steps used for processing. For example, the letters and numbers on a menu sign would be relevant in a fast food price audit. Pre-processing might include a complex sequence of steps, starting with turning the image to grayscale. Thresholding, blurring, edge detection, line and shape detection, and more adjustments might have to take place before running the image through optical character recognition to extract the text in a usable format for analysis.
After that, classification can happen using supervised learning (with a known input and output) or unsupervised learning (figuring out the output from the known input). The former method might be used for regression analysis and forecasting, while the latter could highlight buyer behaviors through clustering and association. According to Ballan, "Unsupervised learning techniques are what we are using in the bountiful world of artificial intelligence and deep learning."
Why wasn't this possible in the past?
"We didn't have enough samples and the algorithms were so costly we couldn't run them." Now, with a massive influx of data from various sources along with cheap storage, compute resources, and open source tools, advanced analytics is much more achievable. In Meltem's words, the actual analysis could become the easy part. In short order, "It will be trivial to run these algorithms with a little understanding of technology." Once users have a basic grasp of platforms and tools like Spark, Hadoop, and Tesseract-OCR, the real question becomes how to use the data.
This is a discussion that's as broad as it is open-ended. As Ballan pointed out at the beginning of her talk, boundaries between disciplines have been eroded to the point of being almost invisible. There are many more voices at the table now. "You are sitting in a room with data scientists or analytics people with no business background and sitting with a business owner or VP of Marketing to discuss the data."
When this happens, understanding the data is only a portion of the issue at hand. "It's about understanding people." As part of her own job in consulting with business clients, communicating about the purpose of image analytics projects is as important as exploring the possibilities of the data. Perhaps that's the newest skill set all data scientists should learn as they prepare for a future that will, in other ways, make their jobs much simpler.
