Find the right model for developing AI applications
Developing AI apps isn't a walk in the park. Here's a guideline on how it differs from traditional application development and what your IT teams need to be aware of.
The process of developing AI applications requires a different approach from the more traditional methods, and you'll need to work with code and models from both approaches to create more technologically advanced apps.
A key difference is that developers tend to have some idea of how to translate a set of requirements into working code. Developing AI applications starts with an idea of something that might work, and then a data scientist conducts a lot of experiments to find out if it is practical and how best to make it work. Inaccurate data can throw a monkey wrench into the planning process, and data scientists will also need to balance the use of GPUs and special-purpose processors to find the correct combination of algorithms and processors to create a practical app.
At this stage, data scientists throw it over the wall to a data engineering team to figure out how to put a machine learning model into production. Developers use these models to write applications against, either to consume them or feed them the appropriate data from the AI application.
The data science process
Data science research processes are exploratory in nature and focus on data preparation and model training, said Matei Zaharia, co-founder and chief technologist at Databricks, which maintains the Spark platform. The bulk of the data science process first requires the acquisition of suitable data for the machine learning topic at hand, without which machine learning would be irrelevant in this endeavor. The data is processed in various ways until the program is capable of making correct and repeatable predictions about the topic being learned. The process is iterative and tedious, as data scientists incrementally improve the model's accuracy by exploring and tuning multiple data preparation steps and models.
After these steps, the actual deployment to production focuses on scale-sized computing and predictions and often falls into the hands of a separate data engineering team. Once the model is deployed, the team will need to continuously monitor the performance and accuracy of the model and manage further training and tuning based on new data.
Organizations face many challenges throughout the machine learning lifecycle due to a lack of standardized tools and practices to move models to production. "Unlike traditional software applications, [machine learning] applications can go wrong in production in new ways, including different properties based on the input data, different distributions of data and unforeseen behavior on rare inputs, such as a rare category of users," Zaharia said. These applications can also be a challenge to monitor, as they require careful analysis to track application-specific metrics and behavior on specific types of inputs.
In the data science research phase, it's difficult for teams to track, document and later reproduce results. Data scientists go through many versions of the input and preparation steps, and they adjust parameters and training code as they build a model. When they finally have a working version, data scientists need to be able to reliably and exactly reproduce these steps to have it rebuilt or served in production.
Ongoing development of AI applications
Most organizations assign a separate data engineering team to make the application operational because it's not an area of expertise for data scientists, Zaharia said. This can greatly limit the model transfers to production, as engineers aren't familiar with the work scientists did and which modifications might need to be made to the original code.
After a model is deployed, it's critical for both data scientists and data engineers to monitor its ongoing performance and tune it. Developers and engineers will struggle to pinpoint the proper area to make an alteration because of design or infrastructure changes made since its creation.
Finally, across all these steps, data scientists and engineers can use hundreds of different software tools -- such as open source data science and machine learning packages -- to build their work. Tool management and support can be particularly challenging, Zaharia said. Some organizations limit the set of tools they use, but this can hamper data scientists' ability to build good models since the machine learning algorithm landscape rapidly changes.
Build a machine learning platform
One approach is to use or build a platform that manages the machine learning lifecycle from data exploration to model deployment. Potentially, this platform would oversee all phases of developing AI applications, from data preparation and initial model builds to production sends and, finally, rebuilds and tunes. Machine learning lifecycle platforms enable both data scientists and data engineering teams to exchange workflows and manage these phases of the lifecycle. One area to be especially aware of is step reproduction and replication, so developers and engineers can duplicate their efforts on future models. Some potential options for tools of this sort include MLflow, FfDL and H2O.
One common practice is to focus on ways to automatically manage workflows and minimize the amount of customization needed for each change. If developers work with a standard process that automatically makes sure that new models can be deployed, reproduced and monitored, this will greatly improve their time to market and ability to maintain and evolve production applications, Zaharia said. At the same time, developing AI applications should be flexible enough that data scientists can use the best available machine learning libraries and algorithms to build great models.
