predictive analytics Archives | dotData | AutoML 2.0 Solutions for Enterprise

Carl Bowen
Blog
September 1, 2020

Predictive Analytics 101: What is it, Why now, and How to Get Started?

TL: DR: Predictive Analytics is using historical and real-time data to generate useful insights and predicting critical outcomes in the future. A large number of organizations are leveraging this AI-powered technique to reduce risks, improve operations, cut business costs, and increase the bottom line.
Gartner defines Predictive Analytics (PA) as a form of advanced analytics which examines data or content to answer the question “What is going to happen?” or more precisely, “What is likely to happen?”, and is characterized by techniques such as regression analysis, multivariate statistics, pattern matching, predictive modeling, and forecasting. Grandview research recently estimated that the global market for PA is growing at a CAGR of 23.2% and projected to grow to $23.9 Billion by 2025. Initially, the purview of a few visionary companies, PA is rapidly gathering momentum in the market. Several industries such as banking, financial services, insurance, and manufacturing are using PA to forecast demand, prevent customer churn, and perform anomaly detection for predictive maintenance.
According to this Forbes article on the state of AI and ML, advanced algorithms and predictive analytics are among the highest-priority projects for enterprises adopting AI and machine learning in 2019. As this illustration from Dresner Advisory Services shows, PA is a top technology initiative for business intelligence (BI). Since reporting, dashboards, data integration, and visualization are mature BI capabilities, augmenting BI with AI for predictive modeling is becoming critically important.

The focus of BI has been analyzing historical performance – what happened in the past and to an extent what can be done to prevent a recurrence using traditional analytics. Predictive analytics, on the other hand, offers the ability to look into the future and enable organizations to take appropriate actions. It can enable organizations to solve very complex business problems, manage risks, and identify new opportunities. PA is ideal for solving complex multivariate problems such as IoT time-series data or situations where traditional approaches such as classical statistical techniques cannot be applied. For example, timely predictions can identify financial fraud, predict buyer behavior, enable dynamic pricing, or detect a faulty component from going into a final product. By providing real-time dashboards, predictive analytics can improve the performance of BI platforms. Due to advances in cloud computing infrastructure and technological breakthroughs in AutoML platforms (that hide all the technical complexities with AI-Powered Automation), PA is very much affordable for even small and medium businesses without huge analytics or IT teams.
Organizations must leverage predictive analytics, an advanced form of analytics, and data science automation to gain greater agility and faster, more accurate decision-making. The best way to get started with predictive analytics is to use automated machine learning (AutoML) tools. A key hurdle in analytics and AI/ML has been finding talent – data scientists, data architects, machine learning engineers skilled in building, testing, and deploying ML models. That’s where automation can play a big role. AutoML platforms allow non-data scientists to automatically build, validate, and deploy predictive models at the touch of a button. The new breed of AI automation platform empowers BI professionals to leverage their data skills and bring predictive analytics to BI stack.
However, those organizations that don’t have any analytics capability or in-house data science expertise will require more than just an AutoML platform. The software platform needs to be bundled with training, use-case co-development, as well as a fully-managed SaaS environment as a predictive analytics program. A dedicated program will be ideal for companies with a strong BI practice and that wish to add predictive analytics capabilities without adding or growing expensive resources and data science teams. The BI and analytics leaders at these companies may have prioritized predictive analytics use cases but either don’t have a budget for hiring more data scientists or simply don’t have in-house data science skills. CIOs, BI, and analytics leaders should look for the following features to accelerate the predictive analytics journey:

AI-Focused data preparation that includes data value cleansing, statistical data profiling, and data re-architecting
AI-Powered Feature Engineering with feature hypothesis, query generation, and feature relevance validation
Model Production with prediction and retraining endpoint generation, model package, and containerized model generation.

The goal of a predictive analytics program should be to leverage existing resources, in-house BI teams, data engineers, or business analysts, and allow them to add more value with AI + BI quickly. Predictive analytics is bringing exciting new opportunities to digital-ready enterprises. A fast start program that is designed to provide everything organizations need to get started with AI is the best approach to incorporate predictive analytics in business applications.

Carl Bowen
Blog
August 12, 2020

AI Automation To Enable Smart Manufacturing

Heavy industries such as Steelmaking are ramping up digital transformation initiatives to improve throughput, efficiency, safety, and reliability of their operations. Metals, mining, and machine tool building companies are embarking on multi-year journeys to digitize operations by adding connectivity, automation, and advanced analytics. According to McKinsey, most heavy-industry sectors are at the middle stages of digital maturity (Digital 2.0) relying on rule-based automation and distributed control systems. Some have made progress in digital maturity (Digital 3.0) and are using collaborative robots and advanced process control systems. However, a few digital pioneers are leveraging AI Automation and applying Machine Learning (ML) to operational data (Digital 4). These digital leaders offer powerful lessons that others can emulate and follow to successfully deploy ML at scale.

Drowning in Sensor Data

With the ubiquitous sensor network and pervasive connectivity in industrial manufacturing, data from production machines has continued to grow at an unprecedented scale. Manufacturing and process engineers need to review operational data across many sources such as ERP systems, MES, historians, computerized maintenance management systems, etc. Due to the sheer volume and streaming nature of data, manual inspection and analysis is not practical and unsustainable. Data from diverse systems come in multiple units, formats, and incompatible protocols. To get a better handle on data, companies have invested in building massive data warehouses, data marts, and data lakes, but that has brought new challenges about data governance and validation. Industrial operations teams need a scalable, secure, and high-performance environment for operational data management, review, learning, and analysis.

AI In Manufacturing

Implementing AI and ML techniques in manufacturing can be challenging for a variety of reasons:

The standard AI workflow is complicated and laborious, with long lead times. A typical AI project involves data preparation, feature engineering, algorithm selection, model training, testing, and deployment and takes anywhere between 6-9 months.
Getting quality data in a manufacturing environment is very difficult as each manufacturing site can have multiple ERP and manufacturing execution systems. With disparate systems, cleansing and aggregating data, building features, and generating models manually can become tedious, expensive, and labor-intensive. By the time you perform training and validation, AI/ML models have a high likelihood of being already outdated.
The majority of AutoML platforms are designed for data science professionals and are not easy enough to use for manufacturing teams. These platforms have a steep learning curve and are cumbersome to use. Often the vendors don’t provide sufficient training or education in data science and how to implement AI for common use cases.
With just-in-time production, many applications have low latency requirements that demand real-time processing. An ML system deployed under these tough operating conditions should have streaming analytical capabilities with the ability to process data under one second (often millisecond processing). A batch processing approach will not work.
Manufacturing sites may not allow cloud-based analytics due to the sensitive nature of data. An on-premises, edge deployment for remote, resource-constrained production environments may be preferred. However, many vendors lack the ability to deploy models on the edge on small memory footprint such as edge servers or gateways.

Industrial IoT, manufacturing, in particular, requires real-time prediction capabilities for use cases such as quality monitoring and anomaly detection. A traditional modeling approach gets very complicated requiring significant investments in infrastructure.

AI Automation At Scale

Any AI/ML solution for the manufacturing environment must be robust, flexible to address multiple use cases, and support real-time streaming capability. Moreover, it should offer the flexibility to deploy models in the cloud, on-premises, or at the edge depending on site requirements.
The ideal solution should incorporate a platform designed and built for industrial manufacturing; a system that is usable by the operations team without requiring additional data science resources. A system that leverages operators domain expertise (Human-Centered AI) to augment performance. Using a supervised ML technique, the system should find patterns in production data, identify deviation in quality, and empower SMEs to prevent recurrence. One of the biggest steel manufacturing companies in Japan, JFE Steel recently deployed dotData’s AutoML 2.0 full-cycle automation platform on their manufacturing plant. JFE Steel has a vision to deploy hundreds of various AI models to implement intelligent IoT Cyber-Physical Systems in their steel manufacturing plants and dotData Stream, our containerized ML model, is a key component to realize the vision at JFE Steel.

Using AI automation, this system hides the pipeline complexities, automates feature engineering, and other onerous steps of the AI process and enables manufacturing customers to build predictive models in days instead of months.

To make AI transparent, it provides explanations and metrics about features with details into the modeling approach and ensemble of models selected. Explainable AI gives insight into predictions, allowing users to understand the correlations, and enables SMEs to understand which process variables affect the outcome for root cause analysis.
Whether it is optimizing efficiency, preventing quality defects, or reducing asset downtime, industrial manufacturing presents multi-dimensional problems. And because the manufacturing environment generates so much data, AI is the perfect tool to solve manufacturing problems. A domain-agnostic, end-to-end AI platform offers analytical flexibility to address multiple use cases, and dramatically improves the life of operational experts as they don’t have to learn new tools for each new project. This is the preferred approach among digital pioneers as it saves time, allows repeatable ML processes, and delivers superior ROI.
AI automation allows SME to focus on day to day jobs, automate data pre-processing, and feature engineering and build models at the click of a button. Explainable AI and transparent features build trust and garner buy-in from domain experts. Containerized prediction model allows real-time prediction capability and accelerates AI deployment at the edge on the manufacturing floor. Empowering manufacturing and production SMEs to do more with less using AI automation is the way to achieve Digital 4.0 maturity level in the heavy industry at scale.

Carl Bowen
Press Releases EN
July 7, 2020

dotData Launches dotData Stream – Containerized AI Model for Real-Time Prediction

Highly Scalable and Effective AI/ML Container, Easily Deployable Either in the Cloud for ML Orchestration or at the Edge for Intelligent IoT
SAN MATEO, Calif., July 7, 2020 /PRNewswire/ — dotData, a leader in full-cycle data science automation and operationalization for the enterprise, today launched dotData Stream, a new containerized AI/ML model that enables real-time predictive capabilities for dotData users. dotData Stream was developed to meet the growing market demand for real-time prediction capabilities for use cases such as fraud detection, automated underwriting, dynamic pricing, industrial IoT, and more.
dotData Stream performs real-time predictions using AI/ML models developed on the dotData Platform, including feature transformation such as one-hot encoding, missing value imputation, data normalization, and outlier filter. It is highly scalable and effective – a single prediction can be performed as fast as tens of milliseconds or even faster for microbatch predictions. Its deployment is as easy and simple as launching a docker container with AI/ML models downloaded from the dotData Platform with just one click. An end-point for real-time predictions becomes immediately available. In addition, dotData Stream can run in cloud MLOps Platforms for enterprise AI/ML orchestration or at the edge servers for intelligent IoT applications.
JFE Steel, one of the world’s leading integrated steel producers, recently implemented dotData to support the deployment of intelligent IoT in their manufacturing plants.
“After testing several leading autoML platforms, we chose dotData as we were impressed with dotData’s autoML 2.0 full-cycle automation of ML processes, including automated feature engineering on our manufacturing data,” said Mr. Kazuro Tsuda, Staff General Manager, Data Science Project Dept. JFE Steel Corporation. “JFE Steel has a vision to deploy various AI models to implement Cyber-Physical Systems in our steel manufacturing plants. dotData Stream will be a key component to realize our vision and JFE Steel is looking forward to expanding its partnership with the dotData team.”
“We are seeing an increasing demand for real-time prediction capability, which has become an essential necessity for many enterprise companies. dotData Stream allows our customers to leverage AI/ML capability in a real-time environment,” said Ryohei Fujimaki, Ph.D., founder and CEO of dotData. “We are honored and excited about our partnership with JFE Steel. Their intelligent IoT application is the perfect use case to demonstrate the ability of dotData Stream, and we are fully committed to supporting their vision to adopt AI/ML in smart manufacturing and achieve the full potential of Industry 4.0.”
dotData provides AutoML 2.0 solutions that help accelerate the process of developing AI and Machine Learning (AI/ML) models for use in advanced predictive analytics BI dashboards and applications. dotData makes it easy for BI developers and data engineers to develop AI/ML models in just days by automating the full life-cycle of the data science process, from business raw data through feature engineering to implementation of ML in production utilizing its proprietary AI technologies. dotData’s AI-powered feature engineering automatically applies data transformation, cleansing, normalization, aggregation, and combination, and transforms hundreds of tables with complex relationships and billions of rows into a single feature table, automating the most manual data science projects that are fundamental to developing predictive analytics solutions.
dotData democratizes data science by enabling BI developers and data engineers to make enterprise data science scalable and sustainable. dotData automates up to 100 percent of the AI/ML development workflow, enabling users to connect directly to their enterprise data sources to discover and evaluate millions of features from complex table structures and huge data sets with minimal user input. dotData is also designed to operationalize AI/ML models by producing both feature and ML scoring pipelines in production, which IT teams can then immediately integrate with business workflows. This can further automate the time-consuming and arduous process of maintaining the deployed pipeline to ensure repeatability as data changes over time. With the dotData GUI, AI/ML development becomes a five-minute operation, requiring neither significant data science experience nor SQL/Python/R coding.
For more information or a demo of dotData’s AI-powered full-cycle data science automation platform, please visit dotData.com.
About JFE Steel Corporation
JFE Steel is a steelmaker engaged in the total steel-making process, taking iron ore raw material and turning it into final products. Boasting one of the world’s greatest capacities for steel production, JFE Steel satisfies customers by producing steel under a corporate philosophy of “contributing to society with the world’s most innovative technology.” The company also contributes to environmental protection by developing reduced-impact ironmaking processes and high-performance steel materials.
Official web site: https://www.jfe-steel.co.jp/en/company/about.html
About dotData
dotData Pioneered AutoML 2.0 to help business intelligence professionals add AI/ML models to their BI stacks and predictive analytics applications quickly and easily. Fortune 500 organizations around the world use dotData to accelerate their ML and AI development to drive higher business value. dotData’s automated data science platform accelerates ROI and lowers the total cost of model development by automating the entire data science process that is at the heart of AI/ML. dotData ingests raw business data and uses an AI-based engine to automatically discover meaningful patterns and build ML-ready feature tables from relational, transactional, temporal, geo-locational, and text data.
dotData has been recognized as a leader by Forrester in the 2019 New Wave for AutoML platforms. dotData has also been recognized as the “best machine learning platform” for 2019 by the AI breakthrough awards, was named an “emerging vendor to watch” by CRN in the big data space and was named to CB Insights’ Top 100 AI Startups in 2020. For more information, visit www.dotdata.com, and join the conversation on Twitter and LinkedIn.

Carl Bowen
Blog
July 7, 2020

Take Advanced Analytics into Overdrive with AutoML 2.0

The term “Advanced Analytics” was coined by the Gartner Group and is defined as the “…autonomous or semi-autonomous examination of data or content using sophisticated techniques and tools, typically beyond those of traditional business intelligence (BI), to discover deeper insights, make predictions, or generate recommendations.” Advanced analytics, by definition, requires the use of advanced techniques like data mining, machine learning, pattern matching, and other sophisticated manipulation of data in an effort to gain greater insights. The most broadly used category of advanced analytics is also known as predictive analytics. Predictive analytics itself is not new, but has traditionally been the exclusive domain of data scientists and highly skilled statisticians due to the extremely complex mathematical models required to effectively build predictive dashboards. While many organizations can benefit from predictive analytics, only a few are able to create and deploy dashboards powered by predictive algorithms, due to the high cost of hiring and retaining talent.

Why Advanced (Predictive) Analytics?

The benefits of advanced analytics and predictive analytics are relatively intuitive, given the typical use cases where predictive modeling can be beneficial. Customer churn, for example, is one of the most highly used and beneficial use-cases for predictive analytics. Predicting which customers are likely to churn can provide a business with an increased focus to be able to target those customers for upgrades and promotional offers, lowering churn rates. Similarly, predicting the likelihood of default on loans or outstanding payables can provide huge savings to organizations by limiting exposure to long-term collections. In marketing, predicting the likelihood of a campaign’s performance can have a massive impact on return on investment and can help marketing teams provide better focus for their efforts. Even as early as 2011, research firm The Aberdeen Group found that businesses using predictive analytics could identify the right target audience and make precise offerings to them at twice the rate of companies that were not using predictive analytics. The benefits of being able to predict business outcomes is tangible and of high value. The challenge, historically, has been that developing predictive analytics systems has been difficult, time-consuming and expensive.

The Traditional Workflow of Building Predictive Dashboards

When most people think of predictive analytics, the first thought that comes to mind is “expensive.” For most organizations, the challenge of predictive analytics is the cost involved in building out effective models that are delivered in business-friendly dashboards that can be used by line-of-business users. The reason achieving a well-formed predictive workflow is challenging is because of the steps involved in going from “data” to “predictive models.” Fundamentally, there are 5 steps involved in moving from just having “data” to using it to predict business outcomes:

Data Collection & Consolidation – Anyone who works in a large enterprise organization knows that enterprises love data. The problem, however, is that data lives in silos – separate systems for sales, marketing, operations, accounting etc. – sometimes sharing data – sometimes not. The first challenge of moving from data to predictions is that you need to take all that data and consolidate it into a unified analytics platform that can provide all relevant data for you to use and analyze.
Data Prep and Data Cleansing – Another major challenge in preparing data for predictive analytics is often referred to as normalization. Data normalization has two distinct phases – first, data must be unified and normalized across systems – this is typically a highly manual process that involves performing actions like ensuring that field values are consistent across systems and that duplicate data is removed from the final data warehouse. The second phase is preparing data for AI modeling – also often manual and designed to minimize errors in AI model development due to problems with the data.
Business Hypothesis Ideating, Testing & Validation – Once data is ready for analysis, the predictive analytics process requires what are commonly referred to as “features.” Features are nothing more than ways of using data to describe a potential useful outcome. Features will, in turn, be used in machine learning and AI models to derive predictions. Feature generation is often very time-consuming and manual and requires input from subject matter experts as well as data scientists and engineers in order to create, test and evaluate the usefulness of individual features.
ML Model Development and Testing – With features built, the next step in the process is to use features to test against multiple ML algorithms to test which ML models might provide better results. Again, this can be a very iterative and time-consuming process. In recent years, software tools known as “AutoML” have made the process of evaluating and testing ML models more automated.
Deploying Models into Production Environments – Once a model is built and has been tested, the next phase is to deploy the model into the ultimate production environment. For BI-based predictive applications, this might be a PowerBI dashboard or a Tableau dashboard that provides some form of predictive scoring based on user input (filters, drop-downs, etc.), allowing users to perform “what if” analysis on the business problems being predicted.

AutoML 2.0: Automating Your Workflow

The five steps outlined above are actually a fairly simplified version of what actually tends to happen. There are multiple steps where manual work involved requires multiple experts and multiple types of work. While AutoML 1.0 tools have allowed for the rapid development of Machine Learning models, they have still relied on prepared data. New platforms, however, are becoming available that can automate nearly the entire process – from AI-based data prep all the way to model deployment, allowing for the first time BI teams to develop, test and deploy predictive models without having to hire expensive data scientists. These AutoML 2.0 platforms are ideally suited for mid-sized organizations and smaller enterprises that can benefit from predictive analytics, but may not have the data science skills or staff to execute on traditional workflows.

Next Steps

So how do you get started? As a first step, it’s important to understand the core differences between AutoML and AutoML 2.0 platforms. Investing in the wrong type of product could create a nightmare scenario where additional staff and new software packages are required before you can create value from your advanced analytics infrastructure. Organizations serious about advanced analytics must leverage data science automation to gain greater agility and faster, more accurate decision-making. The emergence of AutoML platforms allows enterprises to be more nimble by allowing them to tap into current teams and resources without having to recruit and additional talent. AutoML 2.0 platforms empower BI developers and business analytics professionals to leverage AI/ML and add predictive analytics to their BI stack quickly and easily. AutoML 2.0 platforms not only generate features automatically, eliminating the most complex and time-consuming part of workflow, but also select the best algorithm depending upon the application. By providing automated data preprocessing, model generation, and deployment with a transparent workflow, AutoML 2.0 is bringing AI to masses thereby accelerating data science adoption.

Carl Bowen
Press Releases EN
April 22, 2020

dotData to Host Webinar “AutoML 2.0: From BI to BI+AI in 5 Minutes”

CEO Dr. Ryohei Fujimaki Hosts Live Webinar to Discuss how AutoML 2.0 Can Accelerate Predictive BI Work to Days, Not Months.

SAN MATEO, Calif., April 20, 2020 – dotData, focused on delivering full-cycle data science automation and operationalization for the enterprise, today announced that it will present a live webinar, “AutoML 2.0: From BI to BI+AI in 5 Minutes,” on Wednesday, April 29, 2020 at 2:00pm ET/11:00am PT. Those interested can register for the live webinar at https://bit.ly/AIandBIin5mins. The webinar will also be archived on the dotData site under Resources.

Adding AI, machine learning, or predictive capabilities to your BI stack has typically meant finding a data scientist or, even worse, learning about data science yourself. In the webinar, dotData’s CEO, Dr. Ryohei Fujimaki, will discuss the latest advancements in automated machine learning: autoML 2.0, and how it can help any business intelligence professional add AI and ML capabilities to their dashboards in record time.

dotData provides AutoML 2.0 solutions that help accelerate the process of developing AI and Machine Learning models for use in advanced predictive analytics BI dashboards and applications. dotData makes it easy for BI developers and data engineers to develop AI/ML models in just days by automating the full life-cycle of the data science process, from business raw data through feature engineering to implementation of ML in production utilizing its proprietary AI technologies. dotData’s AI-powered feature engineering automatically applies data transformation, cleansing, normalization, aggregation, and combination, and transforms hundreds of tables with complex relationships and billions of rows into a single feature table, automating the most manual data science projects that are fundamental to developing predictive analytics solutions.

dotData democratizes data science by enabling BI developers and data engineers to make enterprise data science scalable and sustainable. dotData automates up to 100 percent of the AI/ML development workflow, enabling users to connect directly to their enterprise data sources to discover and evaluate millions of features from complex table structures and huge data sets with minimal user input. dotData is also designed to operationalize AI/ML models by producing both feature and ML scoring pipelines in production, which IT teams can then immediately integrate with business workflows. This can further automate the time-consuming and arduous process of maintaining the deployed pipeline to ensure repeatability as data changes over time. With the dotData GUI, AI/ML development becomes a five-minute operation, requiring neither significant data science experience nor SQL/Python/R coding.

For more information or a demo of dotData’s AI-powered full-cycle data science automation platform, please visit dotData.com.

About dotData

dotData Pioneered AutoML 2.0 to help business intelligence professionals add AI/ML models to their BI stacks and predictive analytics applications quickly and easily. Fortune 500 organizations around the world use dotData to accelerate their ML and AI development to drive higher business value. dotData’s automated data science platform accelerates ROI and lowers the total cost of model development by automating the entire data science process that is at the heart of AI/ML. dotData ingests raw business data and uses an AI-based engine to automatically discover meaningful patterns and build ML-ready feature tables from relational, transactional, temporal, geo-locational, and text data.

dotData has been recognized as a leader by Forrester in the 2019 New Wave for AutoML platforms. dotData has also been recognized as the “best machine learning platform” for 2019 by the AI breakthrough awards, was named an “emerging vendor to watch” by CRN in the big data space and was named to CB Insights’ Top 100 AI Startups in 2020. For more information, visit www.dotdata.com, and join the conversation on Twitter and LinkedIn.

MEDIA CONTACT:

Jennifer Moritz
Zer0 to 5ive for dotData
jmoritz@0to5.com
917-748-4006

###

Carl Bowen
Events
April 13, 2020

AutoML 2.0 – From BI to BI+AI in 5 Minutes (Online Event)

Adding AI, Machine Learning, or Predictive Capabilities to your BI stack has typically meant finding a data scientist or, even worse, learning about data science yourself. Join dotData’s CEO, Dr. Ryohei Fujimaki, as he discusses the latest advancements in Automated Machine Learning, AutoML 2.0, and how it can help any business intelligence professional add AI and ML capabilities to their dashboards in record time.

Webinar Registration: AutoML 2.0 – From BI to BI+AI in 5 Minutes

Live Webinar Scheduled: April 29, 2020 @ 11:00am PT

Key Takeaways:

What is Augmented Analytics – and how can it help?
See how adding AI/ML to your BI stack is done today, and why it fails;
Learn how AutoML 2.0 can accelerate your predictive BI work to days,
not months.

Sachin Andhare
Blog
March 26, 2020

Shattering 5 Misconceptions about Automated Machine Learning

Ask data engineers about the most frustrating part of their job and the answer will most likely include “data preparation.” Talk to a data scientist about the AI/ML workflow and what bogs them down, the answer invariably will be feature engineering.

Analytics and data science leaders are well aware of the limitations of current AI/ML development platforms. They often lament about their team’s ability to only manage a few projects per year. BI leaders, on the other hand, have been trying to embed predictive analytics in their dashboards but face the daunting task of learning how to build AI/ML models. Automated machine learning (AutoML) was built specifically to address some of the challenges of data science – the underlying practice at the heart of both problems.

Like every new technology, there is a lot of confusion surrounding AutoML. Here are the top 5 misconceptions about AutoML:

1. AutoML means selecting the algorithms and building ML models automatically: In the early days of AutoML, the focus was on building and validating models. But the next generation AutoML 2.0 platforms include end-to-end automation and are able to do much more – from data preparation, feature engineering to building and deploying models in production. These new platforms are helping development teams reduce the time required to build and deploy ML models from months to days. AutoML 2.0 platforms address hundreds of use cases and dramatically accelerate enterprise AI initiatives by making AI/ML development accessible to BI developers and data engineers, while also accelerating the work of data scientists.
2. Feature Engineering (FE) implies selecting features once they are manually built: FE involves exploring features, generating and selecting the best features using relational, transactional, temporal, geo-locational or text data across multiple tables. Many traditional AutoML platforms require data science teams to generate features manually, a very time-consuming process that requires a lot of domain knowledge. AutoML 2.0 platforms provide AI-powered FE that enables any user to automatically build the right features, test hypotheses and iterate rapidly. FE automation solves the biggest pain point in data science.
3. Traditional AutoML platforms can ingest raw data from enterprise data sources to build ML pipelines: A typical enterprise data architecture includes master data preparation tools designed for data cleansing, formatting and standardization before the data is stored in data lakes and data marts for further analysis. This processed data requires further manipulation that is specific to AI/ML pipelines including additional table joining and further data prep and cleansing. Traditional AutoML platforms require data engineers to write SQL code and perform manual joins to complete these remaining tasks. AutoML 2.0 platforms, on the other hand, perform automatic data pre-processing to help with profiling, cleansing, missing value imputation and outlier filtering, and help discover complex relationships between tables creating a single flat-file format ready for ML consumption.
4. Model Accuracy is more important than feature transparency and explanation: This depends on the use-case and there needs to be a balance between accuracy and interpretability. Many ML platforms and data scientists create complex features that are based on non-linear mathematical transformations. These features, however, cannot be logically explained. Incorporating these types of features leads to a lack of trust and resistance from business stakeholders and, ultimately, project failure. In the case of heavily regulated industries such as financial services, insurance and healthcare, feature explainability is critical.
5. AutoML is not for BI teams and requires a data science background: First generation AutoML platforms were cumbersome, lacked user experiences for BI developers and provided challenging workflows. Even today many AutoML platforms are geared towards data scientists and require a strong ML background. AutoML 2.0 has unleashed a revolution by empowering citizen data scientists – BI analysts, data engineers and business users to embark on data science projects without requiring data scientists. AutoML 2.0 is the secret weapon the BI community can leverage to build powerful predictive analytics solutions in days – instead of the months typically associated with Augmented Analytics.

Learn more about dotData:

dotData Enterprise

Why dotData

Why AutoML 2.0