Visual AI
Elementary Robotics AI is the machine learning platform for web app. Use the model development to train custom inspections for QA.
My Role
Lead UX/UI designer for creating the flow of no code machine learning platform with ML team, Web team, CTO, and CEO.
Research, Prototype, User testing, wireframe, info architecture, UI design,
Results
Shipped the web app with a few main functionalities:
1. Configure the ML model.
2. Label and train the model.
3. Deploy and Evaluate the model.
Handed users the tool to create and train their own machine-learning models.
Provide a no-code experience for a broader audience.
What Is It
Elementary’s easy-to-use software, deep learning AI, and camera systems are built to capture visual data, deliver fast and reliable real-time judgments.
This No-code machine learning platform lets the user capture their inspection routines and label, train, deploy their customized deep learning models.
User Types
Inspection managers or leads are the main types of user who interacts with the no-code machine learning platform.
Manager
Their goal is to create a customized inspection routine for each object and train the ML model to catch defects.
After the models are deployed to different stations, they need to supervise each station from a distance.
User Flow
1
Capture Data
Configure
Data
2
Label & Train ML Model
3
Deploy
ML Model
4
5
Evaluate
ML Model
1. Capture Data
- Create inspection routines and images for various objects with different hardware platforms.
- Use the inspection routine that was created to collect images from a set of objects to provide enough data to label and train the model.
Challenge
Users used to only care about which area contains what defect. So they preferred to see results get organized by areas of interest. But they will assign multiple machine learning models to an area of interest. For example, they might look for a missing cap and some printed text in the same area. Each ML model also has different labeling requirements. They also get trained individually. How do we design the info architecture to meet both needs?
Solution
We guided our user to create and label multiple inspections on one area of interest during the input phase.
But we delivered inspection results on the area of interest bases.
2. Configure Data
- Create bounding boxes around areas of interest on each position in a routine.
- Classify areas of interest according to the type of defects users are looking for. e.g. Barcode, Text, etc.
- Create alignment fixtures to tell the machine which orientation to read the image.
- We named the end result of this step as Inspections.
This is how users add an inspection to an area of interest.
Challenge
Our customers will configure the threshold to decide how strict they want the AI model to perform. They need some product data to support in making that decision. A ROC chart is used in the machine learning world to show false positives and true positives. But it is a hard concept for our users to understand.
Solution
We converted the ROC value into defective shipped and excess scrap which made the data relatable for our user's production line. When the threshold value goes out of range, it would cause the production line also showed the suggested range for the threshold with a green bg.
3. Label & Train ML Model
3. Label & Train ML Model
3. Label & Train ML Model
- Label each tool with the image set that was captured in step 1. (Each label will contain results of Pass, Fail or Unknown. This is an essential step for our users to teach ML model how to find the right defects.)
- Once a certain number of image sets are labeled, users can hit the “train” button.
- Once the model finishes training, an entire inspection routine is created and ready to go.
Challenge
We found out that the labeling process is focus intense. User often asks how many more data do they need to label. We want to respect the focus that and guide them through the process with less confusion.
Solution
We decided to break down label requirements into 4 small labeling goals. In each goal, we provide a metrics panel to indicate how many labels needed clearly.
4,5. Deploy & Evaluate
- Deploy different versions of the ML model to inspection stations.
- Users will be able to monitor each inspection routine and see their performance on the production line.
And we launched! Check out the Elementary Robotics for more info about the product.
