Zapier makes it doable to automate duties with out having to jot down any code. And now, it has the present of sight. With the Roboflow Zapier integration you cross a picture to any Roboflow mannequin and get the predictions again to make use of downstream in your Zap.
On this weblog publish, we are going to present the way to join your Roboflow account to your Zapier account. We may also stroll by the way to use a Roboflow classification mannequin to automate an inbound e mail circulation. Let’s start!
Mission Overview
For this tutorial, we’ll put ourselves within the footwear of a big auto mechanic store that receives many questions by way of e mail about varied automobiles. The entire inquiries come to the identical e mail and step one that should occur is to route every inquiry to the suitable division.
For this instance we’ll have three departments:
- Business Automobiles
- Massive Shopper Automobiles
- Small Shopper Automobiles
For this process, we’ll use a classification mannequin from Roboflow Universe to route emails. The mannequin we are going to use takes a picture as enter and returns a rating for every class it is aware of about. Every rating represents how possible it’s that the picture comprises the goal class; the upper the rating, the extra possible a picture comprises an object.
The courses on which the mannequin is skilled are:
- Ambulance
- Bus
- Automotive
- Limousine
- Bike
- Segway
- Taxi
- Truck
- Van
Word: There are different courses within the dataset, however we’ll ignore them for this tutorial.
Utilizing Roboflow and Zapier we’ll arrange an automatic circulation that listens for brand spanking new emails to a specific inbox, passes the attachment from the e-mail to our Roboflow classification mannequin, then routes the e-mail to a brand new tackle based mostly on the classification outcomes.
The Roboflow + Zapier Workflow
Our workflow will take the next steps:
- Pay attention for an e mail with an attachment;
- Filter based mostly on sender e mail tackle (ignore for now);
- Save the attachment to Google Drive;
- Get a public hyperlink for the Google Drive;
- Go the general public hyperlink to the Roboflow classification mannequin;
- Proceed if the classification mannequin is assured it is aware of what’s within the picture and;
- Ahead the e-mail to the suitable division based mostly on the classification end result.
Listed below are the forwarding guidelines we need to implement:
- Predicted class is considered one of Ambulance, Bus, Limousine, Taxi, Truck → Ship to the business car division.
- Predicted class is considered one of Automotive, Van → Ship to massive shopper car division.
- Predicted class is considered one of Bike, Segway → Ship to small shopper car division.
Let’s implement this workflow with Zapier!
Stipulations
Find Your Roboflow API Key
To make use of the Roboflow Zapier integration, you’ll want a Roboflow API key. First create a Roboflow account, then comply with our directions on the way to retrieve you API key.
Classification Mannequin
To create our Zap – the time period used for a workflow in Zapier – we first want to select a Roboflow mannequin to make use of. For this, you’ll be able to deliver your personal knowledge and prepare a customized mannequin on Roboflow, or you’ll be able to browse Roboflow Universe for a public mannequin.
For this tutorial, we’re utilizing the general public Car Classification mannequin from Roboflow Universe.
Google Drive Folder
Once you obtain an e mail attachment, that attachment is saved in a safe location. We have to take the attachment and put it aside in a location that our Roboflow mannequin has entry to in order that it could actually course of the picture. For this, we’ll create a brand new folder on Google drive. As soon as created, we’ll must set the permissions to “Anybody with the hyperlink…” → “Viewer”.
This modification will be certain that anybody with a hyperlink to the information in that folder will be capable to entry it with out credentials, specifically our Roboflow mannequin.
Electronic mail Addresses
If this instance had been for a manufacturing utility, we would want an inbox for receiving inquiries, and a number of other different inboxes that we’d ahead these inquiries to based mostly on the division.
For this instance we benefit from a characteristic of Gmail e mail addresses. With Gmail e mail addresses, you’ll be able to create an arbitrary variety of e mail addresses by appending any string utilizing the “+” image. So, in case your e mail is “paul@roboflow.com”, it’s also possible to use “paul+test@roboflow.com”. Any emails despatched to “paul+test@roboflow.com” will present up in the identical inbox as “paul@roboflow.com”. For this tutorial, we’ll use the next setup:
Create the Roboflow Zap
Now we’re able to create our Zap that makes use of the Roboflow Zapier integration to implement the aforementioned logic. In your Zapier account, choose Create > New Zap.
Step #1: Set Up the Gmail Integration
You’ll be requested so as to add the set off (the primary block in any Zap):
Choose the “Gmail” integration. If it’s not proven by default, use the search bar to seek out it.
Then select the occasion “New Attachment”. Subsequent, choose your Gmail account (or join one should you haven’t finished so already). Depart all fields on the “Set off” part clean. Then transfer on to the “Take a look at” part.
It will likely be useful to have instance e mail to work with for establishing the remainder of this Zap, so earlier than you check this set off, ship an e mail to your chosen Gmail tackle with an attachment of a picture of a motorbike. Any picture will work, however right here’s one from our Roboflow dataset you should utilize:
After having obtained the e-mail with the motorbike picture attachment, click on the “Take a look at Set off” button. You will have a number of attachments to select from (labeled “Attachment A, Attachment B, and so on.). Proceed deciding on completely different attachments till you discover the one with the motorbike picture. Word that you just gained’t be capable to see the picture so that you’ll have to pick it based mostly on the textual content within the e mail or the timestamp of when it was obtained.
Subsequent, add a Google Drive block to your Zap and select the occasion “Add File”. Within the step particulars select the drive and folder you created as a part of the stipulations. Set the “File” subject to the “1. Attachment: (Exists however not proven)” subject from the Gmail block. Set “Convert to Doc?” to false. Set “File Identify” to “1. Attachments Filename: motorbike.jpeg”. Depart the “File Extension” and all different elective fields clean.
If we stopped right here, our Zap would solely add attachments we obtain to our Google Drive folder. However we need to run inference with our picture utilizing Roboflow! Let’s construct that half now.
Now we have to get a hyperlink to our file on Google Drive.
Step #2: Retrieve the URL for an Picture
For this, we’ll use one other Google Drive block with the motion “Discover a File”. Set the “File Identify” subject to the identical we utilized in our earlier block, “1. Attachments Filename: motorbike.jpeg”. Equally set the drive and folder fields. For “File Varieties” select “photos/”. Set the sphere “Ought to this step be thought-about a ‘success’ when nothing is discovered?” to False. And depart “Create a Google Drive File if it doesn’t exist but?” unchecked.
Step #3: Go the Picture to Roboflow
Add a brand new block and seek for Roboflow. Choose the motion “Infer Classification”. Join your Roboflow account utilizing the API key you discovered through the prerequisite steps of this tutorial. For the “Dataset” subject, use “vehicle-classification-eapcd”. That’s the dataset slug of the car classification mannequin we need to use. If you wish to use your personal customized mannequin, you’ll must find your dataset’s slug. For the “Model” subject, use “1”. Depart confidence as “0.5”.
For the “Picture URL” subject, use “4. Net Content material Hyperlink: https://drive.g…export=download”. That is the publicly accessible hyperlink to the file we saved in Google Drive.
After we check this block, we’ll see actual classification outcomes from our Roboflow car classification mannequin. You’ll be able to learn extra in regards to the JSON output in our hosted API documentation. For our functions, we’ll use the “High” subject and the “Confidence” subject from the output. The “High” subject is the category title that’s almost certainly to explain the picture.
The “Confidence” is a quantity between Zero and 1 that describes how assured the mannequin was when predicting that class title. For instance, if the “High” subject was “Bike” and the “Confidence” was 0.85, which means the classification mannequin was 85% assured that the picture is of a motorbike. Learn extra about mannequin confidence and mannequin metrics right here.
Subsequent, we solely need to proceed if the classification mannequin was assured about what it noticed. Usually, we use the worth 0.5 as a threshold. This implies the mannequin should return a confidence price equal to or larger than 0.5 to return a prediction.
Create a brand new block and select the Zapier “Solely proceed if…” block. Set the primary subject to “5. Confidence: 0.9341”. Set the second subject to “is bigger than”. And set the third subject to “0.5”.
Lastly, we have to route our e mail to the right division. There are a pair choices for the way to do that. On this tutorial we are going to use the Zapier “Paths” motion. This lets us arrange a number of paths, every of which apply a filter, then take an motion if the filter situations are met. This block, nonetheless, is just out there on increased paid tiers.
To keep away from utilizing this block, you’ll be able to merely use one other “Solely proceed if…” block, filter based mostly on the classification end result, then ship an e mail to the division that matches the parameters of the filter block. Should you do that, you’ll must arrange a separate Zap for every division since every filter block shall be arrange in another way.
Inside our “Paths” motion, we’ll arrange three paths. Every path will include two blocks, a “Path guidelines” block (which capabilities as a “Solely proceed if…” block), and a Gmail block.
The primary path we’ll setup is the “Business Car” path. For the “Path guidelines” block set the primary subject to “5. High: Bike”. For the second subject, choose “is in”. For the third subject, sort “Bus, Truck, Ambulance, Limousine, Taxi”. This tells the filter to solely proceed if the expected class from the classification mannequin output is without doubt one of the ones we simply typed.
Subsequent, choose a Gmail block with the motion “Ship Electronic mail”. For the “To” subject sort your e mail and add “+business” earlier than the “@” image. For the topic subject, sort “Inquiry Business Car: 5. High: Bike”. It will insert the title of the expected class into the topic of our forwarded e mail. For the “Physique Sort”, choose HTML and for the “Physique” choose “1. Physique Html: … “. For “Attachments” choose “1. Attachment: (Exists however not proven)”.
We’ll repeat this course of to create two extra paths with some completely different parameters:
- Massive Shopper Automobiles
- “5. High: Bike” is in “Automotive, Van”
- To: “+consumerlarge”
- Topic: “Inquiry Shopper Car Massive: 5. High: Bike”
- Small Shopper Automobiles
- “5. High: Bike” is in “Bike, Segway”
- To: “+consumersmall”
- Topic: “Inquiry Shopper Car Small: 5. High: Bike”
Lastly, we now have one remaining (elective) step. Recall from the overview, step two stated “ignore for now”. Since we’re not truly utilizing completely different inboxes for our instance, we now have simply arrange for ourselves an infinite loop. Our Zap will obtain an e mail, classify it, then ship it again to our inbox. Then it should see the e-mail it simply obtained, classify it once more, and ship it again to our e mail once more. And it’ll proceed doing so till the computer systems lastly get good sufficient to cease taking orders from us.
To keep away from this (the infinite loop, that’s), we are able to add an “Solely proceed if…” block after our set off (first Gmail block). We’ll set the fields to “1. To Emails: …” precisely matches “youremail@youremail.com”. This manner, when the Zap sees an e mail that was despatched to “youremail+commercial@youremail.com” it should cease and gained’t proceed with our automated circulation.
Take a look at the Zap
Now we’re able to publish and check our Zap! To try this, merely click on the publish button, then ship an e mail with an image attachment to the e-mail tackle you used. For my check, I despatched a picture of a Taxi. After my Zap ran, I received this e mail:
We will see every thing labored simply as deliberate! From the topic line, we are able to see Roboflow acknowledged that the image was of a taxi. Then, utilizing the end result from the Roboflow mannequin we had been capable of route the e-mail to my “+business” tackle with the unique physique and attachment included.
Limitless Prospects
This instance showcased simply considered one of seemingly countless prospects. We will’t wait to see what you do with the Roboflow Zapier integration which incorporates blocks for Classification inference, Object Detection inference, Occasion Segmentation inference, and CLIP comparability. We’ll be creating Zapier templates for every of those blocks within the close to future. Don’t overlook to publish on talk about.roboflow.com with any questions. Completely satisfied Zapping!
