Digital Ink Recognition in ML Kit
Posted by Mircea Trăichioiu, Software Engineer, Handwriting Recognition
A month ago, we announced changes to ML Kit to make mobile development with machine learning even easier. Today we’re announcing the addition of the Digital Ink Recognition API on both Android and iOS to allow developers to create apps where stylus and touch act as first class inputs.
Digital ink recognition: the latest addition to ML Kit’s APIs
Digital Ink Recognition is different from the existing Vision and Natural Language APIs in ML Kit, as it takes neither text nor images as input. Instead, it looks at the user’s strokes on the screen and recognizes what they are writing or drawing. This is the same technology that powers handwriting recognition in Gboard – Google’s own keyboard app, which we described in detail in a 2019 blog post. It’s also the same underlying technology used in the Quick, Draw! and AutoDraw experiments.
Handwriting input in Gboard
Turning doodles into art with Autodraw
With the new Digital Ink Recognition API, developers can now use this technology in their apps as well, for everything from letting users input text and figures with a finger or stylus to transcribing handwritten notes to make them searchable; all in near real time and entirely on-device.
Supports many languages and character sets
Digital Ink Recognition supports 300+ languages and 25+ writing systems including all major Latin languages, as well as Chinese, Japanese, Korean, Arabic, Cyrillic, and more. Classifiers parse written text into a string of characters
Other classifiers can describe shapes, such as drawings and emojis, by the class to which they belong (circle, square, happy face, etc). We currently support an autodraw sketch recognizer, an emoji recognizer, and a basic shape recognizer.
Digital Ink Recognition API runs on-device and does not require a network connection. However, you must download one or more models before you can use a recognizer. Models are downloaded on demand and are around 20MB in size. Refer to the model download documentation for more information.
The time to perform a recognition call depends on the exact device and the size of the input stroke sequence. On a typical mobile device recognizing a line of text takes about 100 ms.
How to get started
If you would like to start using Digital Ink Recognition in your mobile app, head over to the documentation or check out the sample apps for Android and iOS to see the API in action. For questions or feedback, please reach out to us through one of our community channels.
Related Google News:
- The bots are here: Use RPA and AI to automate digital tasks March 16, 2021
- How do you eat an elephant? Google SREs talk digital transformation March 12, 2021
- ML Kit is now in GA & Introducing Selfie Segmentation March 9, 2021
- Google Cloud Born-Digital Summit: Inspiring the next generation of technology leaders March 2, 2021
- Hear how digital businesses grew in our new podcast, PubCast March 2, 2021
- Digital skills training for 100,000 Black women February 12, 2021
- TELUS and Google Form Strategic Alliance to Bring Digital Transformation to Key Industries,… February 9, 2021
- The time for digital excellence is here—Introducing Apigee X February 4, 2021