{"id":20617,"date":"2021-05-28T14:10:28","date_gmt":"2021-05-28T13:10:28","guid":{"rendered":"https:\/\/www.inovex.de\/blog\/?p=20617"},"modified":"2023-05-08T09:23:05","modified_gmt":"2023-05-08T07:23:05","slug":"deep-learning-for-mobile-devices-with-tensorflow-lite-ai-powered-android-application","status":"publish","type":"post","link":"https:\/\/www.inovex.de\/de\/blog\/deep-learning-for-mobile-devices-with-tensorflow-lite-ai-powered-android-application\/","title":{"rendered":"Deep Learning for Mobile Devices with TensorFlow Lite: Create an AI-Powered Android Application"},"content":{"rendered":"<p>This is the third article of our blog post series about Deep Learning for mobile devices. The <a href=\"https:\/\/www.inovex.de\/blog\/tensorflow-lite-concepts-architectures\/\">first post<\/a> tackled some of the theoretical background of on-device machine learning, including quantization and state-of-the-art model architectures. The <a href=\"https:\/\/www.inovex.de\/blog\/deep-learning-mobile-tensorflow-lite\/\">second<\/a> explored how to do quantization-aware model training with the TensorFlow Object Detection API. In this article, we will describe how to convert a model to TensorFlow Lite and how to build an AI-powered mobile app by using the model in an Android application.<!--more--><\/p>\n<p>As we\u2019re going to be working with the model we trained on part 2 of this series, I would recommend you to <a href=\"https:\/\/www.inovex.de\/blog\/deep-learning-mobile-tensorflow-lite\/\">start there<\/a> and come back afterwards, if you haven\u2019t read that yet and you\u2019re interested in reproducing our use-case. However, if you have a different use-case in mind or just want to find out how to create awesome apps enhanced with machine learning, you&#8217;re very welcome to just keep reading as the steps described here will also apply.<\/p>\n<p>The use-case we\u2019re building is an app that \u2013 thanks to an object detector trained on the <a href=\"https:\/\/www.tensorflow.org\/datasets\/catalog\/cars196\">cars196 dataset<\/a> \u2013\u00a0 is able to continuously identify cars seen by the phone\u2019s back camera in real time and display the most probable classifications as well as where they are situated on the screen. Cool, right? No more wondering what make and model those cars on the streets are. Let\u2019s see how to do that!<\/p>\n<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_83 counter-hierarchy ez-toc-counter ez-toc-custom ez-toc-container-direction\">\n<div class=\"ez-toc-title-container\"><p class=\"ez-toc-title\" style=\"cursor:inherit\"><\/p>\n<\/div><nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/www.inovex.de\/de\/blog\/deep-learning-for-mobile-devices-with-tensorflow-lite-ai-powered-android-application\/#Converting-the-model-to-TensorFlow-Lite\" >Converting the model to TensorFlow Lite<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/www.inovex.de\/de\/blog\/deep-learning-for-mobile-devices-with-tensorflow-lite-ai-powered-android-application\/#Model-Metadata\" >Model Metadata<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/www.inovex.de\/de\/blog\/deep-learning-for-mobile-devices-with-tensorflow-lite-ai-powered-android-application\/#Adding-metadata-to-the-model\" >Adding metadata to the model<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/www.inovex.de\/de\/blog\/deep-learning-for-mobile-devices-with-tensorflow-lite-ai-powered-android-application\/#Importing-the-model\" >Importing the model<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/www.inovex.de\/de\/blog\/deep-learning-for-mobile-devices-with-tensorflow-lite-ai-powered-android-application\/#Structure-of-the-application\" >Structure of the application<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-6\" href=\"https:\/\/www.inovex.de\/de\/blog\/deep-learning-for-mobile-devices-with-tensorflow-lite-ai-powered-android-application\/#Gathering-the-data\" >Gathering the data<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-7\" href=\"https:\/\/www.inovex.de\/de\/blog\/deep-learning-for-mobile-devices-with-tensorflow-lite-ai-powered-android-application\/#Transforming-the-data\" >Transforming the data<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-8\" href=\"https:\/\/www.inovex.de\/de\/blog\/deep-learning-for-mobile-devices-with-tensorflow-lite-ai-powered-android-application\/#Loading-the-model\" >Loading the model<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-9\" href=\"https:\/\/www.inovex.de\/de\/blog\/deep-learning-for-mobile-devices-with-tensorflow-lite-ai-powered-android-application\/#Running-the-inference\" >Running the inference<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-10\" href=\"https:\/\/www.inovex.de\/de\/blog\/deep-learning-for-mobile-devices-with-tensorflow-lite-ai-powered-android-application\/#Interpreting-the-results\" >Interpreting the results<\/a><ul class='ez-toc-list-level-4' ><li class='ez-toc-heading-level-4'><a class=\"ez-toc-link ez-toc-heading-11\" href=\"https:\/\/www.inovex.de\/de\/blog\/deep-learning-for-mobile-devices-with-tensorflow-lite-ai-powered-android-application\/#Number-of-boxes-and-detection-scores\" >Number of boxes and detection scores<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-4'><a class=\"ez-toc-link ez-toc-heading-12\" href=\"https:\/\/www.inovex.de\/de\/blog\/deep-learning-for-mobile-devices-with-tensorflow-lite-ai-powered-android-application\/#Categories\" >Categories<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-4'><a class=\"ez-toc-link ez-toc-heading-13\" href=\"https:\/\/www.inovex.de\/de\/blog\/deep-learning-for-mobile-devices-with-tensorflow-lite-ai-powered-android-application\/#Bounding-Boxes\" >Bounding Boxes<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-14\" href=\"https:\/\/www.inovex.de\/de\/blog\/deep-learning-for-mobile-devices-with-tensorflow-lite-ai-powered-android-application\/#Displaying-the-results\" >Displaying the results<\/a><\/li><\/ul><\/li><\/ul><\/nav><\/div>\n<h2><span class=\"ez-toc-section\" id=\"Converting-the-model-to-TensorFlow-Lite\"><\/span>Converting the model to TensorFlow Lite<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>In order to develop an AI-powered app, we decided to go for TensorFlow Lite, which allows the inference to happen on-device. Thus, it removes latency, privacy, connectivity as well as power consumption issues, as all computation happens on the Android device itself. It also allows enhancing Android apps with machine learning models in an accessible way.<\/p>\n<p>First, we need to convert the TensorFlow model we trained to TensorFlow Lite format (an optimized FlatBuffer format identified by the .tflite file extension). To start with a regular TensorFlow model and then convert it is always the way to go, since one cannot directly create or train a model using TensorFlow Lite.<\/p>\n<p>To do that, we can use the TensorFlow Lite Converter in one of the following two ways:<\/p>\n<ol style=\"list-style-type: lower-alpha;\">\n<li>Using the <a href=\"https:\/\/www.tensorflow.org\/lite\/convert\/index#python_api\">Python API<\/a><\/li>\n<li>Via the <a href=\"https:\/\/www.tensorflow.org\/lite\/convert\/index#cmdline\">Command line<\/a><\/li>\n<\/ol>\n<p>The recommended way is using the Python API. It offers more features and makes it easier to convert models as part of the model development pipeline and apply optimizations, such as post training quantization and adding metadata. Therefore, we make use of the API in a short python script like this:<\/p>\n<pre class=\"lang:python decode:true\">import tensorflow as tf\r\n# Convert the model\r\nconverter = tf.lite.TFLiteConverter.from_saved_model(saved_model_dir)\u00a0\r\ntflite_model = converter.convert()\r\n# Save the model.\r\nwith open('model.tflite', 'wb') as f:\r\n\u00a0\u00a0f.write(tflite_model)<\/pre>\n<p>It\u2019s as simple as that. We create a converter, convert the saved model and save it as a file with the extension .tflite. Converting a keras model is also possible using the converter.<\/p>\n<p>Although TensorFlow Lite plans to provide high performance on-device inference for any TensorFlow model, the interpreter currently supports only a limited subset of operators that have been optimized for on-device use. Keep that in mind if you\u2019re embedding a custom model other than the one we\u2019ve trained in this series, one the pre-trained models the TFLite team offers <a href=\"https:\/\/www.tensorflow.org\/lite\/models\">here<\/a> or one from the <a href=\"https:\/\/www.tensorflow.org\/hub\">Hub<\/a>. Some models will require additional steps to work with TensorFlow Lite. You can check which operators are available here: <a href=\"https:\/\/www.tensorflow.org\/lite\/guide\/ops_compatibility\">Operator compatibility<\/a>.<\/p>\n<p>The model we converted has the following input and four outputs:<\/p>\n<ul>\n<li><strong>input<\/strong>: Float32 Tensor of shape [1, 640, 640, 3]<\/li>\n<li><strong>detection_boxes<\/strong>: a float32 tensor of shape [1, num_boxes, 4] with box locations<\/li>\n<li><strong>detection_classes<\/strong>: a float32 tensor of shape [1, num_boxes] with class indices<\/li>\n<li><strong>detection_scores<\/strong>: a float32 tensor of shape [1, num_boxes] with class scores<\/li>\n<li><strong>num_boxes<\/strong>: a float32 tensor of size 1 containing the number of detected boxes<\/li>\n<\/ul>\n<p>This is important to know, since we will be working with them in our application.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Model-Metadata\"><\/span>Model Metadata<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>The APIs are developed with general methods rather than specific so they can be used for all kinds of model tasks. Therefore, metadata, a source of knowledge about what the model does and information about its inputs and outputs is required to adapt the code to a specific model.<\/p>\n<p>TensorFlow Lite metadata provides a standard for model descriptions. The metadata consists of both human-readable parts, which convey the best practice when using the model, and machine- readable parts that are leveraged by code generators, such as the TensorFlow Lite Android code generator and the Android Studio ML Binding feature. In fact, it is mandatory to be able to use the TensorFlow Lite Android code generator, as well as the ObjectDetector API from the Task Library.<\/p>\n<h3><span class=\"ez-toc-section\" id=\"Adding-metadata-to-the-model\"><\/span>Adding metadata to the model<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>In order to be able to add metadata to your model, you will need a Python programming environment setup for running TensorFlow. There is a detailed guide on how to set this up <a href=\"https:\/\/www.tensorflow.org\/install\">here<\/a>.<br \/>\nAfter setting up the Python programming environment, install the <em>tflite-support<\/em>\u00a0toolkit:<\/p>\n<pre class=\"lang:default decode:true \">pip install tflite-support\r\n<\/pre>\n<p>There are three parts that need to be present in the model metadata:<\/p>\n<ol>\n<li>Model information \u2013 Overall description of the model as well as items such as license terms.<\/li>\n<li>Input information \u2013 Description of the inputs and pre-processing required such as normalization.<\/li>\n<li>Output information \u2013 Description of the output and post-processing required such as mapping to labels.<\/li>\n<\/ol>\n<p>TensorFlow Lite metadata for inputs and outputs are not designed for specific model types but rather input and output types, which must consist of the following or a combination of the following:<\/p>\n<ul>\n<li>Feature &#8211; Unsigned integer or float32.<\/li>\n<li>Image &#8211; Either an RGB or grayscale image.<\/li>\n<li>Bounding box &#8211; Rectangular shape bounding boxes. The schema supports several numbering schemes from which we will be using the <em>Boundaries<\/em> type. It represents the bounding box by a combination of boundaries in the form: {left, top, right, bottom}.<\/li>\n<\/ul>\n<p>The associated files also need to be included. In this case, the <a href=\"https:\/\/gist.github.com\/SilviaSantano\/391a657001381669549a992693fe9a81\">labelmap.txt<\/a> file, which contains the labels.<\/p>\n<p>As this feature is relatively new, the official website only provides an example for populating metadata into Image Classification models. <a href=\"https:\/\/gist.github.com\/SilviaSantano\/7bda68f3f867081be72c968b9751fa8b\">Here<\/a> you can find the metadata writer Python script we used for our Object Detector, where we describe all information about the input and outputs and include the associated file.<\/p>\n<p>In order to execute it, you need to pass the model, the labels file and the export directory as follows:<\/p>\n<pre class=\"lang:default decode:true\">python metadata_writer_for_object_detector.py --model_file=detect.tflite --label_file=labelmap.txt --export_directory=export<\/pre>\n<p>At the moment of writing this post, a new library, the <a href=\"https:\/\/github.com\/tensorflow\/tflite-support\/tree\/master\/tensorflow_lite_support\/metadata\/python\/metadata_writers\">Metadata Writer library<\/a> is under development for image classifiers and object detectors as part of the <em>tflite-support<\/em> library and aims at simplifying the process by means of a wrapper class. This will make the task much easier for the developer since she will not need to code all of the above.<\/p>\n<p>Inferencing can be as easy as just a few lines of code after metadata has been successfully added, since it contains a rich description of how to use it.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Importing-the-model\"><\/span>Importing the model<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>To import the newly converted model in an Android Project in Android Studio we will use the new Android Studio ML Model Binding and import the TensorFlow Lite model through the graphical interface. Open your project or create a new one and import it by clicking on File, then New &gt; Other &gt; TensorFlow Lite Model. After selecting the location of your TFLite file, the tooling will automatically configure the module&#8217;s dependency with ML Model binding and all dependencies will be inserted into the Android module&#8217;s build.gradle file.<\/p>\n<p>After the model was successfully imported, you will see information about the model, such as the description and the shape of its inputs and outputs. Now that our model contains metadata, sample code to execute the inference that we can simply copy\/paste will additionally be provided there. For a model that doesn\u2019t yet contain metadata, you would see minimal information and standard sample code using TensorBuffers for the inputs and the outputs. Always in both kotlin and java (for those who still have good reasons to use java in their Android apps \ud83d\ude09 ). You can see this info again anytime by opening the model file in Android Studio.<\/p>\n<p><em>Note: the ML Model Binding is a new component and it requires Android Studio version 4.1 or above.<\/em><\/p>\n<p>If you\u2019re using the TensorFlow Lite Task Library or the Interpreter API for the inference (more on that later), the way to import it is by including it in your Android project under the <em>assets<\/em> folder.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Structure-of-the-application\"><\/span>Structure of the application<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Our application is composed of an Activity, a ViewModel, where the recognition data will be stored as a LiveData list object, and a data model for the recognition item objects. Recognition item objects have fields for the label, the probability and the location of the bounding box. Additionally, we will also work with two helper classes for displaying the labels and the bounding boxes. The layout of the activity holds a PreviewView for the camera preview and an ImageView over it on which we will draw the results. The Activity will update the views whenever there is new data.<\/p>\n<p>In this article, we will just cover the most relevant parts of the code but, if you\u2019re interested, you can find the full code of the application <a href=\"https:\/\/github.com\/SilviaSantano\/RecognizeCarsWithTensorFlowLite\">here<\/a>. Please keep in mind that all TensorFlow Lite libraries are still at a very early stage and are therefore subject to change. The code represents the current state as of the publication of this article.<\/p>\n<p>These are the steps to follow to run the model and display meaningful information about the camera images for the user:<\/p>\n<ol>\n<li><strong>Gather the data<\/strong>: capture the camera stream and pass the frames (we only keep the last image) to the analyzer, the function that performs the inference.<\/li>\n<li><strong>Transform the data<\/strong>: since raw input data generally will not match the input data format expected by the model, some adjustments need to be made. For example, you might need to resize, crop or rotate an image or change the image format for it to be compatible with the input the model expects.<\/li>\n<li><strong>Load the model<\/strong>: load the .tflite model into memory, which contains the model&#8217;s execution graph.<\/li>\n<li><strong>Run the inference<\/strong>: use the TensorFlow Lite APIs to execute the model using the input and extract results of the prediction in form of outputs.<\/li>\n<li><strong>Interpret the results<\/strong>: extract meaningful information from the results that is relevant to the application and apply transformations in case needed.<\/li>\n<li><strong>Display the results<\/strong>: present the acquired information to the user.<\/li>\n<\/ol>\n<h3><span class=\"ez-toc-section\" id=\"Gathering-the-data\"><\/span>Gathering the data<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>Although we could use static images, we want our app to be able to recognize cars directly from the camera stream as we walk by them. Currently, the easiest way to capture the camera stream is to use the CameraX library. It is part of the Jetpack support library and offers an easy-to-use interface that makes using the camera much easier as it was with camera2. Another great advantage is its lifecycle awareness.<\/p>\n<p>We will be using both the <em>preview<\/em> use-case, to display a preview on the screen, and the <em>image analysis<\/em> use-case, to extract the image buffers and perform the analysis. We build and attach both of them to the lifecycle of the activity. Afterwards, for the preview we just attach the output of the preview object to the PreviewView object from our layout. As for the image analysis, each image is provided to the <em>analyze<\/em> method of an implementation of a class inheriting from <em>ImageAnalysis.Analyzer<\/em>, where it can access image data via an ImageProxy.<\/p>\n<h3><span class=\"ez-toc-section\" id=\"Transforming-the-data\"><\/span>Transforming the data<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>We will avail ourselves of the TensorFlow Lite Android Support Library for transforming the data. It provides high-level APIs that help us transform raw input data into the form required by the model, and interpret the model&#8217;s output, and thus, reduces the amount of boilerplate code required. It supports common data formats for inputs and outputs, including images and arrays. It also provides pre- and post-processing units that perform tasks such as image resizing, rotating and cropping.<br \/>\nIn our case, the size of the image used by the model is 640 x 640 pixels, as mentioned earlier.<\/p>\n<p>Usually, the image captured will be larger than that. It\u2019s likely that it will be something like 1920 x 1080 pixels with new devices. That\u2019s why we need to resize. The question is, how to get a non-square image into a square? There are several ways to do that. We will go for centerCrop. In this operation, only the center will be kept and the surroundings outside of this box will be discarded. For that reason, our app will work best when the cars are as much centered in the image as possible as having objects too close to the edges will unfortunately make them be out of the image.<\/p>\n<p>The next step is rotation. We need to take into account what the current rotation of the device is in order to know if and how much it needs to be rotated to pass it in the expected orientation to the model. When it is used in portrait mode, for example, it will need to be rotated three times (to the right) whereas in landscape mode, it will depend on the direction.<\/p>\n<p>To convert the input from the back camera to a bitmap using the following kotlin extension (the image format used may vary among different devices):<\/p>\n<pre class=\"lang:default decode:true\">fun Image.toBitmap(): Bitmap {\r\n    val yBuffer = planes[0].buffer\r\n    val vuBuffer = planes[2].buffer\r\n    val ySize = yBuffer.remaining()\r\n    val vuSize = vuBuffer.remaining()\r\n    val nv21 = ByteArray(ySize + vuSize)\r\n    yBuffer.get(nv21, 0, ySize)\r\n    vuBuffer.get(nv21, ySize, vuSize)\r\n    val yuvImage = YuvImage(nv21, ImageFormat.NV21, this.width, this.height, null)\r\n    val out = ByteArrayOutputStream()\r\n    yuvImage.compressToJpeg(Rect(0, 0, yuvImage.width, yuvImage.height), 50, out)\r\n    val imageBytes = out.toByteArray()\r\n    return BitmapFactory.decodeByteArray(imageBytes, 0, imageBytes.size)\r\n}<\/pre>\n<p>This is the code to create an ImageProcessor and process the bitmap we converted with the operations we described to feed a TensorImage:<\/p>\n<pre class=\"lang:default decode:true\">\/\/ Create an image processor\r\nval imageProcessor = ImageProcessor.Builder()\r\n    \/\/ Center crop the image\r\n    .add(ResizeWithCropOrPadOp(HEIGHT, WIDTH))\r\n    \/\/ Rotate\r\n    .add(Rot90Op(calculateNecessaryRotation()))\r\n    .build()\r\nvar tImage = TensorImage(DataType.UINT8)\r\ntImage.load(imageProxy.image!!.toBitmap())\r\ntImage = imageProcessor.process(tImage)<\/pre>\n<p>A tip always useful for troubleshooting is to check the final image that the model is receiving to ensure the operations applied were correct and the result is indeed what we expect it to be, as this has a lot of potential to be the reason why the model returns wrong results.<\/p>\n<p>Optionally: we extract the dominant color from the image using the Palette API to compute what the most suitable color for displaying text over it would be to use it later.<\/p>\n<pre class=\"\">val palette = Palette.from(bitmap).generate()\r\nval color = palette.dominantSwatch?.bodyTextColor<\/pre>\n<h3><span class=\"ez-toc-section\" id=\"Loading-the-model\"><\/span>Loading the model<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>After importing the model into the project as described above, we need just one line of code in our\u00a0<em>analyze<\/em> method\u00a0to load it into memory:<\/p>\n<pre class=\"lang:default decode:true\">private val carsModel = CarsModel.newInstance(context)<\/pre>\n<p>The class from which we create a new instance is automatically generated by the TensorFlow Lite Android Wrapper Code Generator from the imported model. We supply the activity Context as argument.<\/p>\n<h3><span class=\"ez-toc-section\" id=\"Running-the-inference\"><\/span>Running the inference<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>When talking about inference in this context, we refer to the process of running a machine learning model in order to process the input data, which is the camera stream in our case, and make predictions, such as detecting, classifying and localizing the objects it&#8217;s been trained to recognize.<br \/>\nThere are currently several ways to run the inference with Object Detection TensorFlow Lite models in Android:<\/p>\n<ul>\n<li>Using the recently released ML Model Binding in combination with the TensorFlow Lite Android Wrapper Code Generator, part of the TensorFlow Lite Android Support Library. Using the Model Binding explained earlier, Android Studio will automatically configure settings for the project and generate wrapper classes to enable the integration on Android based on the model metadata. Hence, the implementation is very straightforward, as we will not need to interact directly with the internal ByteBuffers. The following dependencies are used:<\/li>\n<\/ul>\n<ol>\n<li style=\"list-style-type: none;\">\n<ul>\n<li>implementation &#8222;org.tensorflow:tensorflow-lite-support:0.1.0&#8220;<\/li>\n<li>implementation &#8222;org.tensorflow:tensorflow-lite-metadata:0.1.0&#8220;<\/li>\n<li>implementation &#8222;org.tensorflow:tensorflow-lite-gpu:2.4.0&#8220; (Optional, for accelerating model inference through the use of delegates and the number of threads)<\/li>\n<\/ul>\n<\/li>\n<\/ol>\n<p style=\"padding-left: 70px;\"><em>Note: TensorFlow Lite wrapper code generator is in experimental (beta) phase.<\/em><\/p>\n<p style=\"padding-left: 70px;\"><em>Note: the ML Model Binding is a new component and it requires Android Studio version 4.1 or above.<\/em><\/p>\n<ul>\n<li>Using the Object Detector API from the TensorFlow Lite Task Library. It provides clean and easy-to-use model interfaces for popular machine learning tasks. It also includes image processing functions and a label map locale. To use the Support Library in the app, use the AAR hosted at MavenCentral for Task Vision library. The following dependencies are used:<\/li>\n<\/ul>\n<ol>\n<li style=\"list-style-type: none;\">\n<ul>\n<li>implementation &#8222;org.tensorflow:tensorflow-lite-task-vision:0.2.0&#8220;<\/li>\n<\/ul>\n<\/li>\n<\/ol>\n<ul>\n<li>Using the TensorFlow Lite Interpreter Java API. This option does not provide high-level methods as the previous ones. That\u2019s why choosing this approach is\u00a0 recommended only if you are using a platform other than Android or iOS, or if you are already familiar with the <a href=\"https:\/\/www.tensorflow.org\/api_docs\/python\/tf\/lite\">TensorFlow Lite APIs<\/a> and not as a first choice for beginners. The following dependencies are used:<\/li>\n<\/ul>\n<ol>\n<li style=\"list-style-type: none;\">\n<ul>\n<li>implementation &#8222;org.tensorflow:tensorflow-lite:2.5.0&#8220;<\/li>\n<li>implementation &#8222;org.tensorflow:tensorflow-lite-metadata:0.2.0&#8220;<\/li>\n<\/ul>\n<\/li>\n<\/ol>\n<p>The support for TensorFlow Models for domains other than Image Classification and Style Transfer by ML Model Binding is, as mentioned earlier,\u00a0 currently still limited. However, it advances very rapidly and in Android Studio Version 4.2 it will officially support Object Detection models. Thus, in this post we will focus on the first option, and use ML Model Binding and the Wrapper Code Generator in our app.<br \/>\nUsing the generated code, running the inference takes just one line of code in our <em>analyzer<\/em> method and takes as an argument the TensorImage we created earlier:<\/p>\n<pre class=\"lang:default decode:true\">val outputs = carsModel.process(tImage)<\/pre>\n<p>The variable outputs will contain the raw results, which we need to be interpreted to extract meaningful information.<\/p>\n<h3><span class=\"ez-toc-section\" id=\"Interpreting-the-results\"><\/span>Interpreting the results<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<h4><span class=\"ez-toc-section\" id=\"Number-of-boxes-and-detection-scores\"><\/span>Number of boxes and detection scores<span class=\"ez-toc-section-end\"><\/span><\/h4>\n<p>The number of boxes, as well as the confidence scores values can be used as exposed from the API by extracting them from the TensorBuffers converted to arrays without further manipulation. For the number of boxes, we get an integer value and for the scores an array of floating point values between 0 and 1 representing the probability that a class was detected.<\/p>\n<p>The higher the confidence is, the most probable it is that the class is indeed right. Results with low confidence should be mostly ignored as it is not probable that they represent the objects in the image. The most suitable value for this threshold will depend on the application.\u00a0The threshold below which we ignore results in our application is 0.4.<\/p>\n<pre class=\"lang:default decode:true\">val numBoxes = outputs.numberOfDetectionsAsTensorBuffer.intArray[0]<\/pre>\n<pre class=\"lang:default decode:true \">val detectionScores = outputs.scoreAsTensorBuffer.floatArray\r\n<\/pre>\n<h4><span class=\"ez-toc-section\" id=\"Categories\"><\/span>Categories<span class=\"ez-toc-section-end\"><\/span><\/h4>\n<p>Perhaps because it is still under development in experimental phase (see the version numbers all start with 0) and, as explained in the <a href=\"https:\/\/developer.android.com\/studio\/releases#supported_models\">release notes of Android Studio 4.1<\/a>, the current implementation officially only supports image classification and style transfer models, we experienced errors when trying to use the <em>categoryAsCategoryList<\/em> property suggested by the code generator, even with our model enhanced with metadata.<\/p>\n<p>That\u2019s why we came up with a workaround for the meantime based on the utils functions and a TensorLabel to parse the category as integer to its corresponding name, which is finally shown to the user graphically. This conversion works as follows, using the Android Support Library once again:<\/p>\n<ul>\n<li>We include the labels text file in the assets folder of the Android project<\/li>\n<li>Load them into memory using the Util function and the activity context:\n<pre class=\"lang:default decode:true \" style=\"padding-left: 120px;\">var associatedAxisLabels: List&lt;String&gt; = FileUtil.loadLabels(context, \u201clabelmap.txt\u201c)<\/pre>\n<\/li>\n<li>Create a TensorProcessor:\n<pre class=\"lang:default decode:true\" style=\"padding-left: 30px;\">val processor = TensorProcessor.Builder().build()<\/pre>\n<\/li>\n<li>Create a TensorBuffer of the fixed size of the number of labels in the file\n<pre class=\"lang:default decode:true\">val buffer = TensorBuffer.createFixedSize(intArrayOf(1, 196), DataType.UINT8)\r\n<\/pre>\n<\/li>\n<li>Finally, create a TensorLabel with those labels\n<pre class=\"lang:default decode:true\">val labels = TensorLabel(associatedAxisLabels, processor.process(buffer))<\/pre>\n<\/li>\n<li>Extract the detected categories from the TensorBuffer output the same way we did with the others and convert the object into an integer array\n<pre class=\"lang:default decode:true\">val detectionClasses = outputs.categoryAsTensorBuffer.intArray<\/pre>\n<\/li>\n<li>Use our Array of N integers, each indicating the index of a class label from the labels file to map the results to obtain category labels as strings\n<pre class=\"lang:default decode:true\">labels.categoryList[detectionClasses[i]].label<\/pre>\n<\/li>\n<\/ul>\n<h4><span class=\"ez-toc-section\" id=\"Bounding-Boxes\"><\/span>Bounding Boxes<span class=\"ez-toc-section-end\"><\/span><\/h4>\n<p>The output includes an array of bounding boxes. However, we can\u2019t simply draw those on top of the preview, since that wouldn\u2019t make much sense. They need to be interpreted and afterwards adapted before being shown to the user. Remember that the pixel values output by the model refer to the position in the cropped and scaled image, so we must undo those operations and then translate between the different coordinate systems to fit the preview image where they will be displayed. This means we need to do some math!<\/p>\n<p>For each detected object, the model will return an array of four numbers representing a bounding rectangle that surrounds its position with the numbers ordered as follows: [ top, left, bottom, right ].<br \/>\nSo, first of all, we extract this information from the Float Array of size numBoxes returned from the inference to create a 2-dimensional array:<\/p>\n<pre class=\"lang:default decode:true\">val boxes = outputs.locationAsTensorBuffer\r\nval detectionBoxes = Array(numBoxes) { FloatArray(4) }\r\nfor (i in detectionBoxes.indices) {\r\n    detectionBoxes[i] = boxes.floatArray.copyOfRange(\r\n    4 * i,\r\n    4 * i + 4\r\n    )\r\n}<\/pre>\n<p>The numbers are floating point values between 0 and 1 indicating the position in the processed image. So we multiply by the height and width, respectively, to obtain the position in that image in pixels and form a RectF object with them.<\/p>\n<p>After that, we apply the inverse transformation of what we did in the image processing step earlier for each of the boxes, in order to translate them to the initial image captured with the Image Analysis use-case. Don\u2019t forget the exact order (top, left, bottom, right), which is different from the order in the RectF constructor (left, top, right, bottom)!<\/p>\n<p>This would be the code for those operations:<\/p>\n<pre class=\"lang:default decode:true\">imageProcessor.inverseTransform(\r\nRectF(\r\n    detectionBoxes[i][1] * WIDTH,\r\n    detectionBoxes[i][0] * HEIGHT,\r\n    detectionBoxes[i][3] * WIDTH,\r\n    detectionBoxes[i][2] * HEIGHT\r\n    ), imageProxy.height, imageProxy.width\r\n)<\/pre>\n<p>Now, we have the exact position of the boxes in the initial image, which is what the <em>analyzer<\/em> function is in charge of computing. The rest of the transformations are part of displaying them.<\/p>\n<p>Once we have all the information we need, we will store it as a list of Recognition item objects, which have fields for the label, the probability and the location of the bounding box.<\/p>\n<pre class=\"\">for (i in 0 until MAX_RESULT_DISPLAY) {\r\n    items.add(\r\n        Recognition(\r\n            labels.categoryList[detectionClasses[i]].label,\r\n            detectionScores[i],\r\n            imageProcessor.inverseTransform(\r\n                RectF(\r\n                    detectionBoxes[i][1] * WIDTH,\r\n                    detectionBoxes[i][0] * HEIGHT,\r\n                    detectionBoxes[i][3] * WIDTH,\r\n                    detectionBoxes[i][2] * HEIGHT\r\n                ), imageProxy.height, imageProxy.width\r\n            )\r\n        )\r\n    )\r\n}<\/pre>\n<p>These will be fed to a LiveData object in our ViewModel and the Activity will update the views with the results whenever new data is available.<\/p>\n<h3><span class=\"ez-toc-section\" id=\"Displaying-the-results\"><\/span>Displaying the results<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>Finally, we show the user the results graphically on top of the camera preview in the form of a bounding box for the recognized car and a label indicating the class and its confidence every time the values change.<\/p>\n<p>As promised, there is still one last step to be done: to translate the boxes into yet another coordinate system, the one from our PreviewView. We do that by mapping the coordinates using a correction matrix we build with <a href=\"https:\/\/gist.github.com\/SilviaSantano\/6b46b7cf47ddfc8a73227b68d246d5bf\">this method<\/a>, which takes as arguments both the ImageProxy obtained and our PreviewView.<\/p>\n<p>Now, by means of the matrix, we can finally translate our RectF object. We also add some width compensation to the sides (orientation-aware) to our box to improve the fitting, since the aspect ratio of both systems differs, and they\u2019re ready to be drawn onto our view using our utils class.<\/p>\n<pre class=\"lang:default decode:true\">(...)\r\nmatrix.mapRect(i.location)\r\nval widthCompensation = originalImageWidth - WIDTH\r\ni.location = addCompensation(i.location, widthCompensation)\r\nrecognizedBoundingBox.drawRect(canvas, i.location)\r\nrecognizedLabelText.drawText(canvas, i.location.left, i.location.top, i.label, i.confidence)<\/pre>\n<p>&nbsp;<\/p>\n<p>And voil\u00e0! We built an Android application that is able to recognize cars from the smartphone&#8217;s camera stream by using a neural network we trained with the TensorFlow Object Detection API. I hope you enjoyed this article!<\/p>\n<div id='gallery-1' class='gallery galleryid-20617 gallery-columns-3 gallery-size-large'><figure class='gallery-item'>\n\t\t\t<div class='gallery-icon portrait'>\n\t\t\t\t<a href='https:\/\/www.inovex.de\/wp-content\/uploads\/2021\/02\/Screenshot_20210204-132053.png'><img loading=\"lazy\" decoding=\"async\" width=\"485\" height=\"1024\" src=\"https:\/\/www.inovex.de\/wp-content\/uploads\/2021\/02\/Screenshot_20210204-132053-485x1024.png\" class=\"attachment-large size-large\" alt=\"Ein schwarzer BMW 1er Cabrio aus dem Jahr 2012, geparkt auf einer Stra\u00dfe, mit einem erkennbaren Rahmen um das Fahrzeug, das von einer App zur Fahrzeugerkennung identifiziert wurde.\" srcset=\"https:\/\/www.inovex.de\/wp-content\/uploads\/2021\/02\/Screenshot_20210204-132053-485x1024.png 485w, https:\/\/www.inovex.de\/wp-content\/uploads\/2021\/02\/Screenshot_20210204-132053-142x300.png 142w, https:\/\/www.inovex.de\/wp-content\/uploads\/2021\/02\/Screenshot_20210204-132053-768x1621.png 768w, https:\/\/www.inovex.de\/wp-content\/uploads\/2021\/02\/Screenshot_20210204-132053-728x1536.png 728w, https:\/\/www.inovex.de\/wp-content\/uploads\/2021\/02\/Screenshot_20210204-132053-970x2048.png 970w, https:\/\/www.inovex.de\/wp-content\/uploads\/2021\/02\/Screenshot_20210204-132053-400x844.png 400w, https:\/\/www.inovex.de\/wp-content\/uploads\/2021\/02\/Screenshot_20210204-132053-360x760.png 360w, https:\/\/www.inovex.de\/wp-content\/uploads\/2021\/02\/Screenshot_20210204-132053.png 1080w\" sizes=\"auto, (max-width: 485px) 100vw, 485px\" \/><\/a>\n\t\t\t<\/div><\/figure><figure class='gallery-item'>\n\t\t\t<div class='gallery-icon portrait'>\n\t\t\t\t<a href='https:\/\/www.inovex.de\/wp-content\/uploads\/2021\/02\/Screenshot_20210204-132001.png'><img loading=\"lazy\" decoding=\"async\" width=\"485\" height=\"1024\" src=\"https:\/\/www.inovex.de\/wp-content\/uploads\/2021\/02\/Screenshot_20210204-132001-485x1024.png\" class=\"attachment-large size-large\" alt=\"Ein wei\u00dfer FIAT 500 Cabrio aus dem Jahr 2012, geparkt an einer Stra\u00dfe, umgeben von anderen Fahrzeugen, mit einem erkennbaren Rahmen, der von einer App zur Fahrzeugerkennung angezeigt wird.\" srcset=\"https:\/\/www.inovex.de\/wp-content\/uploads\/2021\/02\/Screenshot_20210204-132001-485x1024.png 485w, https:\/\/www.inovex.de\/wp-content\/uploads\/2021\/02\/Screenshot_20210204-132001-142x300.png 142w, https:\/\/www.inovex.de\/wp-content\/uploads\/2021\/02\/Screenshot_20210204-132001-768x1621.png 768w, https:\/\/www.inovex.de\/wp-content\/uploads\/2021\/02\/Screenshot_20210204-132001-728x1536.png 728w, https:\/\/www.inovex.de\/wp-content\/uploads\/2021\/02\/Screenshot_20210204-132001-970x2048.png 970w, https:\/\/www.inovex.de\/wp-content\/uploads\/2021\/02\/Screenshot_20210204-132001-400x844.png 400w, https:\/\/www.inovex.de\/wp-content\/uploads\/2021\/02\/Screenshot_20210204-132001-360x760.png 360w, https:\/\/www.inovex.de\/wp-content\/uploads\/2021\/02\/Screenshot_20210204-132001.png 1080w\" sizes=\"auto, (max-width: 485px) 100vw, 485px\" \/><\/a>\n\t\t\t<\/div><\/figure><figure class='gallery-item'>\n\t\t\t<div class='gallery-icon portrait'>\n\t\t\t\t<a href='https:\/\/www.inovex.de\/wp-content\/uploads\/2021\/02\/Screenshot_20210204-132726.png'><img loading=\"lazy\" decoding=\"async\" width=\"485\" height=\"1024\" src=\"https:\/\/www.inovex.de\/wp-content\/uploads\/2021\/02\/Screenshot_20210204-132726-485x1024.png\" class=\"attachment-large size-large\" alt=\"Screenshot einer mobilen Anwendung zur Fahrzeugerkennung mit TensorFlow Lite. Die Anwendung zeigt ein Bild eines Mercedes-Benz C-Class Sedan aus dem Jahr 2012, das mit einer Umrandung hervorgehoben ist. Oben auf dem Bildschirm steht der Titel &#039;Recognize Cars With TFLite&#039;.\" srcset=\"https:\/\/www.inovex.de\/wp-content\/uploads\/2021\/02\/Screenshot_20210204-132726-485x1024.png 485w, https:\/\/www.inovex.de\/wp-content\/uploads\/2021\/02\/Screenshot_20210204-132726-142x300.png 142w, https:\/\/www.inovex.de\/wp-content\/uploads\/2021\/02\/Screenshot_20210204-132726-768x1621.png 768w, https:\/\/www.inovex.de\/wp-content\/uploads\/2021\/02\/Screenshot_20210204-132726-728x1536.png 728w, https:\/\/www.inovex.de\/wp-content\/uploads\/2021\/02\/Screenshot_20210204-132726-970x2048.png 970w, https:\/\/www.inovex.de\/wp-content\/uploads\/2021\/02\/Screenshot_20210204-132726-400x844.png 400w, https:\/\/www.inovex.de\/wp-content\/uploads\/2021\/02\/Screenshot_20210204-132726-360x760.png 360w, https:\/\/www.inovex.de\/wp-content\/uploads\/2021\/02\/Screenshot_20210204-132726.png 1080w\" sizes=\"auto, (max-width: 485px) 100vw, 485px\" \/><\/a>\n\t\t\t<\/div><\/figure>\n\t\t<\/div>\n\n","protected":false},"excerpt":{"rendered":"<p>This is the third article of our blog post series about Deep Learning for mobile devices. The first post tackled some of the theoretical background of on-device machine learning, including quantization and state-of-the-art model architectures. The second explored how to do quantization-aware model training with the TensorFlow Object Detection API. In this article, we will [&hellip;]<\/p>\n","protected":false},"author":57,"featured_media":29422,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"inline_featured_image":false,"ep_exclude_from_search":false,"footnotes":""},"tags":[149,510,511,150,151,353,366,152],"service":[420,76],"coauthors":[{"id":57,"display_name":"Silvia Santano","user_nicename":"ssantano"}],"class_list":["post-20617","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","tag-android","tag-apps-2","tag-artificial-intelligence-2","tag-computer-vision","tag-deep-learning","tag-mobile","tag-object-detection","tag-tensorflow","service-apps","service-artificial-intelligence"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.6 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Create an AI-Powered Android Application with TensorFlow Lite - inovex GmbH<\/title>\n<meta name=\"description\" content=\"In this article you&#039;ll learn to convert a deep learning model to TensorFlow Lite and integrate it into an AI-powered Android application.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.inovex.de\/de\/blog\/deep-learning-for-mobile-devices-with-tensorflow-lite-ai-powered-android-application\/\" \/>\n<meta property=\"og:locale\" content=\"de_DE\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Create an AI-Powered Android Application with TensorFlow Lite - inovex GmbH\" \/>\n<meta property=\"og:description\" content=\"In this article you&#039;ll learn to convert a deep learning model to TensorFlow Lite and integrate it into an AI-powered Android application.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.inovex.de\/de\/blog\/deep-learning-for-mobile-devices-with-tensorflow-lite-ai-powered-android-application\/\" \/>\n<meta property=\"og:site_name\" content=\"inovex GmbH\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/inovexde\" \/>\n<meta property=\"article:published_time\" content=\"2021-05-28T13:10:28+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-05-08T07:23:05+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.inovex.de\/wp-content\/uploads\/tensorflow-car-detector-app-1.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1920\" \/>\n\t<meta property=\"og:image:height\" content=\"1080\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Silvia Santano\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:image\" content=\"https:\/\/www.inovex.de\/wp-content\/uploads\/tensorflow-car-detector-app-1-1024x576.png\" \/>\n<meta name=\"twitter:creator\" content=\"@inovexgmbh\" \/>\n<meta name=\"twitter:site\" content=\"@inovexgmbh\" \/>\n<meta name=\"twitter:label1\" content=\"Verfasst von\" \/>\n\t<meta name=\"twitter:data1\" content=\"Silvia Santano\" \/>\n\t<meta name=\"twitter:label2\" content=\"Gesch\u00e4tzte Lesezeit\" \/>\n\t<meta name=\"twitter:data2\" content=\"18\u00a0Minuten\" \/>\n\t<meta name=\"twitter:label3\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data3\" content=\"Silvia Santano\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/blog\\\/deep-learning-for-mobile-devices-with-tensorflow-lite-ai-powered-android-application\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/blog\\\/deep-learning-for-mobile-devices-with-tensorflow-lite-ai-powered-android-application\\\/\"},\"author\":{\"name\":\"Silvia Santano\",\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/#\\\/schema\\\/person\\\/f2d17a49f3a806bc6cdb4902a32a0ef9\"},\"headline\":\"Deep Learning for Mobile Devices with TensorFlow Lite: Create an AI-Powered Android Application\",\"datePublished\":\"2021-05-28T13:10:28+00:00\",\"dateModified\":\"2023-05-08T07:23:05+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/blog\\\/deep-learning-for-mobile-devices-with-tensorflow-lite-ai-powered-android-application\\\/\"},\"wordCount\":3708,\"commentCount\":2,\"publisher\":{\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/#organization\"},\"image\":{\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/blog\\\/deep-learning-for-mobile-devices-with-tensorflow-lite-ai-powered-android-application\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.inovex.de\\\/wp-content\\\/uploads\\\/tensorflow-car-detector-app-1.png\",\"keywords\":[\"Android\",\"Apps\",\"Artificial Intelligence\",\"Computer Vision\",\"Deep Learning\",\"Mobile\",\"Object Detection\",\"TensorFlow\"],\"articleSection\":[\"Applications\",\"English Content\",\"General\"],\"inLanguage\":\"de\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/www.inovex.de\\\/de\\\/blog\\\/deep-learning-for-mobile-devices-with-tensorflow-lite-ai-powered-android-application\\\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/blog\\\/deep-learning-for-mobile-devices-with-tensorflow-lite-ai-powered-android-application\\\/\",\"url\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/blog\\\/deep-learning-for-mobile-devices-with-tensorflow-lite-ai-powered-android-application\\\/\",\"name\":\"Create an AI-Powered Android Application with TensorFlow Lite - inovex GmbH\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/blog\\\/deep-learning-for-mobile-devices-with-tensorflow-lite-ai-powered-android-application\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/blog\\\/deep-learning-for-mobile-devices-with-tensorflow-lite-ai-powered-android-application\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.inovex.de\\\/wp-content\\\/uploads\\\/tensorflow-car-detector-app-1.png\",\"datePublished\":\"2021-05-28T13:10:28+00:00\",\"dateModified\":\"2023-05-08T07:23:05+00:00\",\"description\":\"In this article you'll learn to convert a deep learning model to TensorFlow Lite and integrate it into an AI-powered Android application.\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/blog\\\/deep-learning-for-mobile-devices-with-tensorflow-lite-ai-powered-android-application\\\/#breadcrumb\"},\"inLanguage\":\"de\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/www.inovex.de\\\/de\\\/blog\\\/deep-learning-for-mobile-devices-with-tensorflow-lite-ai-powered-android-application\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"de\",\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/blog\\\/deep-learning-for-mobile-devices-with-tensorflow-lite-ai-powered-android-application\\\/#primaryimage\",\"url\":\"https:\\\/\\\/www.inovex.de\\\/wp-content\\\/uploads\\\/tensorflow-car-detector-app-1.png\",\"contentUrl\":\"https:\\\/\\\/www.inovex.de\\\/wp-content\\\/uploads\\\/tensorflow-car-detector-app-1.png\",\"width\":1920,\"height\":1080,\"caption\":\"A phone identifies a car model using TensorFlow Lite\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/blog\\\/deep-learning-for-mobile-devices-with-tensorflow-lite-ai-powered-android-application\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Deep Learning for Mobile Devices with TensorFlow Lite: Create an AI-Powered Android Application\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/#website\",\"url\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/\",\"name\":\"inovex GmbH\",\"description\":\"\",\"publisher\":{\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"de\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/#organization\",\"name\":\"inovex GmbH\",\"url\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"de\",\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/www.inovex.de\\\/wp-content\\\/uploads\\\/2021\\\/03\\\/inovex-logo-16-9-1.png\",\"contentUrl\":\"https:\\\/\\\/www.inovex.de\\\/wp-content\\\/uploads\\\/2021\\\/03\\\/inovex-logo-16-9-1.png\",\"width\":1921,\"height\":1081,\"caption\":\"inovex GmbH\"},\"image\":{\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/#\\\/schema\\\/logo\\\/image\\\/\"},\"sameAs\":[\"https:\\\/\\\/www.facebook.com\\\/inovexde\",\"https:\\\/\\\/x.com\\\/inovexgmbh\",\"https:\\\/\\\/www.instagram.com\\\/inovexlife\\\/\",\"https:\\\/\\\/www.linkedin.com\\\/company\\\/inovex\",\"https:\\\/\\\/www.youtube.com\\\/channel\\\/UC7r66GT14hROB_RQsQBAQUQ\"]},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/#\\\/schema\\\/person\\\/f2d17a49f3a806bc6cdb4902a32a0ef9\",\"name\":\"Silvia Santano\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"de\",\"@id\":\"https:\\\/\\\/www.inovex.de\\\/wp-content\\\/uploads\\\/cropped-IMG_6952-2-scaled-e1665488822192-1-96x96.jpegbd867eef3c3c053b3e6e1634ffac9ffc\",\"url\":\"https:\\\/\\\/www.inovex.de\\\/wp-content\\\/uploads\\\/cropped-IMG_6952-2-scaled-e1665488822192-1-96x96.jpeg\",\"contentUrl\":\"https:\\\/\\\/www.inovex.de\\\/wp-content\\\/uploads\\\/cropped-IMG_6952-2-scaled-e1665488822192-1-96x96.jpeg\",\"caption\":\"Silvia Santano\"},\"url\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/blog\\\/author\\\/ssantano\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Create an AI-Powered Android Application with TensorFlow Lite - inovex GmbH","description":"In this article you'll learn to convert a deep learning model to TensorFlow Lite and integrate it into an AI-powered Android application.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.inovex.de\/de\/blog\/deep-learning-for-mobile-devices-with-tensorflow-lite-ai-powered-android-application\/","og_locale":"de_DE","og_type":"article","og_title":"Create an AI-Powered Android Application with TensorFlow Lite - inovex GmbH","og_description":"In this article you'll learn to convert a deep learning model to TensorFlow Lite and integrate it into an AI-powered Android application.","og_url":"https:\/\/www.inovex.de\/de\/blog\/deep-learning-for-mobile-devices-with-tensorflow-lite-ai-powered-android-application\/","og_site_name":"inovex GmbH","article_publisher":"https:\/\/www.facebook.com\/inovexde","article_published_time":"2021-05-28T13:10:28+00:00","article_modified_time":"2023-05-08T07:23:05+00:00","og_image":[{"width":1920,"height":1080,"url":"https:\/\/www.inovex.de\/wp-content\/uploads\/tensorflow-car-detector-app-1.png","type":"image\/png"}],"author":"Silvia Santano","twitter_card":"summary_large_image","twitter_image":"https:\/\/www.inovex.de\/wp-content\/uploads\/tensorflow-car-detector-app-1-1024x576.png","twitter_creator":"@inovexgmbh","twitter_site":"@inovexgmbh","twitter_misc":{"Verfasst von":"Silvia Santano","Gesch\u00e4tzte Lesezeit":"18\u00a0Minuten","Written by":"Silvia Santano"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.inovex.de\/de\/blog\/deep-learning-for-mobile-devices-with-tensorflow-lite-ai-powered-android-application\/#article","isPartOf":{"@id":"https:\/\/www.inovex.de\/de\/blog\/deep-learning-for-mobile-devices-with-tensorflow-lite-ai-powered-android-application\/"},"author":{"name":"Silvia Santano","@id":"https:\/\/www.inovex.de\/de\/#\/schema\/person\/f2d17a49f3a806bc6cdb4902a32a0ef9"},"headline":"Deep Learning for Mobile Devices with TensorFlow Lite: Create an AI-Powered Android Application","datePublished":"2021-05-28T13:10:28+00:00","dateModified":"2023-05-08T07:23:05+00:00","mainEntityOfPage":{"@id":"https:\/\/www.inovex.de\/de\/blog\/deep-learning-for-mobile-devices-with-tensorflow-lite-ai-powered-android-application\/"},"wordCount":3708,"commentCount":2,"publisher":{"@id":"https:\/\/www.inovex.de\/de\/#organization"},"image":{"@id":"https:\/\/www.inovex.de\/de\/blog\/deep-learning-for-mobile-devices-with-tensorflow-lite-ai-powered-android-application\/#primaryimage"},"thumbnailUrl":"https:\/\/www.inovex.de\/wp-content\/uploads\/tensorflow-car-detector-app-1.png","keywords":["Android","Apps","Artificial Intelligence","Computer Vision","Deep Learning","Mobile","Object Detection","TensorFlow"],"articleSection":["Applications","English Content","General"],"inLanguage":"de","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.inovex.de\/de\/blog\/deep-learning-for-mobile-devices-with-tensorflow-lite-ai-powered-android-application\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.inovex.de\/de\/blog\/deep-learning-for-mobile-devices-with-tensorflow-lite-ai-powered-android-application\/","url":"https:\/\/www.inovex.de\/de\/blog\/deep-learning-for-mobile-devices-with-tensorflow-lite-ai-powered-android-application\/","name":"Create an AI-Powered Android Application with TensorFlow Lite - inovex GmbH","isPartOf":{"@id":"https:\/\/www.inovex.de\/de\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.inovex.de\/de\/blog\/deep-learning-for-mobile-devices-with-tensorflow-lite-ai-powered-android-application\/#primaryimage"},"image":{"@id":"https:\/\/www.inovex.de\/de\/blog\/deep-learning-for-mobile-devices-with-tensorflow-lite-ai-powered-android-application\/#primaryimage"},"thumbnailUrl":"https:\/\/www.inovex.de\/wp-content\/uploads\/tensorflow-car-detector-app-1.png","datePublished":"2021-05-28T13:10:28+00:00","dateModified":"2023-05-08T07:23:05+00:00","description":"In this article you'll learn to convert a deep learning model to TensorFlow Lite and integrate it into an AI-powered Android application.","breadcrumb":{"@id":"https:\/\/www.inovex.de\/de\/blog\/deep-learning-for-mobile-devices-with-tensorflow-lite-ai-powered-android-application\/#breadcrumb"},"inLanguage":"de","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.inovex.de\/de\/blog\/deep-learning-for-mobile-devices-with-tensorflow-lite-ai-powered-android-application\/"]}]},{"@type":"ImageObject","inLanguage":"de","@id":"https:\/\/www.inovex.de\/de\/blog\/deep-learning-for-mobile-devices-with-tensorflow-lite-ai-powered-android-application\/#primaryimage","url":"https:\/\/www.inovex.de\/wp-content\/uploads\/tensorflow-car-detector-app-1.png","contentUrl":"https:\/\/www.inovex.de\/wp-content\/uploads\/tensorflow-car-detector-app-1.png","width":1920,"height":1080,"caption":"A phone identifies a car model using TensorFlow Lite"},{"@type":"BreadcrumbList","@id":"https:\/\/www.inovex.de\/de\/blog\/deep-learning-for-mobile-devices-with-tensorflow-lite-ai-powered-android-application\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.inovex.de\/de\/"},{"@type":"ListItem","position":2,"name":"Deep Learning for Mobile Devices with TensorFlow Lite: Create an AI-Powered Android Application"}]},{"@type":"WebSite","@id":"https:\/\/www.inovex.de\/de\/#website","url":"https:\/\/www.inovex.de\/de\/","name":"inovex GmbH","description":"","publisher":{"@id":"https:\/\/www.inovex.de\/de\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.inovex.de\/de\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"de"},{"@type":"Organization","@id":"https:\/\/www.inovex.de\/de\/#organization","name":"inovex GmbH","url":"https:\/\/www.inovex.de\/de\/","logo":{"@type":"ImageObject","inLanguage":"de","@id":"https:\/\/www.inovex.de\/de\/#\/schema\/logo\/image\/","url":"https:\/\/www.inovex.de\/wp-content\/uploads\/2021\/03\/inovex-logo-16-9-1.png","contentUrl":"https:\/\/www.inovex.de\/wp-content\/uploads\/2021\/03\/inovex-logo-16-9-1.png","width":1921,"height":1081,"caption":"inovex GmbH"},"image":{"@id":"https:\/\/www.inovex.de\/de\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/inovexde","https:\/\/x.com\/inovexgmbh","https:\/\/www.instagram.com\/inovexlife\/","https:\/\/www.linkedin.com\/company\/inovex","https:\/\/www.youtube.com\/channel\/UC7r66GT14hROB_RQsQBAQUQ"]},{"@type":"Person","@id":"https:\/\/www.inovex.de\/de\/#\/schema\/person\/f2d17a49f3a806bc6cdb4902a32a0ef9","name":"Silvia Santano","image":{"@type":"ImageObject","inLanguage":"de","@id":"https:\/\/www.inovex.de\/wp-content\/uploads\/cropped-IMG_6952-2-scaled-e1665488822192-1-96x96.jpegbd867eef3c3c053b3e6e1634ffac9ffc","url":"https:\/\/www.inovex.de\/wp-content\/uploads\/cropped-IMG_6952-2-scaled-e1665488822192-1-96x96.jpeg","contentUrl":"https:\/\/www.inovex.de\/wp-content\/uploads\/cropped-IMG_6952-2-scaled-e1665488822192-1-96x96.jpeg","caption":"Silvia Santano"},"url":"https:\/\/www.inovex.de\/de\/blog\/author\/ssantano\/"}]}},"_links":{"self":[{"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/posts\/20617","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/users\/57"}],"replies":[{"embeddable":true,"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/comments?post=20617"}],"version-history":[{"count":6,"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/posts\/20617\/revisions"}],"predecessor-version":[{"id":45295,"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/posts\/20617\/revisions\/45295"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/media\/29422"}],"wp:attachment":[{"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/media?parent=20617"}],"wp:term":[{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/tags?post=20617"},{"taxonomy":"service","embeddable":true,"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/service?post=20617"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/coauthors?post=20617"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}