Evaluating Performance and Quality of TensorFlow Lite Models on Mobile Platforms

  • Students
  • Stockholm, Sweden

Evaluating Performance and Quality of TensorFlow Lite Models on Mobile Platforms

Job description

Background

Modern Machine Learning (ML) models are solving real-life problems in various applications today: identifying objects in images or video, classifying medical anomalies, enabling accurate natural language processing and machine translation, among many others.

By using neural networks with a large number of hidden layers (deep learning), the models can gain an understanding of the data that can build upon itself much like humans do (e.g. early layers find lines and shapes in an image, while later layers classify collections of these as dog-like or human-like). Each layer in such a model typically has a large number of connections, and when asking the network to do a prediction, a lot of computations need to be done to feed the data through all of the neurons in all of the layers in the model to reach an output.

Today, modern use cases for ML include on-device inference that should run on mobile platforms. This has several advantages, including latency and connectivity (no roundtrip to a server) and privacy (data does not leave the device). However, this imposes technical constraints on the way the model is executed, since mobile devices are typically more limited in their hardware, both in terms of computational power, as well as factors like thermal throttling, and users' expectations of a responsive experience in the app itself.

TensorFlow is Google's open-source platform for ML, with high-level APIs for building, training and executing ML models. TensorFlow Lite is an optimized variant for mobile, embedded, and IoT devices, with support for various platforms including Android and iOS.

Job requirements

Task

In this thesis project, you will examine and evaluate TensorFlow Lite on mobile platforms. This is a fairly open-ended project, so you will be responsible for defining the exact problem statement.

An example might be an analysis and evaluation of the tradeoffs when using TensorFlow Lite models as compared to the full models, for example around accuracy, runtime performance, limitations and restrictions, file size, etc. You might also compare performance between mobile platforms (Android/iOS) or qualitative properties like "developer satisfaction".

For models, you could use open-source models (TensorFlow has a number of predefined and pre-trained models, as well as data sets) or your own models. Perhaps there are aspects around the design of the model or data that is relevant for a good result.

This thesis project may suit you if...

  • You have a passion for Machine Learning.
  • You care about the practical aspects of software engineering.
  • You love getting into technical details and benchmarks.