TensorFlow

Before opening FlutterFlow, decide which integration path fits your use case: **On-device (TFLite):** Best for image/vision tasks (classification, detection, pose), text classification, and any feature that benefits from offline operation or ultra-low latency. Works on iOS and Android native builds. Will NOT work in FlutterFlow web preview (throws MissingPluginException or UnimplementedError) — plan to test on a real device. **Server inference (TF Serving):** Best for large models that won't fit in an app bundle, models you retrain frequently, or complex tabular/structured-data models hosted on your infrastructure. For the on-device path, prepare your .tflite file. If you have a TensorFlow SavedModel, convert it: - Use TensorFlow's Python converter: `tf.lite.TFLiteConverter.from_saved_model(path).convert()` - Or download a ready-to-use model from TensorFlow Hub (tfhub.dev) — MobileNet, EfficientDet, PoseNet are all available as .tflite Inspect the model with Netron (netron.app) to confirm: input tensor name, shape (e.g. [1, 224, 224, 3]), and dtype (float32 or uint8). You'll need these exact values in your Dart code — a shape mismatch is the #1 cause of runtime failures. For TF Serving: confirm your server is running (`docker run -p 8501:8501 tensorflow/serving --model_name=mymodel --model_base_path=/models/mymodel`) and the endpoint responds to `curl http://localhost:8501/v1/models/mymodel` before building in FlutterFlow.

In your FlutterFlow project, go to Custom Code in the left navigation panel, then click Dependencies (or Pubspec Dependencies). Click + Add and type `tflite_flutter`. FlutterFlow will look up the package on pub.dev — select the latest stable version. This single package handles model loading and inference for both iOS and Android. Next, you need to add your .tflite model as a Flutter asset. FlutterFlow supports adding asset files through the GitHub integration (Basic plan or above required). Push your .tflite file to your connected GitHub repo under `assets/models/your_model.tflite`. In your pubspec.yaml (accessible through the GitHub repo), add the asset path under `flutter: assets:`. If you don't have GitHub integration set up, use FlutterFlow's built-in Media Assets feature: go to the Media panel (image icon in the left nav), upload your .tflite file. Note the generated asset URL — you'll reference it in the Custom Action code as an asset path. For platform permissions: TFLite itself has no special permissions. If your app uses the camera to capture frames for inference, add camera permission in FlutterFlow's Permissions settings (Settings & Integrations → Permissions → Camera). The Info.plist description and AndroidManifest entries are handled automatically by FlutterFlow's permission system.

Now create the Custom Action that runs inference. In FlutterFlow, go to Custom Code → + Add → Action. Name it `runTFLiteInference`. Add the following arguments: - `imageBytes` (Bytes) — the raw image data to classify - Return type: String (or a JSON string with all results) Here's the complete Dart Custom Action. It loads the model from assets, preprocesses a 224×224 image to float32, runs inference, and returns the top predicted label index: ```dart import 'package:tflite_flutter/tflite_flutter.dart'; import 'dart:typed_data'; import 'dart:io'; Future<String> runTFLiteInference(Uint8List imageBytes) async { // Load model from assets (or from a file path if downloaded at runtime) final interpreter = await Interpreter.fromAsset('assets/models/your_model.tflite'); // Prepare input: resize and normalize to float32 [1, 224, 224, 3] // This assumes imageBytes is already a raw 224x224 RGB bitmap // For camera frames, use the image package to resize first final inputShape = interpreter.getInputTensor(0).shape; // [1, 224, 224, 3] final outputShape = interpreter.getOutputTensor(0).shape; // e.g. [1, 1001] for ImageNet var input = Float32List(1 * 224 * 224 * 3); for (int i = 0; i < imageBytes.length && i < input.length; i++) { // Normalize pixel values from [0, 255] to [-1.0, 1.0] input[i] = (imageBytes[i] / 127.5) - 1.0; } var inputTensor = input.reshape([1, 224, 224, 3]); // Prepare output buffer var output = List.filled(1001, 0.0).reshape([1, 1001]); // Run inference interpreter.run(inputTensor, output); interpreter.close(); // Find top prediction final outputList = (output as List)[0] as List<double>; int topIdx = 0; double topConf = 0.0; for (int i = 0; i < outputList.length; i++) { if (outputList[i] > topConf) { topConf = outputList[i]; topIdx = i; } } return '$topIdx:${(topConf * 100).toStringAsFixed(1)}%'; } ``` Paste this into the Custom Action code editor. Adjust the model filename, input shape, and normalization formula to match your specific model. For uint8 quantized models, use `Uint8List` input without normalization instead of Float32List.

1import 'package:tflite_flutter/tflite_flutter.dart';
2import 'dart:typed_data';
3
4Future<String> runTFLiteInference(Uint8List imageBytes) async {
5  final interpreter = await Interpreter.fromAsset('assets/models/your_model.tflite');
6
7  // Normalize to float32 [-1.0, 1.0] for MobileNet-style models
8  var input = Float32List(1 * 224 * 224 * 3);
9  for (int i = 0; i < imageBytes.length && i < input.length; i++) {
10    input[i] = (imageBytes[i] / 127.5) - 1.0;
11  }
12  var inputTensor = input.reshape([1, 224, 224, 3]);
13
14  // Output shape: [1, 1001] for MobileNet ImageNet (1000 classes + background)
15  var output = List.filled(1001, 0.0).reshape([1, 1001]);
16
17  interpreter.run(inputTensor, output);
18  interpreter.close();
19
20  final outputList = (output as List)[0] as List<double>;
21  int topIdx = 0;
22  double topConf = 0.0;
23  for (int i = 0; i < outputList.length; i++) {
24    if (outputList[i] > topConf) {
25      topConf = outputList[i];
26      topIdx = i;
27    }
28  }
29  return '$topIdx:${(topConf * 100).toStringAsFixed(1)}%';
30}

With the Custom Action ready, build the UI that triggers inference. Add a Camera or Image Picker widget to your screen, or use FlutterFlow's built-in Camera Action (available via Action Flow Editor → Camera → Capture Photo). The camera action returns image bytes that you'll pass to the Custom Action. Add a Button widget labelled 'Analyse' (or trigger automatically on photo capture). In the Action Flow Editor for the button or photo-capture callback: 1. Add action: Camera → Capture Photo → store result in an App State variable `capturedImageBytes` (type: Bytes) 2. Add action: Custom → runTFLiteInference → pass `capturedImageBytes` → store result in `predictionResult` (String App State) 3. Add action: Update UI — bind a Text widget to `predictionResult` to display the label and confidence Important: Custom Actions using native packages (like tflite_flutter) do not execute in FlutterFlow's web-based Run Mode or Test Mode. You must build an APK (Android) or use the FlutterFlow iOS companion app to test on a real device. To build an APK, click the Build icon in the top-right of FlutterFlow → Build for Android → Download APK, then install it on a physical device. Add a platform guard so the inference action is only triggered on iOS or Android — if a web user somehow triggers it, show a friendly message rather than a crash: ```dart if (!Platform.isAndroid && !Platform.isIOS) { return 'On-device inference is only available on iOS and Android.'; } ``` For a more polished experience, show a CircularProgressIndicator while inference is running and clear it once `predictionResult` is populated.

If you chose the server inference path instead of on-device, this step replaces Steps 2–4. Your TensorFlow Serving instance exposes a REST API at `http://your-host:8501/v1/models/{model_name}:predict`. Because this endpoint is on your infrastructure (not a public service), you route calls through a Cloud Function proxy that keeps the endpoint private and resolves CORS for web builds. Deploy a minimal Cloud Function proxy: ```javascript const functions = require('firebase-functions'); const axios = require('axios'); exports.tfServingProxy = functions.https.onRequest(async (req, res) => { res.set('Access-Control-Allow-Origin', '*'); if (req.method === 'OPTIONS') { res.set('Access-Control-Allow-Methods', 'POST'); res.set('Access-Control-Allow-Headers', 'Content-Type'); return res.status(204).send(''); } try { const response = await axios.post( `${process.env.TF_SERVING_URL}/v1/models/${req.query.model}:predict`, req.body ); return res.json(response.data); } catch (err) { return res.status(err.response?.status || 500).json({ error: err.message }); } }); ``` Set TF_SERVING_URL as an environment variable (e.g. `http://10.128.0.5:8501` for an internal GCP VM). In FlutterFlow, create an API Group 'TensorFlowServing' with the Base URL set to your Cloud Function URL. Add an API Call 'predict' — POST, endpoint `/tfServingProxy?model=your_model_name`. Body: `{"instances": [{{ inputData }}]}` with `inputData` as a JSON variable. In Response & Test, paste a sample Serving response: `{"predictions": [[0.02, 0.97, 0.01]]}`. Add JSON paths: `$.predictions[0]` → `predictionScores`. Bind the result to your UI. For the input: if your model takes structured data (e.g. transaction features), pass a JSON array directly. If it takes images, you'll need to either send base64-encoded pixel data or have your server accept a different input format — check your model's serving signature with `saved_model_cli show --all`.

1const functions = require('firebase-functions');
2const axios = require('axios');
3
4exports.tfServingProxy = functions.https.onRequest(async (req, res) => {
5  res.set('Access-Control-Allow-Origin', '*');
6  if (req.method === 'OPTIONS') {
7    res.set('Access-Control-Allow-Methods', 'POST');
8    res.set('Access-Control-Allow-Headers', 'Content-Type');
9    return res.status(204).send('');
10  }
11  try {
12    const response = await axios.post(
13      `${process.env.TF_SERVING_URL}/v1/models/${req.query.model}:predict`,
14      req.body,
15      { headers: { 'Content-Type': 'application/json' } }
16    );
17    return res.json(response.data);
18  } catch (err) {
19    return res.status(err.response?.status || 500).json({ error: err.message });
20  }
21});